Performance Impact of Visual Filters "rebuilds"

While still on 4.0dB2 yesterday, I got a message that there was a 470 kb update, so I accepted the download.

On restart, Logos4 was "hung" with a partly redrawn screen, and "Not Responding". I was unable to use it of course, as the application was not responding.

I could see the disk was busy, so I decided to wait a little and see from disk I/O if I could get a hint of what might be going on.

Based on the file read/writes, it gave a look not unlike when a reindex is running. Lots of reads from the index, and writes to temp files.

After maybe 10-15 minutes, the window redrew fully and the system was operational, but the CPU and disk continued to run near max and was ongoing. The file read/writes still had the same general look.

Anyway, I discovered I still had a Visual Filter defined for "<af = *>" which I had been playing with the day before. That was created after a search which took about an hour to complete.

At the time of the restart, after the 470 kb update, only 2 or 3 panels were open and none had that visual filter applied.

I deleted the visual filter from the Files menu and my system went back to idle disk and CPU within seconds.

Clearly, Logos4 was doing LOTS OF WORK to rebuild the hitlist or other indexing related work for the visual filter, even though nothing was using it.

My guess is that this background rebuilding of Visual Filter Hitlists is a possible source of the performance issues and "Not Responding" seen by many users of Logos4.

My point of reporting this is NOT so that something like <af = *> runs well (but that would be useful), but to focus on the background reprocessing of visual filter hitlists and related that I expect is impacting performance and the general user-experiences at times.

This all above offered in good faith in the hope its useful.

[ Example below is during a search for "the" in "Entire Library" for information/interest, for anyone wondering about how to watch file I/O on Vista32, and I assume Windows 7 too ]

Find more posts tagged with

Comments

Rosie Perera

This doesn't explain anything else you were seing, but the 470 KB update was a new version of the Help file.

Dave Hooton

Anyway, I discovered I still had a Visual Filter defined for "<af = *>" which I had been playing with the day before. That was created after a search which took about an hour to complete.

Jim, your logos4.log might provide some valuable information. But wildcards like that are very expensive to employ in searches & result in many writes to temp (seen in log file).

JimTowler

https://community.logos.com/discussion/comment/137134#Comment_137134

But wildcards like that are very expensive to employ in searches & result in many writes to temp (seen in log file).

YES - i know they are very expensive to run. It took over an hour, and in the end was useless, because when I clicked to expand one of the results, it started to re-run the query. Dan and others have logged bug reports for the "query rerun" issue when one expands search result.

My point here is not about the slow running of the search, but my system seemed to be rebuilding some kind of "hitlist" or index for the Visual Filter, which nothing was even using.

My thinking is that maybe the filter rebuilds might explain the performance issues some users are seeing, and/or the "Not Responding" on startup of the application.

In my example, I would have expected a "clean" startup (but it was "Not Responding" for 10-15 minutes), and for the Visual Filter to rebuild in the background, and get drawn later if required. If need be, display some indication on or near the "three-circle" visual filters control, to indicate that the highlight/filter results may be incomplete.

Its poor design to have the entire application unworkable, if its possible to instead allow most of the application to function correctly, but with a caution that something might be incomplete. Not unlike how Search warns that some results might be missing.

Dave Hooton

https://community.logos.com/discussion/comment/137143#Comment_137143

My thinking is that maybe the filter rebuilds might explain the performance issues some users are seeing, and/or the "Not Responding" on startup of the application.

A large number of Notes may be another. Whilst I was stating the obvious about the effect of wild card searches/filters it is also obvious they should be the first items to discard if one is seeking possible reasons for slow starts.

Melissa Snyder

https://community.logos.com/discussion/comment/137143#Comment_137143

YES - i know they are very expensive to run. It took over an hour, and in the end was useless, because when I clicked to expand one of the results, it started to re-run the query. Dan and others have logged bug reports for the "query rerun" issue when one expands search result.

My point here is not about the slow running of the search, but my system seemed to be rebuilding some kind of "hitlist" or index for the Visual Filter, which nothing was even using.

There is an open case on this issue, generated from this thread from earlier this year: http://community.logos.com/forums/t/9938.aspx. I'll add a link to this new thread to the case.