Bug (performance problem): 4.0c Beta 2: Search for custom highlighting style took over 21 minutes!

Page 1 of 1 (6 items)
This post has 5 Replies | 0 Followers

Posts 19577
Rosie Perera | Forum Activity | Posted: Sun, Apr 4 2010 1:25 PM | Locked

Searching with wildcards for highlighting is extremely slow.

I set up a custom highlighting style for typos I have already reported, so that I don't waste time reporting them again, and so that I can follow-up on them and see when they've been fixed:

 

 When I search in my Entire Library (3061 resources) using a wildcard (*) for all text marked with that highlighting style...

 

... it takes over 21 minutes:

Log file attached: 1106.SlowSearchWildcard.zip

Posts 3163
Dominick Sela | Forum Activity | Replied: Sun, Apr 4 2010 1:32 PM | Locked

I pasted my info at http://community.logos.com/forums/p/14899/113928.aspx#113928, putting 3 custom highlights into 1 resource and doing a similar each took 9 minutes, and I am not sure the results returned (22 results) were correct.

Posts 27857
Forum MVP
Dave Hooton | Forum Activity | Replied: Sun, Apr 4 2010 6:23 PM | Locked

Rosie Perera:
Searching with wildcards for highlighting is extremely slow

With 1/3 your resources and 6 of 8 cores screaming away on my Desktop it took over 18 minutes to find zero highlights!

I put one typo highlight in one resource and the search took over 7.5 minutes.  When I navigated by Highlight in the locator bar it took less than 1 second!

Dave
===

Windows 11 & Android 8

Posts 19577
Rosie Perera | Forum Activity | Replied: Sun, Apr 4 2010 6:46 PM | Locked

Dave Hooton:

Rosie Perera:
Searching with wildcards for highlighting is extremely slow

With 1/3 your resources and 6 of 8 cores screaming away on my Desktop it took over 18 minutes to find zero highlights!

I put one typo highlight in one resource and the search took over 7.5 minutes.  When I navigated by Highlight in the locator bar it took less than 1 second!

There is clearly a major inefficiency in the code. Highlighting could be quickly accessible through an index that could get built up on the fly as the highlighting is added (not by doing a long "indexing" process). There is no need to search through the entire content of the book, matching the wildcard character with every word, and then checking to see if it's highlighted. That's what it seems like they must be doing.

Posts 27857
Forum MVP
Dave Hooton | Forum Activity | Replied: Sun, Apr 4 2010 7:47 PM | Locked

Rosie Perera:
Highlighting could be quickly accessible through an index that could get built up on the fly as the highlighting is added (not by doing a long "indexing" process)

Agreed. When a book is opened/open there is a mapping of highlights and search results when applicable, as the Navigation options do not otherwise appear.

I tend to avoid highlighting because there is no (efficient) way to identify all  the resources where it is present and erase them.

Dave
===

Windows 11 & Android 8

Posts 19577
Rosie Perera | Forum Activity | Replied: Sun, Apr 4 2010 8:15 PM | Locked

Dave Hooton:

Rosie Perera:
Highlighting could be quickly accessible through an index that could get built up on the fly as the highlighting is added (not by doing a long "indexing" process)

Agreed. When a book is opened/open there is a mapping of highlights and search results when applicable, as the Navigation options do not otherwise appear.

I tend to avoid highlighting because there is no (efficient) way to identify all  the resources where it is present and erase them.

The only reason I'm using it so far is to test it and make Logos improve the performance of searching for it so that I can use it for real someday. So I have random bits of highlighting in several resources. I trust I will someday be able to search for it all efficiently and clean up my resources. But for now I'm leaving it in there as a test case.

Page 1 of 1 (6 items) | RSS