A plea for speeding up the "select resource" drop down in search

Mark Barnes said:

When I'm conducting a search, it currently takes between 10.3s (cached) and 18.5s (uncached) to populate the dropdown menu

This takes about 1s on my system. (My library shows 4561 resources.)

Just wondering: does your anti-virus do active scanning of files before opening? I set the entire Logos4 directory (with subdirectories)  in my 'Exclusions' list in Avast for its "Real Time Scanning."

Mark Barnes said:

Can I ask what happens to Logos' memory usage when you click that menu? Mine goes up an additional 380Mb every time I click on it (the memory is released when I execute a search).

It's kind of hard to tell, because it goes so fast, but it doesn't look like the Private jumps more than about 10MB, and the Commit much more than 20GB. My entire Private RAM usage on the Resource Monitor in Win7 is 381GB, and in Commit is 458GB (both with L4 idle), Working Set is at about 485GB.

My physical memory is at about 65% (of 6GB RAM), because I have a lot running at the same time at the moment.

Are you running any guides, or anything else that might be using the library cache? (Another wild guess.)

I reran this a few times and noticed the only time the delay comes up is the first time I open the Search dialog box. after opening Logos After that caching seems to occur and the drop-down list populates almost instantaneously. Sometimes upon opening Logos 4 I have almost no delay in the list being populated from the start. There must be something in the background causing the delay when I notice it. I did not monitor what else was going on in the background.

Rosie Perera said:

Disk hits are going through the roof during this time. Why?

I saw this and went to check on my system. I saw a similar amount of disk hits when I was doing nothing in Logos (just booted it up and literally didn't click on anything. So I decided to close Firefox and Thunderbird (the only apps I have running - except for background/tray apps like anti-virus, etc.). Disk hits went way down.

Then I ran a search. There was some disk activity, but not much. Activating the Logos window (by clicking on it to make it the foreground app), I saw a huge jump in network activity. I let that die down and clicked on the search icon and the on the "Entire Library" text. There was minimal disk activity; some but nothing like what you're showing.

Getting back to another theory mentioned, how many collections do you have? (I just counted 48 in my system)

Also how many tags? (Sorting my Library by tags, I count 26)

Rosie Perera said:

Being hemmed in by poor performance, and then getting the answer "well what did you expect? you had 150 collections" would leave one somewhat underwhelmed.

I get that.

However, knowing what/where the problem is would be the first step in fixing it. Right?

Rosie Perera said:

Mark Barnes said:

The only thing I can think that's different is all those PB Bibles I have.

I have one PB Bible on this machine. I wonder if that has anything to do with it.

I do not have any PB Bibles. (If that helps.)

EDIT: I believe the PB's have their own index (right?). If so, can that index be rebuilt (and so optimized)?

Bradley Grainger said:

Rosie Perera said:

I'd be happy to rebuild my PB Bible once this is fixed.

The bug affects any resource with milestones (including page numbers); unfortunately, you'll probably need to rebuild every affected resource to completely fix this performance problem. (Without profiling, it's hard for me to know exactly what areas of the app will be affected, and by how much, but it would probably have a negative impact on many components that query the Library Catalog.)

Wouldn't it be helpful then to have a command like rebuilt all PB that batch-processes these and could run by itself overnight for those with a large number of large/complex PBs? 

EDIT: if you should consider building this command, you may not want to copy my typo - be free to call it "rebuild all PB".... 

MJ. Smith said:

Bradley Grainger said:

It looks like you have a lot (150+) of collections.

150 doesn't sound like a lot to me. Given that metadata is not very reliable and that the Guide encourage small discrete units because of their sort functions, I would expect LOTS of collections.

Looking in ResourceCollectionManager.db file, noticed table ResourceCollections has 183 rows.

In the Collections Tool, wonder about adding a check box: "Show in Search Menu" ? (similar to "Show in parallel resources")

The Cited By tool has check boxes to filter collections, series, mytags, and ratings for Cited By use.  Wonder if Cited By collection list could be used for search menu list when choosing a resource ? (perhaps in same order as Cited By show collections)

Keep Smiling [:)]

MJ. Smith said:

Bradley Grainger said:

It looks like you have a lot (150+) of collections.

150 doesn't sound like a lot to me. Given that metadata is not very reliable and that the Guide encourage small discrete units because of their sort functions, I would expect LOTS of collections.

150 isn't "a lot" - I have around 110 and I'm not using Logos professionally. Actually I wasn't expecting the collections to be a performance burden.

We are told in threads and on the wiki to build these in order to be able to manage the thousands of ressources in our libraries. 

So, for commentaries alone, I have a collection for every biblical book, one for one-volume commentaries per testament, one for study bibles, one for all commentaries, then around ten for different levels of commentaries (technical, devotional etc). We're at eigthy now. I have several for the various periods of greek texts, then some to manage the Perseus and the Vyrso ressources.

Of course, Bradley, if you tell us that we shouldn't have more than a dozen collections (ten for weaker machines, twenty for the "mongo" ones), then I can act accordingly. I don't need a collection for "Judges" or "Greek texts 3rd - 5th century" today - but maybe next week or next year. But please in this case, give us the chance to set existing collections as active/inactive, so we don't need to invent/copy and taylor the dynamic rules every time.