When I'm conducting a search, it currently takes between 10.3s (cached) and 18.5s (uncached) to populate the dropdown menu where I can specify which resources I want to search in. The menu lists Open Resources, Collections, Tags, Ratings, and Series, but the code is clearly also reading in every book from my library to make the filtering quicker.
Looking at ProcessMonitor, pretty much all the time is spent querying catalog.db, but:
Mark Barnes:When I'm conducting a search, it currently takes between 10.3s (cached) and 18.5s (uncached) to populate the dropdown menu
This takes about 1s on my system. (My library shows 4561 resources.)
Just wondering: does your anti-virus do active scanning of files before opening? I set the entire Logos4 directory (with subdirectories) in my 'Exclusions' list in Avast for its "Real Time Scanning."
Help links: WIKI; Logos 6 FAQ. (Phil. 2:14, NIV)
Logos User Voice suggestion => Improve Logos 4 Menu Responsiveness
Keep Smiling
Logos Wiki Logos 7 Beta Free Support
Richard DeRuiter:This takes about 1s on my system. (My library shows 4561 resources.)
Wow. I have a few more (6,622), but that doesn't explain the differences. Maybe there's some kind of bug where 5,000 or 6,000 resources pushes it over the edge and it gets horribly slow.
Can I ask what happens to Logos' memory usage when you click that menu? Mine goes up an additional 380Mb every time I click on it (the memory is released when I execute a search).
Mark Barnes:When I'm conducting a search, it currently takes between 10.3s (cached) and 18.5s (uncached) to populate the dropdown menu where I can specify which resources I want to search in.
I've noticed this, too. When doing a Bible search it takes about 16 seconds to populate the list so I can choose a Bible. I don't know what can be done, but perhaps cache the last list generated and only reset the cache if I change collections or add resources. Something, at any rate to make this almost instantaneous.
Pastor, North Park Baptist Church
Bridgeport, CT USA
Mark Barnes:Can I ask what happens to Logos' memory usage when you click that menu? Mine goes up an additional 380Mb every time I click on it (the memory is released when I execute a search).
It's kind of hard to tell, because it goes so fast, but it doesn't look like the Private jumps more than about 10MB, and the Commit much more than 20GB. My entire Private RAM usage on the Resource Monitor in Win7 is 381GB, and in Commit is 458GB (both with L4 idle), Working Set is at about 485GB.
My physical memory is at about 65% (of 6GB RAM), because I have a lot running at the same time at the moment.
Are you running any guides, or anything else that might be using the library cache? (Another wild guess.)
Mark Barnes: When I'm conducting a search, it currently takes between 10.3s (cached) and 18.5s (uncached) to populate the dropdown menu where I can specify which resources I want to search in.
When I'm conducting a search, it currently takes between 10.3s (cached) and 18.5s (uncached) to populate the dropdown menu where I can specify which resources I want to search in.
I'm not able to reproduce this delay (on a test system with thousands of resources and several hundred custom collections).
Are you in Basic, Bible, or Morph Search? (Roughly) How many collections, unique tags, open resources do you have?
I reran this a few times and noticed the only time the delay comes up is the first time I open the Search dialog box. after opening Logos After that caching seems to occur and the drop-down list populates almost instantaneously. Sometimes upon opening Logos 4 I have almost no delay in the list being populated from the start. There must be something in the background causing the delay when I notice it. I did not monitor what else was going on in the background.
Mark my experience is about the same as Richard. I wonder if it has more to do with the number of collections and tags you have.
Logging Tool
I've noticed this recently too. Here's a log file from 4.5a Beta2 when it took 13 seconds just to populate the list of Bibles in a new Bible Search tab: 800067.Logos4.log
Disk hits are going through the roof during this time. Why? Shouldn't this list be created and cached for future use shortly after boot-up during some idle time, as it's one of the most likely-to-be used lists in Logos, and isn't likely to change often.
How to Ask for Help | Logos Wiki | My Machine Specs | My Blog
Bradley Grainger: I'm not able to reproduce this delay (on a test system with thousands of resources and several hundred custom collections). Are you in Basic, Bible, or Morph Search? (Roughly) How many collections, unique tags, open resources do you have?
When I click on the drop-down menu I experience the following:
Thanks for looking into this. It really is quite painful at the moment.
Rosie Perera:Disk hits are going through the roof during this time. Why?
I saw this and went to check on my system. I saw a similar amount of disk hits when I was doing nothing in Logos (just booted it up and literally didn't click on anything. So I decided to close Firefox and Thunderbird (the only apps I have running - except for background/tray apps like anti-virus, etc.). Disk hits went way down.
Then I ran a search. There was some disk activity, but not much. Activating the Logos window (by clicking on it to make it the foreground app), I saw a huge jump in network activity. I let that die down and clicked on the search icon and the on the "Entire Library" text. There was minimal disk activity; some but nothing like what you're showing.
Getting back to another theory mentioned, how many collections do you have? (I just counted 48 in my system)
Also how many tags? (Sorting my Library by tags, I count 26)
Richard DeRuiter: Getting back to another theory mentioned, how many collections do you have? (I just counted 48 in my system) Also how many tags? (Sorting my Library by tags, I count 26)
Lots (150+) and lots (750+). But I expect whatever algorithm Logos is using to be scalable (or is it scaleable?) to large datasets. If you give people this powerful system, they are going to want to use it to its full capacity. Being hemmed in by poor performance, and then getting the answer "well what did you expect? you had 150 collections" would leave one somewhat underwhelmed.
I seem to recall Bradley and I had a conversation about the number of collections I have at some other time, and he was able, nonetheless, to speed up some performance issue I was complaining about at the time (which may very well have been this one -- it seems awfully familiar) to the point where it was a non-issue for me. So either this is a regression, or I'm losing my memory and it was some other problem.
What happens to your RAM when you click on the dropdown, Rosie?
Mark Barnes: What happens to your RAM when you click on the dropdown, Rosie?
RAM usage doesn't change much:
Before clicking the dropdown:
While the dropdown is populating (a couple of seconds after clicking the dropdown):
BTW, dropping down this menu (the Bibles selection menu in Bible search) is also slow (10-13 sec) even on second, third, and subsequent drop-downs. You'd really think the contents of that list would be cached by then.
Rosie Perera: Being hemmed in by poor performance, and then getting the answer "well what did you expect? you had 150 collections" would leave one somewhat underwhelmed.
I get that.
However, knowing what/where the problem is would be the first step in fixing it. Right?
Richard DeRuiter:However, knowing what/where the problem is would be the first step in fixing it. Right?
Yes, that's true.
Very interesting. I just tried this on my other desktop (similar specs to this one), and have a near instantaneous response. So there's something particular on this machine that's causing the problem. The only thing I can think that's different is all those PB Bibles I have.
Mark Barnes:The only thing I can think that's different is all those PB Bibles I have.
I have one PB Bible on this machine. I wonder if that has anything to do with it.
Rosie Perera: Mark Barnes:The only thing I can think that's different is all those PB Bibles I have. I have one PB Bible on this machine. I wonder if that has anything to do with it.
I do not have any PB Bibles. (If that helps.)
EDIT: I believe the PB's have their own index (right?). If so, can that index be rebuilt (and so optimized)?