Layouts taking forever to open
Comments
-
Hi Bruce,
In view of Bradley's last post, regarding referral to the desk top team, it would be great if you could keep us all informed on this thread of any resolution to your problem.
Thanks and blessings,
Gary
0 -
(Additionally, the collection filter system was not built anticipating large collection rules like this. I think there are ways we can improve the performance of collections like this and I am creating a case for that.)
Wouldn't it be better to eliminate the need for such large collections rules based on theological/denominational categories, by having FL tag the resources for us?
Since the collection cache is rebuilt each time the application is launched, it would be far more efficient if these types of filter collection rules could be reduced to a single criteria to match a specific theology or denomination.
The Logos Library Theology/Denomination Tags group has over 1300 followers. I imagine there have been suggestions over the years for FL to implement that type of tagging for us!
Perhaps the collection cache could only be rebuilt when out-of-date (such as when the library was updated/indexed), instead of every time the application launches?
Thanks to FL for including Carta and a Hebrew audio bible in Logos 9!
0 -
I'll ask the desktop team to get in contact with you to help track this down.
Thanks!
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Gary Mendenhall said:
In view of Bradley's last post, regarding referral to the desk top team, it would be great if you could keep us all informed on this thread of any resolution to your problem.
I will do Gary. I hope we can all learn from it.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
PetahChristian said:
Wouldn't it be better to eliminate the need for such large collections rules based on theological/denominational categories, by having FL tag the resources for us?
Since the collection cache is rebuilt each time the application is launched, it would be far more efficient if these types of filter collection rules could be reduced to a single criteria to match a specific theology or denomination.
The Logos Library Theology/Denomination Tags group has over 1300 followers. I imagine there have been suggestions over the years for FL to implement that type of tagging for us!
This has been suggested before but perhaps there is more reason to do so now, if indeed collections are responsible. But it makes me wonder...if there are 1300 others using these collection rules why would more people not be experiencing the same slowdowns I have experienced? If anyone else has these collections could you let us know whether you have experienced any performance issues as a result?
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Bruce Dunning said:
Okay, now I'm jealous. I just tried my two most popular layouts. The most complex one still took 2 1/2 minutes and the other one took 1 1/2 minute.
If you upload logs, we'll be able to see a little of where the delay is coming from. If you've got an i5 and an SSD, anything over 30 seconds would point to either an error, or a VERY inefficient visual filter IMO.
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
PetahChristian said:
Wouldn't it be better to eliminate the need for such large collections rules based on theological/denominational categories, by having FL tag the resources for us?
Since the collection cache is rebuilt each time the application is launched, it would be far more efficient if these types of filter collection rules could be reduced to a single criteria to match a specific theology or denomination.
In the meantime, you can tag the resources yourself.
Paste the rule into the Library filter box, select all the resources, and apply a tag to all of them. Then change your collection rule to just look for that tag.
(You'd have to redo this after every new purchase of resources that might be included in these rules.)
0 -
Mark Barnes said:
If you upload logs, we'll be able to see a little of where the delay is coming from. If you've got an i5 and an SSD, anything over 30 seconds would point to either an error, or a VERY inefficient visual filter IMO.
I uploaded logs last night which is shown on page one of this thread but here is a copy for convenience - 5074.Logos.log
Using adventure and community to challenge young people to continually say "yes" to God
0 -
PetahChristian said:
Perhaps the collection cache could only be rebuilt when out-of-date (such as when the library was updated/indexed), instead of every time the application launches?
Metadata often gets updated on application launch, which would invalidate the collection cache.
However, the collection cache doesn't wait for metadata (inc. community metadata) to finish updating before it begins. That means you can end up with the collection cache being updated two or three times, as it's constantly being invalidated by the new metadata.
Fixing that wouldn't make a huge difference, but it would help on some occasions, I think. For example, in the log below, the collection cache is updated three times on startup (taking 13.26s, 11.74s and 11.62s on each occasion). The second one is begun right in the middle of a bunch of community metadata updates, which seems particularly wasteful.
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
In the meantime, you can tag the resources yourself.
Paste the rule into the Library filter box, select all the resources, and apply a tag to all of them. Then change your collection rule to just look for that tag.
That is one option but....
(You'd have to redo this after every new purchase of resources that might be included in these rules.)
This is the exact reason why it is not an attractive option.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Bruce Dunning said:
I uploaded logs last night which is shown on page one of this thread but here is a copy for convenience - 5074.Logos.log
Sorry I missed that.
- Your biggest problem is your layouts database file. That's horribly slow, and it's being run multiple times.
- shortcuts.db is also very bad. One query on that database took 22.5s (admittedly in parallel with other queries), but that should be a tiny file that's super-quick. However, this is accessed less often, so it has less overall impact.
- Your collections are also dragging things down, and they were generated three times during this startup (see my post above). They take a minute to update each time. I'm not sure you can blame Logos for that when you have 572 collections! I would imagine that a large number of those you don't use regularly, and could just recreate using quick collections if/when you need them. Many of them have errors in or are unnecessarily complex. For example, Commentary AND (Subject:N.T.Hebrews,Title:"Horae Homileticae Vol. 19") OR type:”bible commentary” AND (title:hebrews OR subject:hebrews), could be re-written simply as type:bible-commentary subject:"N. T. Hebrews".
I don't really understand why layouts.db is so slow. If you uploaded it (and shorcuts.db), I could take a look. You'll need to zip them first if you want to do that.
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
Incidentally, your most expensive collections are:
- mytag:CommentaryHistorical
- Author:("Alden, Robert L.", "Allen, Desmond", "Baptist Bible College", "Beale, David", "Bl…
- type:bible-commentary series:("A Commentary on the Book of Genesis", "An Exegetical Commen…
- type:bible-commentary (series:("A Bible Commentary in the Wesleyan Tradition", "An Exposit…
- Author:("Abendroth, Mike", "Abernathy, David", "Adams, W. Milton", "Addington, Gordon L.",…
- Author:("Bates, William", "Baxter, Richard", "Beeke, Joel R.", "Brook, Benjamin", "Brooks,…
- Author:("Abraham, William J.", "Albers, Robert H.", "Albright, W. F.", "Allen, O. Wesley",…
- type:bible-commentary series:("A Library of Fathers of the Holy Catholic Church", "Ancient…
- Author:(“Acklin, Thomas”, “Althann, Robert”, “Armstrong, Dave”, “Attridge, Harold W.”, “Ba…
- Author:("Adams, W. Milton", "Advent Source", "Anderson, Bruce", "Andrews University", "Bac…
- Author:(“Baxter,Richard”, “Brook,Benjamin”, “Brooks,Thomas”, “Bunyan,John”, “Burroughs,Jer…
- Author:("à Brakel, Wilhelmus", "Abernathy, David", "Achtemeier, Elizabeth", "Achtemeier, P…
- Author:(“Stevens, W. C.”, “Tozer, A. W.”)
- Author:("Reformed Episcopal Church", "Whitacre, Rodney A.")
- Author:("Abbott, Edwin A.", "Abbott, T. K.", "Abbott-Smith, G.", "Ackroyd, Peter R.", "Ada…
- Author:("Aageson, James W.", "Abbe, Paul", "Abbott, Edwin A.", "Abbott, T. K.", "Abraham, …
- Author:(“Albright, W. F.”, “Anderson, Cheryl”, “Banks, J. S.”, “Barker, Margaret”, “Barret…
- Author:(“Freeman, James”, “Mace, Daniel”, “Palfrey, J. G.”, “Parker, Theodore”, “Sawyer, L…
- mytag:free ANDNOT mytag:perseus
- Author:("Adar, Zvi", "Bamberger, Bernard J.", "Bar-Efrat, Shimon", "Benisch, Abraham", "Be…
- title:"moody handbook of theology" OR title:"Compendium of Christian Theology" OR title:".…
- Author:(“Anders, Max”, “Anderson, Kenton C.”, “Ankerberg, John”, “Archer, Clint”, “Baban, …
- Author:("Baptist Bible College", "Central Baptist Theological Seminary", "Central Conserva…
- Author:(“Anderson, Neil T.”, “Baker, Heidi”, “Baker, Rolland”, “Batterson, Mark”, “Bennett…
- Author:("Abraham, William J.", "Achtemeier, Elizabeth", "Achtemeier, Paul J.", "Adamson, J…
- Author:(“Andrew of Crete”, “Augustine of Hippo”, “Basil the Great”, “Benedict of Nursia”, …
- Author:("Abraham, William J.", "Albers, Robert H.", "Albright, W. F.", "Allen, O. Wesley",…
- Encyclopedia AND subject:(Bible,Christian,History,Religion,Theology)
- Author:("Abelard, Peter", "Alighieri, Dante", "Anselm of Canterbury", "Anthony of Padua", …
This seems to suggest Logos is struggling with denominational collections (understandable because they're very complex), with collections based on tags (perhaps understandable if you have thousands or tens of thousands of tags), plus a few others that don't make much sense.
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
Mark Barnes said:
- Your biggest problem is your layouts database file. That's horribly slow, and it's being run multiple times.
- shortcuts.db is also very bad. One query on that database took 22.5s (admittedly in parallel with other queries), but that should be a tiny file that's super-quick. However, this is accessed less often, so it has less overall impact.
Thanks for taking time to look at this. I've known that it was slow for a long time but have just been putting off looking into it until now.
Mark Barnes said:Your collections are also dragging things down, and they were generated three times during this startup (see my post above). They take a minute to update each time. I'm not sure you can blame Logos for that when you have 572 collections! I would imagine that a large number of those you don't use regularly, and could just recreate using quick collections if/when you need them. Many of them have errors in or are unnecessarily complex. For example, Commentary AND (Subject:N.T.Hebrews,Title:"Horae Homileticae Vol. 19") OR type:”bible commentary” AND (title:hebrews OR subject:hebrews), could be re-written simply as type:bible-commentary subject:"N. T. Hebrews".
I'm certainly not blaming Faithlife for the number of collections I have. Many of the collections that I had were generated from joining Faithlife groups and I have also been putting off cleaning them up. Last night I did and now only have 140. 67 of these are composed of Theology/Denominations Tags in Faithlife and the others are not easy to create on the fly. I only have 10 Bible commentary collections
Mark Barnes said:I don't really understand why layouts.db is so slow. If you uploaded it (and shorcuts.db), I could take a look. You'll need to zip them first if you want to do that.
I'll try to do this later when I have a few minutes.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Is the shortcuts.db related to the shortcut icons on the top of the screen? I feel adding or deleting those icons is very slow, maybe some inherent problem in it? I confess I have many shortcut icons.
Gold package, and original language material and ancient text material, SIL and UBS books, discourse Hebrew OT and Greek NT. PC with Windows 11
0 -
Veli Voipio said:
Is the shortcuts.db related to the shortcut icons on the top of the screen? I feel adding or deleting those icons is very slow, maybe some inherent problem in it? I confess I have many shortcut icons.
That's correct. Personally, I've noticed no slowness, but Bruce certainly seems to have an issue.
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
Mark Barnes said:
I don't really understand why layouts.db is so slow. If you uploaded it (and shorcuts.db), I could take a look. You'll need to zip them first if you want to do that.
Mark, I don't think this is possible. My zipped layouts.db file is 87.5 MB so it won't upload, unless I'm doing something wrong. Suggestions?
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Mark Barnes said:
This seems to suggest Logos is struggling with denominational collections (understandable because they're very complex), with collections based on tags (perhaps understandable if you have thousands or tens of thousands of tags), plus a few others that don't make much sense.
For sure this is an issue but I think I may be experiencing something else as well since removing two of the larger ones as Bradley suggested didn't help at all.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
Mark Barnes said:
I just uploaded them to your dropbox. Thanks for taking a look at them Mark as this is certainly not my forte.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
PetahChristian said:
(Additionally, the collection filter system was not built anticipating large collection rules like this. I think there are ways we can improve the performance of collections like this and I am creating a case for that.)
Wouldn't it be better to eliminate the need for such large collections rules based on theological/denominational categories, by having FL tag the resources for us?
Since the collection cache is rebuilt each time the application is launched, it would be far more efficient if these types of filter collection rules could be reduced to a single criteria to match a specific theology or denomination.
The Logos Library Theology/Denomination Tags group has over 1300 followers. I imagine there have been suggestions over the years for FL to implement that type of tagging for us!
Perhaps the collection cache could only be rebuilt when out-of-date (such as when the library was updated/indexed), instead of every time the application launches?
This suggestion (https://logos.uservoice.com/forums/42823-logos-bible-software-7/suggestions/9285627-library-hierarchy-and-filtering) isn't exactly the same, but the way I understand it, it incorporates this suggestion, so people reading this thread may want to consider voting for it. I think each author could have their denomination saved within the database and remove a great deal of the problem. Each author (with perhaps one exception out of over 6000 authors currently classified) has just one denomination stream and one denomination. There are only a small number of other collections within the group mentioned.
0 -
(Additionally, the collection filter system was not built anticipating large collection rules like this. I think there are ways we can improve the performance of collections like this and I am creating a case for that.)
Bradley, I'd love to engage with you on this if you think it appropriate, given that I'm the owner of the group and have created most of the data. I used to be an analyst/programmer working with SQL and databases.
0 -
At the moment, we're still mostly guessing at the cause of the performance issue. Without a detailed performance trace from your computer, we won't really know what's causing the slowdown.
Bradley,
Bruce's layouts.db file has 1,626 rows, of which 1,455 rows are for the "00 - Bible Reading Layout", some of which go back to 2013. I'm fairly sure that's the major cause of the slowdown in Bruce's case. Couldn't these old layouts be trimmed after a certain period?
I also noticed the IsDeleted column doesn't have an index. There might be a small improvement if it did.
This is my personal Faithlife account. On 1 March 2022, I started working for Faithlife, and have a new 'official' user account. Posts on this account shouldn't be taken as official Faithlife views!
0 -
Bruce Dunning said:
My SSD is just over a year old and is a Samsung 840 EVO 500GB.
This may or may not be part of the problem here, but I know this model of SSD saw some performance degradation over a relatively short amount of time and was addressed by Samsung. If you google around you can find more about it.
0 -
Andrew Baguley said:
(Additionally, the collection filter system was not built anticipating large collection rules like this. I think there are ways we can improve the performance of collections like this and I am creating a case for that.)
Bradley, I'd love to engage with you on this if you think it appropriate, given that I'm the owner of the group and have created most of the data. I used to be an analyst/programmer working with SQL and databases.
Please create a new thread and we can discuss there. (I may not be able to reply in depth until early next week.)
0 -
Tommy Ball said:Bruce Dunning said:
My SSD is just over a year old and is a Samsung 840 EVO 500GB.
This may or may not be part of the problem here, but I know this model of SSD saw some performance degradation over a relatively short amount of time and was addressed by Samsung. If you google around you can find more about it.
Thanks for bringing this to my attention although I don't like the implications. Over the past 4-5 years I've had to replace 3 SSDs so I hope I don't have to replace this one too. I will do some more research about this. Thanks again.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Bruce Dunning said:
Thanks for bringing this to my attention although I don't like the implications. Over the past 4-5 years I've had to replace 3 SSDs so I hope I don't have to replace this one too. I will do some more research about this. Thanks again.
That is an awful lot of SSDs for that period of time. I wonder if it may be better to just go for or stay with 7200rpm drives. Most of my drives outlast the general architecture of the machine, and that is with heavy use and barely no air conditioning.
0 -
Erwin Stull, Sr. said:
That is an awful lot of SSDs for that period of time. I wonder if it may be better to just go for or stay with 7200rpm drives. Most of my drives outlast the general architecture of the machine, and that is with heavy use and barely no air conditioning.
Yes I was disappointed with the life of these drives but I could not think of using Logos without a SSD as I would spend all of my time waiting for it to index etc. which is what it used to be like.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
PetahChristian said:
Wouldn't it be better to eliminate the need for such large collections rules based on theological/denominational categories, by having FL tag the resources for us?
Since the collection cache is rebuilt each time the application is launched, it would be far more efficient if these types of filter collection rules could be reduced to a single criteria to match a specific theology or denomination.
In the meantime, you can tag the resources yourself.
Paste the rule into the Library filter box, select all the resources, and apply a tag to all of them. Then change your collection rule to just look for that tag.
(You'd have to redo this after every new purchase of resources that might be included in these rules.)
I thought I'd give an update on things from my perspective. Just to be sure that collections were not the cause of the problems I've experienced loading layouts I went through all of my collections, transferred them into tags and then deleted all of my larger collection rules. Then I restarted Logos and it still took 3 minutes and 40 seconds to load my Bible Reading layout. Indeed something more is going on.
Using adventure and community to challenge young people to continually say "yes" to God
0 -
Bruce Dunning said:
Indeed something more is going on.
Have you updated the firmware and Magician as per http://www.anandtech.com/show/9196/samsung-releases-second-840-evo-fix ?
Dave
===Windows 11 & Android 13
0 -
Bruce Dunning said:Erwin Stull, Sr. said:
That is an awful lot of SSDs for that period of time. I wonder if it may be better to just go for or stay with 7200rpm drives. Most of my drives outlast the general architecture of the machine, and that is with heavy use and barely no air conditioning.
Yes I was disappointed with the life of these drives but I could not think of using Logos without a SSD as I would spend all of my time waiting for it to index etc. which is what it used to be like.
I am so used to the indexing time being what it is, that I forgot to consider it. :-)
Once I go with an SSD (and I will at some point in time), I probably wouldn't be able to go back to standard drives either.
0
