This Happened, Even Though Everything Is Working as Designed

Logos, for one, I need your help to prevent a recurrence.

I lead a Bible study that exists ONLY within Logos 5. The outline is in Notes. The materials are all Logos resources. The outline is mostly hotlinks from the outline to the resources.

Yesterday, in the 1-hour prep time before class, the perfect storm happened. The impact? We were NOT able to use Logos 5. When Logos finally became available about 1hr after the start time, we were close to being finished with an impromptu study of Ephesians using Pradis----ugh!---, the only Bible software I could open. we decided to finish the Ephesians passage rather than have TWO incomplete Bible studies.

How could it happen that I couldn't open L5 for nearly an hour?

Logos released a 500+ Mb download to update . Once I realized it had started, I stopped it. By this point, nearly 400Mb had already downloaded.
Microsoft's Patch Tuesday was yesterday. This meant that 13 or so patches from Microsoft wanted installation. Since this was offered nearly an hour before class, I let it go ahead.
MS patches required a reboot, just before class was due to start. Let it reboot & restarted Logos.
Logos started "preparing the Library" for those books it ALREADY downloaded BEFORE I stopped the download & began downloading more.
Time for class. I couldn't get past the "preparing the library" message with the spinning pizza. Nothing I tried would get past it. But here's what I tried:

Since class is beyond range of office wireless, close L5, check box to open to blank layout, & click Work Offline.
Since class is beyond range of office wireless, leave class & go back to office, THEN restart L5 & sign in.
Wait in office in case it's related to sync.
Each time (while connected), stop some portion of the remaining 100Mb download. But each time, it'd finish some resource that it added to the stack of indexing at the next start.

I hope Logos folks are as embarrassed that this COULD happen in front of a group of customers as I was that it DID happen.

If I got a vote, there'd be some way to bring Logos up where we could reach the resources IMMEDIATELY even if it was a little slow while indexing.

Speaking of slow while indexing, why is it done at such a high priority that it's able to consume 100% of my disk drive for intervals of up to 5 minutes at a time? EACH time it gets to that point in the index cycle, my entire PC becomes useless, as indexing ends up at a higher priority than my use of the cursor (touchpad)!

This all said, L5 is a GREAT product. But I sure need some help on the related impacts of big downloads & indexing. If you'll notice, my machine isn't slow, even if isn't the fastest one on the block. I can only imagine how bad this would've been on a slower machine (like my old core 2 duo).

Find more posts tagged with

Comments

Lee

I have three reflections on this:

1. In the process of preparing library or index or whatever magic sauce, keep on using the old version until the new version is 100% ready. [naturally there would be the added complication of syncing changes and operations during the indexing window]

2. Choke indexing to 80% CPU and/or move thread priority down.

3. Concentrate, really concentrate, on graceful exit conditions or notexists.

Lo and behold, we're looking at Logos 6.

William Gabriel

Logos seems to still be on a huge learning curve when it comes to UI.

There is way, way too much background processing that interferes with actually using the program.

Logos, please hire a real UI expert to make changes for version 6. It does not suffice to use MSFT's UI model from over a decade ago in a modern program. It's frustrating for us users. I know that Logos is a very serious program doing very serious work and it makes a lot of computations. That's exactly why you have to get the UI part right.

Doc B

why is it done at such a high priority that it's able to consume 100% of my disk drive for intervals of up to 5 minutes at a time? EACH time it gets to that point in the index cycle, my entire PC becomes useless

Bill, I've wondered about this myself. I'm a big fan of Logos, like you. I understand (and agree with) the need to index rather than take longer times each time some kind of search is requested. What I haven't figured out is, why does it take up 100% of the available resources for indexing?

In another thread yesterday, I timed an index cycle (had to do with the big Perseus update which I mostly avoided because I'd hidden many of those resources). It took exactly 30 minutes and 2 seconds to index. I have 2284 resources right now. I'm fine with all that.

But during that 30 minutes, my computer was completely unusable. I can't even get the Ctrl-Alt-Del Windows Task Mangler to come up when Logos is indexing. Now that's a resource hog of biblical proportions (pun intended).

I'm not mad at the Logos folks for this, and I'm not whining about it, but I do hope those good folks are paying attention to the need to mitigate the amount of resources used by a machine to complete the indexing cycle in the next release. That would be a more valuable bit of customer service than any number of new resources that we've been requesting.

[Edit: I want to be clear that I'm not being critical, and I'm not sure I was clear.

I suspect the folks at Logos know about these desires to improve the UI (as another poster calls it). I also suspect they care and have worked on improving it. Those folks didn't get where they are by guessing and misunderstanding the market. But I don't think it is harmful for we, the customer base, to remind them every once and a while, nicely, that this is important to us. Not important because we think we deserve a perfect product or would demand something that can't be produced, but important in the sense that we are part of a team, working together to spread the Gospel, all in our own little corners of the world.]

Mark Smith

If I got a vote, there'd be some way to bring Logos up where we could reach the resources IMMEDIATELY even if it was a little slow while indexing.

A very good suggestion. I don't know why what you tried wouldn't work however. A Panic Button would be nice.

I hope Logos folks are as embarrassed that this COULD happen in front of a group of customers as I was that it DID happen.

Most would probably understand software is fallible. But even so, I would have been fit to be tied. Feel for you, Bill.

DMB

Not arguing with Bill's frustration, but there must be more to the story inside Logos?

My indexing is pretty transparent. I have a huge Logos footprint (1g) plus running a virtual box, plus ArtRage. It does its indexing but no apparent impact on my other programs.

BillS

Not arguing with Bill's frustration, but there must be more to the story inside Logos?

My indexing is pretty transparent. I have a huge Logos footprint (1g) plus running a virtual box, plus ArtRage. It does its indexing but no apparent impact on my other programs.

Hi Denise,

I'm no longer frustrated, though I was pretty embarrassed yesterday. I waited until today to post so it could all be constructive feedback, as detailed as I could make it. I'd LOVE to find that something about your setup can be replicated for the rest of us.

For example, if running in a virtual box means you've found a way to LOWER the priority Logos gets so that it CAN'T negatively impact you, that confirms what I'd hoped: that Logos should be able to request a lower priority for its tasks. To me, that'd be an ideal solution to the 2nd issue. The 1st issue of immediate access to the resources is tied up in the "preparing resources" process, & I'm still hoping we'll hear from Logos on that something can be done.

I've been an evangelist for Logos for a long time, & this is the 1st time I've ever had to go on without Logos. I can see it happening to other users, too, as libraries get larger, too.

I'm at 5.7k resources...

Jack Caviness

Strange, with two different desktop machine and a MacBook Pro, I have never had Logos indexing render my computer inoperable. I do not doubt that it happens to other, as there have been multiple complaints reports of that on these forums.

Just out of curiosity, it this a Windows only experience or has it happened with the Mac version also?

BillS

Just out of curiosity, it this a Windows only experience or has it happened with the Mac version also?

That's just what I was starting to wonder......

fgh

Just out of curiosity, it this a Windows only experience or has it happened with the Mac version also?

You must have read fewer of my posts over the years than I thought...

DMB

Bill ... my miscommunicating ... I was just saying the Logos indexing doesn't impact the VirtualBox which also draws a lot of CPU power.

I looked at your stats. All that's different is I have L5 running on an SSD. But I wouldn't think that's going to impact the CPU priority.

I notice also you're on W8? But others having issues are also on W7 (like me but no issues).

I DO throttle the CPU back on battery (how I run). Maybe a throttled CPU forces Logos to lower its indexing priority?

BillS

I DO throttle the CPU back on battery (how I run). Maybe a throttled CPU forces Logos to lower its indexing priority?

Hi Denise,

Interesting... does it do anything to shorten how long L5 spends in the "preparing resources" phase?

And when this laptop comes out of warranty, I'll crack the case & replace its h/d with an SSD... But until it's out of wtty, I don't want to crack the case.

Thanks for the response....

Dave Hooton

How could it happen that I couldn't open L5 for nearly an hour?

Logos released a 500+ Mb download to update . Once I realized it had started, I stopped it. By this point, nearly 400Mb had already downloaded

Bill, I'm sorry but your situation was avoidable.

You can control a Logos download in the same way you can control a Windows Update i.e. elect to receive a notification but decide when/if you will accept it! In Program Settings have Automatically Download Updates = NO. If you also have Show the Notification Bar = NO the only alert will be a glowing yellow Notification icon near the Layouts menu (much less obtrusive if you are projecting the screen for an audience).

Speaking of slow while indexing, why is it done at such a high priority that it's able to consume 100% of my disk drive for intervals of up to 5 minutes at a time?

It's threads have a low priority so it is interruptable when you want to run a task, otherwise it will consume all CPUs of a double/quad core processor. It was pleasantly easy to run another task on my dual core Atom powered Tablet (full Win 8), but i did find it harder on the desktop i7.

Mark Barnes

There's a lot more going on when 'indexing', than just indexing. That's why there's multiple stages:

Before indexing, new resources are downloaded and placed in the downloaded folder.
Then there's the Preparing Your Library stage:

Stage 1 of 'preparing your library' is the discovery mode. It scans through the downloads, compares them to existing resources, and moves them to the correct location. It's pretty quick, just 1 or 2s per resource.
Stage 2 of 'preparing your library' updates the library catalog. This makes sure all the data in your library is up to date with the new resources, and copies in the thumbnails and other metadata. This takes 1-2s per resource, plus another 5-30s depending on the size of your library.
Stage 3 of 'preparing your library' create the keylink databases. This is your prioritisations, so creating this database is crucial for Logos to be able to access your resources from Guides, Biblical Facts, Parallel Resources, and so on. This is usually fairly quick.
All of these three stages are necessary if you're to access the resources that you've just added. It would theoretically be possible for Logos to run these processes in the background with Logos loaded, and then quickly replace the old files with the new ones. However, this would be non trivial (for example, you'd have to wait until none of the files were in use, even though they're used almost all the time), and you might lose user data (for example in the library catalog).
Because this part of indexing is necessary for Logos to begin, I believe it runs and normal or high priority.

Then indexing proper begins. There are four stages:

Creating the new Bible index.
Creating the new main index.
Merging the new Bible index into the existing index.
Merging the new main index into the existing index.
These indexes are done in the background, because they only effect searches. They therefore run on low priority.

There's probably more to it than even this, but hopefully that helps in understanding why it's hard to defer Preparing Your Library.

Jack Caviness

Just out of curiosity, it this a Windows only experience or has it happened with the Mac version also?

You must have read fewer of my posts over the years than I thought...

If there was anyone whose posts I wanted to avoid, it would not be yours. But I do not really remember this particular problem described in your posts. I will believe you if you say this has happened to you, and I will not need a dozen links as proof.

BillS

Then there's the Preparing Your Library stage:

Stage 1 of 'preparing your library' is the discovery mode. It scans through the downloads, compares them to existing resources, and moves them to the correct location. It's pretty quick, just 1 or 2s per resource.

Stage 2 of 'preparing your library' updates the library catalog. This makes sure all the data in your library is up to date with the new resources, and copies in the thumbnails and other metadata. This takes 1-2s per resource, plus another 5-30s depending on the size of your library.

Stage 3 of 'preparing your library' create the keylink databases. This is your prioritisations, so creating this database is crucial for Logos to be able to access your resources from Guides, Biblical Facts, Parallel Resources, and so on. This is usually fairly quick.

All of these three stages are necessary if you're to access the resources that you've just added. It would theoretically be possible for Logos to run these processes in the background with Logos loaded, and then quickly replace the old files with the new ones. However, this would be non trivial (for example, you'd have to wait until none of the files were in use, even though they're used almost all the time), and you might lose user data (for example in the library catalog).

Because this part of indexing is necessary for Logos to begin, I believe it runs and normal or high priority.

Hi Mark, my lockout occurred at the "Preparing Your Library" message. 45 minutes. Slow response while indexing? I could've paused it, & a few minutes later it'd have released me with access to enough to complete the planned Bible study, even if we had to defer real time searches until the next session.

But when the ONLY access to Logos is the splash screen "Preparing Your Library", I'm stuck until it's done.

BillS

your situation was avoidable.

I'll try setting it that way on Tuesday mornings. Otherwise, I WANT the download immediately. Only this time (at 400Mb) would I have chosen to stop it. And only with 1000 resources (Perseus) to index was the indexing / preparing going to take so long on this laptop as to be a problem.

It's threads have a low priority so it is interruptable when you want to run a task, otherwise it will consume all CPUs of a double/quad core processor. It was pleasantly easy to run another task on my dual core Atom powered Tablet (full Win 8), but i did find it harder on the desktop i7.

Interesting that the faster machines end up slower!

Randy W. Sims

It's threads have a low priority so it is interruptable when you want to run a task, otherwise it will consume all CPUs of a double/quad core processor. It was pleasantly easy to run another task on my dual core Atom powered Tablet (full Win 8), but i did find it harder on the desktop i7.

Interesting that the faster machines end up slower!

I've mentioned this a few times before. I do understand from Bradley that the indexing threads run at a lower priority, but there is a very noticeable improvement in system responsiveness when I use task manager to reduce the process priority to below normal. I do this often on large updates and on the few occasions I've had to rebuild the index. It works great!

BillS

I've mentioned this a few times before. I do understand from Bradley that the indexing threads run at a lower priority, but there is a very noticeable improvement in system responsiveness when I use task manager to reduce the process priority to below normal. I do this often on large updates and on the few occasions I've had to rebuild the index. It works great!

Thanks for confirming it's possible... Because of past involvement with "big iron" computers, I knew there would be a way... suspected it could be done in task manager... and after you confirmed it, I found the control. On a trivial (200kb resource download & index) instance this morning, I tried it & indexing completed. However, because it WAS trivial (only 1 resource), the "Preparing Your Resources" screen was also trivial.

It sounds like my only real remaining issue is the amount of time "Preparing Your Resources" when Logos releases a 1,000 resource, 400+Mb download that then has to go through the process. Here, Dave's counsel (on days it's critical to have L5, turn off automatic updates) will help.

If a lower priority on indexing allows me to use the cursor / touchpad WHILE L5 is indexing afterward, I'm a happy camper again.

Thanks, all, for your suggestions. I really appreciate the help as a short term answer using all the controls we currently have.

I'd still ask Logos about the long term strategy as it stands: is forcing us to turn OFF auto updates to ensure access to the software REALLY the way you want to go? If so, then what about "batching" bulk resource updates into a "patch Tuesday" kind of event that we can then schedule on a day that we can handle the impact?

BillS

It's threads have a low priority so it is interruptable when you want to run a task, otherwise it will consume all CPUs of a double/quad core processor. It was pleasantly easy to run another task on my dual core Atom powered Tablet (full Win 8), but i did find it harder on the desktop i7.

I tried setting the priority of indexing to LOWER THAN NORMAL, but that ONLY helps with the CPU. Unfortunately, I verified in an experiment this morning, that cpu load isn't the main problem. When Indexer gets to the point in its process that it consumes 100% of the disk (it does this for significant lengths of time (minutes, not seconds)-----THAT's when the PC is locked up until it's finished.

Dan Pogue

Hey Bill,

I was having problems with the indexer some time back. I am not sure this is the best way or may cause more problems, however, I would open the task manager, click on services, right click Logos indexer and stop that process. Was then able to continue to use Logos without any problems. This worked for me, not recommending...just something to consider if you are in that situation again...

Blessings,

danp

BillS

open the task manager, click on services, right click Logos indexer and stop that process.

Thanks for the suggestion, Dan. If worst comes to worst, I may try that... Pausing it for 4hr (from the system tray) has also worked & may do the same thing, though perhaps not as quickly.

Since Dave Hooton pointed out that the only remedy for both original issues is avoidance (don't let the big download happen to begin with), my main reason for continuing the thread has been 2-fold. I've been trying to find out if anyone else has found some system or task manager setting that could prevent system lockout (prevent the disk from getting 100% busy---from indexer?). And I've also been hoping we'd capture the attention of a Logos IS type who'd offer hope for a long term cure--or at least say "we've got the problem logged..."

Thanks again, & blessings to you!

Donnie Hale

When Indexer gets to the point in its process that it consumes 100% of the disk (it does this for significant lengths of time (minutes, not seconds)-----THAT's when the PC is locked up until it's finished.

To me that sounds like your disk needs some attention. I've got a several thousand book library, and when I was running Logos from a traditional (spinning) disk, indexing never made my system completely unresponsive. Worst case, I might get a quick hiccup in a song if I was playing music from that disk while indexing was in progress. Now that I've got Logos on an SSD, there's no noticeable impact at all.

So... Defragment? Faster disk? SSD? Maybe the "Reindex All" command (whatever the right syntax is in Logos) to rebuild the entire index and get to a better starting point for future indexing?

Donnie

BillS

Hi Donnie,

Just checked... less than 200Mb recovered in disk cleanup, & less than 3% fragmented BEFORE running defrag, which because disk is fast (7200rpm + 32Gb SSD built in). Defrag (which auto-runs + I often run it whenever (as now) its performance is in question. It'll only take a few minutes... While typing this, it's already got Pass One 70% relocated. And, even better, the disk is bouncing around at 100% busy without any adverse impact . The more I experiment, the more I'm convinced that Logos is the culprit. The ONLY time I experience that problem is when Logos is indexing & the disk is at 100%. Oddly, the cores aren't busy at all when the touchpad is locked out. (I am sometimes still able to control the laptop from the keyboard sans touchpad.)

EDIT: 20 minutes after start, 750Gb disk is 0% fragmented. We'll see if that helps.

I'll also try your suggestion to REBUILD INDEX. [:)]

It'll hurt when it gets to that point, but today's my day off & I can easily afford the experiment.