4.0c Beta 1 - Indexing takes a long time

After the new interlinear files finished downloading, L4 started indexing. Initially the remaining time was 14 or 15 minutes, then it rose to 17 or 18 hours. CPU is pegged at 80-90% running the indexer and the laptop is running very hot for hours. Is anyone else experiencing that?

I have a dual core HP laptop running Windows 7 64-bit, 4GB RAM. AMD Turion 64 X2 TL-60 2.00 GHz processor. This is the longest running indexing in many months.

Any ideas?

Peter

Find more posts tagged with

Comments

Terry Poperszky

Peter, did you have logging enabled? If so post the logs, mine only ran a couple of hours and my specs are not that much better than yours and I am running inside Fusion.

LaRosa Johnson

Mine took a while also... I'm assuming that because of the new interlinear Bibles it triggered a full re-index of both the "big" index & the Bible index, which could account for the added time.

I'm expecting this to keep taking place for as long as the new reverse interlinears are being released to us during this beta process.

Rosie Perera

https://community.logos.com/discussion/comment/106218#Comment_106218

Mine took a while also... I'm assuming that because of the new interlinear Bibles it triggered a full re-index of both the "big" index & the Bible index, which could account for the added time.

Ever since one of the late betas of 4.0b (so it would still be the case in 4.0c as well), they've changed indexing so it no longer does a Merge Index if it senses something has changed -- there were too many bugs in Merge Index. Now it forces a complete rebuild of your index. I'm not sure if that is just a temporary situation to get people up and running and stop having all the problems people were having with corrupted indexes (which would sometimes cause Logos to crash), until they get a chance to rework the Merge Index code, or whether this is a permanent change.

As always, it's probably best do do your downloads and indexing just before heading to bed so it will run overnight and be done by morning. And that way you can have other apps closed and let Logos have full control of the CPU so it will finish faster.

https://community.logos.com/discussion/comment/106228#Comment_106228

Ever since one of the late betas of 4.0b (so it would still be the case in 4.0c as well), they've changed indexing so it no longer does a Merge Index if it senses something has changed -- there were too many bugs in Merge Index. Now it forces a complete rebuild of your index. I'm not sure if that is just a temporary situation to get people up and running and stop having all the problems people were having with corrupted indexes (which would sometimes cause Logos to crash), until they get a chance to rework the Merge Index code, or whether this is a permanent change.

I think they mentioned somewhere that they'll try to fix the problems with Merge Index and bring it back (with 4.0c?)

As always, it's probably best do do your downloads and indexing just before heading to bed so it will run overnight and be done by morning. And that way you can have other apps closed and let Logos have full control of the CPU so it will finish faster.

Except that my computer desk is right next to my bed...

Peter

Ken Shawver

https://community.logos.com/discussion/comment/106240#Comment_106240

Except that my computer desk is right next to my bed...

Ah Peter, I ran it right before bed too thinking it might not take as long. But ended up sticking my iPod headphones in and listening to a great series on 1John before drifting of to sleep. Does wonders for not hearing my system chug away at indexing, thankfully my wife had her earplugs in too [:D].

Steven Baxley

I must be missing something ... I updated to 4.0c Beta 1, Logos downloaded a 278 MB download, but I still don't have any reverse interlinears. Logos is indexing, but I don't know what. Where should I look for the interlinears?

Thanks!

LaRosa Johnson

https://community.logos.com/discussion/comment/106302#Comment_106302

Steven,

Try opening up the ESV and there should be a "Display" button next to the "Interlinear" button, click that & it'll bring a menu to select Reverse Interlinear elements to be displayed inline.

Steven Baxley

https://community.logos.com/discussion/comment/106306#Comment_106306

Thank you, that's the trick.

John Graves

https://community.logos.com/discussion/comment/106502#Comment_106502

I really hope they find a way to release these things without causing a full reindex. This is the biggest problem i have with l4. My computer indexed all day and for all I know it is still indexing right now (my cpu in my office not the one I am typing on).

Dave Hooton

After the new interlinear files finished downloading, L4 started indexing. Initially the remaining time was 14 or 15 minutes, then it rose to 17 or 18 hours. CPU is pegged at 80-90% running the indexer and the laptop is running very hot for hours.

I need a household fan to stop my laptop from shutting down during indexing!

When I install this beta it will be on my quad core Desktop!

Dave Hooton

https://community.logos.com/discussion/comment/106661#Comment_106661

I really hope they find a way to release these things without causing a full reindex.

A full re-index is performed on the Bible Index, but a Supplemental index may be built or re-built for the main Library Index, not necessarily a full re-index (assuming that Merge is still absent).

EDIT: both indexes are fully rebuilt because 4.0c is introducing a new format, probably one that will allow Merge to work reliably.

John Graves

https://community.logos.com/discussion/comment/106691#Comment_106691

I really hope they find a way to release these things without causing a full reindex.

A full re-index is performed on the Bible Index, but a Supplemental index may be built or re-built for the main Library Index, not necessarily a full re-index (assuming that Merge is still absent).

EDIT: both indexes are fully rebuilt because 4.0c is introducing a new format, probably one that will allow Merge to work reliably.

Oh ok I hope you are right.

Ray Timmermans

https://community.logos.com/discussion/comment/106775#Comment_106775

Man, is this ever annoying! Not only do I not have a fully indexed set of books, I also have a slower computer to boot (thus far for nearly a full day with Logos4Indexer taking up most of my resources)--and this in the name of progress??! Whoever in the development department thought up the indexing idea needs to have a talking to!

I'm getting ready to switch back to Libronix 3. This just isn't worth it.

Ray Timmermans

TCBlack

https://community.logos.com/discussion/comment/107019#Comment_107019

Whoever in the development department thought up the indexing idea needs to have a talking to!

Indexing was our idea. [:)] Way back at Logos 2.x the program would index your resources. I don't know how the methods compared. But when L3 came out, the indexing was no longer there. The result was loooooooong searches through thousands of resources. In the Newsgroups we began (on occasion) to clammor for indexing to return - because we wanted google fast results.

Whether the current indexing implementation is a result of those requests, or exclusively an internal decision (I suspect both) - I have to side with indexing over not indexing. When I get on a thought path and start searching I dont want to have to take a walk break every time I think, "Hmmm. I wonder if someone mentions this....".

With indexing I can hit a search term and get my answer and move on.

Yes the indexing is painful and complicated by the fact that you're using a beta; but just envision the benefits. And if you have a dual core system, set the indexer to only be allowed one core if it's slowing your system too much.

Rosie Perera

https://community.logos.com/discussion/comment/107590#Comment_107590

And if you have a dual core system, set the indexer to only be allowed one core if it's slowing your system too much.

How do you do this? (I have a Quad Core system.)

Jonathan

https://community.logos.com/discussion/comment/107680#Comment_107680

And if you have a dual core system, set the indexer to only be allowed one core if it's slowing your system too much.

How do you do this? (I have a Quad Core system.)

Open Task Manager

Select tab "processes"

Right Click on "Logos4Indexer.exe"

Click on "Set Affinity"

Uncheck <all processors>

Check the box next to one or more of your processors.

Rosie Perera

https://community.logos.com/discussion/comment/107731#Comment_107731

And if you have a dual core system, set the indexer to only be allowed one core if it's slowing your system too much.

How do you do this? (I have a Quad Core system.)

Open Task Manager

Select tab "processes"

Right Click on "Logos4Indexer.exe"

Click on "Set Affinity"

Uncheck <all processors>

Check the box next to one or more of your processors.

So you mean you can only do this while the Indexer is running? Will it take effect only for that time, or for any future time that .exe gets run?

I'm done with Indexing for now, so Logos4Indexer.exe isn't running on my machine. Is there any way to set this from elsewhere than the Task Manager so it will have the right affinity next time I run it (and thereafter)? (I suppose I could force a rebuild of the index to get it running in the Task Manager and do it that way, but I'm looking for a more straightfoward solution.)

Bradley Grainger (Logos)

https://community.logos.com/discussion/comment/107731#Comment_107731

Click on "Set Affinity"

Uncheck <all processors>

Check the box next to one or more of your processors.

Unfortunately, the Indexer does not detect this situation; it still thinks you have a quad-core computer and will launch four threads, which Windows will now run on only one or two cores. The extra context-switching will impair the performance of the application. The Indexer already asks Windows to run it at the lowest possible priority, so manually restricting the cores it can run on shouldn't be any better than letting the kernel manage it for you, but YMMV. (Low-priority processes are guaranteed a regular timeslice even if there is other work, so I suppose this could increase throughput for other foreground tasks. It could also let the hardware disable the unused cores, possibly reducing power usage and heat generation.)

TCBlack

https://community.logos.com/discussion/comment/107953#Comment_107953

So you mean you can only do this while the Indexer is running? Will it take effect only for that time, or for any future time that .exe gets run?

Yes, it has to be running to manipulate it in this way. Plus it's not "sticky" so you'll have to adjust it when you want to.

But Bradley's caveat is interesting, it sounds like you'll more than double your index time.

Ray Timmermans

https://community.logos.com/discussion/comment/107590#Comment_107590

Yes the indexing is painful and complicated by the fact that you're using a beta; but just envision the benefits. And if you have a dual core system, set the indexer to only be allowed one core if it's slowing your system too much.

Thomas,

I would like to do more than envision the benefits. I would like to experience them. I do not have a dual core system and have no intention of purchasing one in the near future. Logos 4 is a drag on my system resources--especially when indexing. It is a resource hog. I don't feel the inclination to have to purchase entirely new hardware to run this program. This seems like an ill-thought-out development strategy.

TCBlack

https://community.logos.com/discussion/comment/108181#Comment_108181

I would like to do more than envision the benefits. I would like to experience them.

I understand Ray. I happen to have an "old" dual core system myself. (3-4 years old with chances of it getting much older being very good). I understand that upgrading hardware isn't possible for everyone.

This seems like an ill-thought-out development strategy.

Many would agree with you it seems. Mind you that L3 still functions perfectly - that was an intentional move to not replace L3 but to Add L4 because they understood that L4 would be power hungry.

FWIW I ran L4 during the pre-release beta on an aging P4 laptop and had first hand experience of very long indexing times, and that was an experience repeated several times over! It was not fun. So please know that I do understand where you're coming from.

The indexing is painful especially on slower machines. The program was designed for machines purchased within the last two years and as such I still don't "qualify" in that sense. I've been able to upgrade my RAM and Video card a bit, and that has made the whole process much better.

That said, how much progress have you made on the indexing?

LaRosa Johnson

https://community.logos.com/discussion/comment/108181#Comment_108181

I don't feel the inclination to have to purchase entirely new hardware to run this program. This seems like an ill-thought-out development strategy.

I hope I don't come off rude or crass when I say this (and if I do, I apologize), but at what point does a software developer continue to feel an obligation to continue supporting old hardware? When a new version of a software application comes out, as consumers, should we immediately assume that just because we buy it that it should work regardless of the hardware we run it on? I used to be an avid computer gamer (before finally giving that up permanently for console systems [i.e. Xbox 360 & the like]), and there were several titles that I would purchase annually when the new version came out. I would buy the new version, but I always came in with the expectation that it was going to run slower & have performance issues because: 1) I'm still running the same hardware, and 2) they've made improvements to the game that demand newer hardware in order for me to take advantage of those benefits. Now, I didn't complain or moan when my games ran slower because I knew that it was my system that was the problem, not the software. Just like with all of the people that are now upgrading from XP to Windows 7; sure, they may be able to upgrade because they meet system requirements, but that doesn't mean that it's going to run as well as a brand new Windows 7 machine purchased this morning; and that's to be expected.

My point is this, why should a software company stifle their development and not bring out new stuff because they want their product to run on software that is several years old (especially considering the life cycle of most computers these days is roughly 3 years)? As a consumer, I expect software companies to take advantage of the latest technology & tools that are available to them, but realizing that if I'm running an older system, I may need to upgrade to take advantage of what's being offered.

So, is it really an ill thought out development strategy? I don't think so. I think they're doing exactly what they should be, which is keeping up with where computers currently are & are headed.

Ray Timmermans

https://community.logos.com/discussion/comment/108203#Comment_108203

That said, how much progress have you made on the indexing?

It took about a day and a half and was stuck on "10 hours" remaining for a looooong time....but finally did finished. I certainly hope that development is looking at someway of streamlining the indexing process..either based on library purchases or registration of products--just something to speed this up! What bothers me is the inability to do anything else with my laptop while it decides it needs to index again. It's almost as though it must have a dedicated machine for nothing else than Logos--surely they can't say that that was what they envisioned?! Also I've noticed that my laptop gets very hot during the process of indexing. I think others have mentioned this also.

What I would like to see, if indexing is a must, is a downloadable index based on a customer's purchases. After all, Logos knows what each of us has purchased...certainly enough of us have Scholar's Gold to use that as a base. It certainly would make me feel as though I have a complete program. As it stands it seems like I'm not only running a beta but an alpha. I was shocked as a beta tester that the 4.0 went to RC

Ray

Ray Timmermans

https://community.logos.com/discussion/comment/108214#Comment_108214

I don't feel the inclination to have to purchase entirely new hardware to run this program. This seems like an ill-thought-out development strategy.

I hope I don't come off rude or crass when I say this (and if I do, I apologize), but at what point does a software developer continue to feel an obligation to continue supporting old hardware? When a new version of a software application comes out, as consumers, should we immediately assume that just because we buy it that it should work regardless of the hardware we run it on?

Microsoft seems to manage it as do plenty of smaller companies developing products using their platforms.

My point is this, why should a software company stifle their development and not bring out new stuff because they want their product to run on software that is several years old (especially considering the life cycle of most computers these days is roughly 3 years)? As a consumer, I expect software companies to take advantage of the latest technology & tools that are available to them, but realizing that if I'm running an older system, I may need to upgrade to take advantage of what's being offered.

In my opinion it is not stifling--it is making them be creative in their problem solving skills--rather than applying a "one size fits all" approach. As a consumer also, after having paid hundreds of dollars to the company for resources, I expect the ability to use them without feeling that I am unofficially on the development team of Logos at my own expense...or that this product really isn't ready for market in its present condition but it's all I got because all my resources have been morphed into Logos 4 format. Please, don't get me wrong, I would love to LOVE Logos 4. I really would--and if they found user-friendly way to create the index on THEIR end I would probably be much happier with their product!

But it is hard to approach Bible study with the thought, "OMG, I have to open Logos 4 again <sigh>" not knowing if it is going to spend the next day and a half re-indexing.

TCBlack

https://community.logos.com/discussion/comment/108250#Comment_108250

What I would like to see, if indexing is a must, is a downloadable index based on a customer's purchases. After all, Logos knows what each of us has purchased...certainly enough of us have Scholar's Gold to use that as a base.

I am certain this was discussed during the beta. At one point I thought it was a done deal; but then apparently not....

IIRC the index would end up being nearly the same size as the Library when shipping (or something related to that). I'm searching, but can't find that particular bit of information.

Rosie Perera

https://community.logos.com/discussion/comment/108354#Comment_108354

What I would like to see, if indexing is a must, is a downloadable index based on a customer's purchases. After all, Logos knows what each of us has purchased...certainly enough of us have Scholar's Gold to use that as a base.
I am certain this was discussed during the beta. At one point I thought it was a done deal; but then apparently not....
IIRC the index would end up being nearly the same size as the Library when shipping (or something related to that). I'm searching, but can't find that particular bit of information.

Even if the index were the same size as the Library, I think it would be faster to download it than to generate it fresh on one's own machine (depending on your network connection speed) and I'd prefer to have an index come with new resources I buy, which could then be merged (on my machine) in with my existing index. On the other hand, my machine is pretty fast at indexing so I'm not complaining.

George Allakhverdyan

https://community.logos.com/discussion/comment/108315#Comment_108315

In my opinion it is not stifling--it is making them be creative in their problem solving skills--rather than applying a "one size fits all" approach. As a consumer also, after having paid hundreds of dollars to the company for resources, I expect the ability to use them without feeling that I am unofficially on the development team of Logos at my own expense...or that this product really isn't ready for market in its present condition but it's all I got because all my resources have been morphed into Logos 4 format. Please, don't get me wrong, I would love to LOVE Logos 4. I really would--and if they found user-friendly way to create the index on THEIR end I would probably be much happier with their product!

But it is hard to approach Bible study with the thought, "OMG, I have to open Logos 4 again <sigh>" not knowing if it is going to spend the next day and a half re-indexing.

This was indeed considered heavily in the past and it can't be set in stone because users purchase new resources pretty often. If we were to have an index for every package and an index for every resource seperately then the issue seems trivial with merging indexes (which would still take some time) like you mention. That being said the merging indexing feature is currently unoptimized and doesn't benefit users, and is still a work in progress. Until it is fixed and until we handle the more prominent features and bugs we can't be certain creating prebuilt indexes to download/ship with the product is worth investing time in at the moment. I will post a link to this thread in the case regarding this issue, maybe if it gains enough popularity the devs can think about implementing it sooner. Thanks.

MJ. Smith

https://community.logos.com/discussion/comment/108250#Comment_108250

What I would like to see, if indexing is a must, is a downloadable index based on a customer's purchases.

Remember that the resources that you create for yourself are also indexed. Indexing also takes into account the fact that you have hidden (deleted) resources from the base package. While I agree that indexing can be a pain, I'd rather have Logos staff working on PBB's and lectionaries than indexing. Yes, I have an older machine and a smaller number of resources, but that is my personal preference.

Dave Hooton

https://community.logos.com/discussion/comment/108250#Comment_108250

What I would like to see, if indexing is a must, is a downloadable index based on a customer's purchases. After all, Logos knows what each of us has purchased...certainly enough of us have Scholar's Gold to use that as a base.

You are effectively asking Logos to generate your index and the two indexes would be about 4 GB for any Scholar's library that would have at least 7 GB of resources. Much as I dislike indexing on my laptop, I do have a download cap and would be very unhappy with GB's of extra downloads. Now that I have a quad core desktop I let it generate the indexes and export them to the laptop via Method 2 at http://wiki.logos.com/Quick_Installation_onto_multiple_computers! I don't mind the laptop performing supplementary (partial) indexing, but I export the resources via Method 3.

Ray Timmermans

https://community.logos.com/discussion/comment/108612#Comment_108612

In my opinion it is not stifling--it is making them be creative in their problem solving skills--rather than applying a "one size fits all" approach. As a consumer also, after having paid hundreds of dollars to the company for resources, I expect the ability to use them without feeling that I am unofficially on the development team of Logos at my own expense...or that this product really isn't ready for market in its present condition but it's all I got because all my resources have been morphed into Logos 4 format. Please, don't get me wrong, I would love to LOVE Logos 4. I really would--and if they found user-friendly way to create the index on THEIR end I would probably be much happier with their product!

But it is hard to approach Bible study with the thought, "OMG, I have to open Logos 4 again <sigh>" not knowing if it is going to spend the next day and a half re-indexing.

...like right now a day and a half after an upgrade and re-indexing and then another 100+MB resource update and total re-indexing again...currently timed at "10 hours" but more likely like another day and a half. Totally wasted time.

Dave Hooton

https://community.logos.com/discussion/comment/109610#Comment_109610

Ray

My suggestion above may not be possible for you but I always manage my downloads by setting Automatically Download Updates to NO! If I expect some new resources I might allow the download, else I refuse it (by clicking on the X in the notification popup).

Ray Timmermans

https://community.logos.com/discussion/comment/109610#Comment_109610

In my opinion it is not stifling--it is making them be creative in their problem solving skills--rather than applying a "one size fits all" approach. As a consumer also, after having paid hundreds of dollars to the company for resources, I expect the ability to use them without feeling that I am unofficially on the development team of Logos at my own expense...or that this product really isn't ready for market in its present condition but it's all I got because all my resources have been morphed into Logos 4 format. Please, don't get me wrong, I would love to LOVE Logos 4. I really would--and if they found user-friendly way to create the index on THEIR end I would probably be much happier with their product!

But it is hard to approach Bible study with the thought, "OMG, I have to open Logos 4 again <sigh>" not knowing if it is going to spend the next day and a half re-indexing.

...like right now a day and a half after an upgrade and re-indexing and then another 100+MB resource update and total re-indexing again...currently timed at "10 hours" but more likely like another day and a half. Totally wasted time.

It is 10:25 AM (eastern time) the following day with "35 minutes" remaining on the new indexing.

George Allakhverdyan

https://community.logos.com/discussion/comment/110189#Comment_110189

In my opinion it is not stifling--it is making them be creative in their problem solving skills--rather than applying a "one size fits all" approach. As a consumer also, after having paid hundreds of dollars to the company for resources, I expect the ability to use them without feeling that I am unofficially on the development team of Logos at my own expense...or that this product really isn't ready for market in its present condition but it's all I got because all my resources have been morphed into Logos 4 format. Please, don't get me wrong, I would love to LOVE Logos 4. I really would--and if they found user-friendly way to create the index on THEIR end I would probably be much happier with their product!

But it is hard to approach Bible study with the thought, "OMG, I have to open Logos 4 again <sigh>" not knowing if it is going to spend the next day and a half re-indexing.

...like right now a day and a half after an upgrade and re-indexing and then another 100+MB resource update and total re-indexing again...currently timed at "10 hours" but more likely like another day and a half. Totally wasted time.

It is 10:25 AM (eastern time) the following day with "35 minutes" remaining on the new indexing.

How big is your library? What is your computer specs?