Logos Indexer is a resource hog

I love Logos 5 but I hate the indexing process.

Why?

The indexing process causes my brand new Dell laptop to become virtually useless for other tasks. The cursor stops moving for seconds at a time rendering mouse usage unbelievably frustrating. I can't even type without things abruptly pausing for several seconds. It makes using my computer quite painful.

I would prefer that indexing took longer but left my computer usable for other tasks during the indexing process.

Is it not possible, Logos, to have the indexing process run in background mode?

PLEASE????

Find more posts tagged with

Comments

Rick

Is it not possible, Logos, to have the indexing process run in background mode?

Hi Taxee. If you use Windows this little known option might help you.

When Logos is indexing open Task Manager.
Click on the "Processes" tab.
Locate Logos Indexer
Right click on Logos Indexer and set the priority to a lower level.

This will cause indexing to be slower but you should see a bit of an improvement on your computer's performance while the indexing is taking place.

Thomas Ball

I love Logos 5 but I hate the indexing process.

Why?

The indexing process causes my brand new Dell laptop to become virtually useless for other tasks. The cursor stops moving for seconds at a time rendering mouse usage unbelievably frustrating. I can't even type without things abruptly pausing for several seconds. It makes using my computer quite painful.

I would prefer that indexing took longer but left my computer usable for other tasks during the indexing process.

Is it not possible, Logos, to have the indexing process run in background mode?

PLEASE????

Hi Taxee. I'm not a decision maker, so I can't speak to your request regarding the Indexer taking less processing power. You could request the option over at UserVoice though. After updates if you refrained from immediately installing them by re-starting the application, and instead installed them in the evening when you don't need the machine, then you'll be able to allow the install and index occur while you are asleep or otherwise occupied.

mab

https://community.logos.com/discussion/comment/545150#Comment_545150

Try turning Logos off while indexing runs. This releases more memory to finish the process faster. Sadly if you don't have a lot of RAM you will experience a distinct slowing down because the virtual memory gets used and that makes for a lethargic response. It is not unusual for the index process to use more than 4 gigs to run.

Randy W. Sims

Is it not possible, Logos, to have the indexing process run in background mode?

This should be happening now. Or at least it is on the beta channel for several weeks now. LogosIndexer.exe should be running at LOW (Idle) priority. If it is and you are still experiencing slowdowns, can you please post your system info: Press [Windows Key] + [Pause/Break Key], and post Windows edition, Processor, RAM, & System type. And whether using SSD or HDD.

DMB

https://community.logos.com/discussion/comment/545233#Comment_545233

I recently finished my every-now-and-then update. Just short of 2g download. Indexer ran at near 100% on an i7 with SSD.

Looks like the background indexing is working pretty well. (No offense, Randy. I know you have a lot more confidence in Bellingham than I have.)

Bruce Dunning

https://community.logos.com/discussion/comment/545241#Comment_545241

I recently finished my every-now-and-then update. Just short of 2g download.

Wow Denise that is large. How often do you update?

Todd Phillips

https://community.logos.com/discussion/comment/545241#Comment_545241

Indexer ran at near 100% on an i7 with SSD.

Looks like the background indexing is working pretty well.

Low priority indexing doesn't mean the indexer won't use all of the available CPU. It will get whatever CPU cycles are left over by more important processes. It means that the more important processes will get a higher percentage of the CPU than the indexer. So if nothing else is running, then the indexer will still get to use 100% of the CPU.

DMB

https://community.logos.com/discussion/comment/545244#Comment_545244

You're right, Todd! When I see Logos chewing up near 100%, I leave it alone and avoid downloads as much as possible. Sinaiticus became available but it missed the download. I don't want it indexing again.

Bruce, that included the monster update about 2 months ago maybe? I'll probably update again sometime in February or so. In between Libronix, my iPod and Amazon, L5 doesn't get much use. With my Billy Graham devotional, I thought I'd try out the home page for that. But sure enough, Logos cludged it up with other stuff I had set for 'no!'. Back to the iPod.

Scott S

The indexing process causes my brand new Dell laptop to become virtually useless for other tasks. The cursor stops moving for seconds at a time rendering mouse usage unbelievably frustrating. I can't even type without things abruptly pausing for several seconds. It makes using my computer quite painful.

Does the new Windows PC you purchased meet the recommended system requirements for Logos 5? (See a base package page e.g. https://www.logos.com/product/24539/silver ) If not, you should consider returning the PC and getting one that is adequate.

I can tell you that on a high performance PC, the indexing process goes unnoticed, and has no effect on typing, nor does it slow work in L5 or other applications.

However, the drag from Logos on your new PC is so horrible; my reaction is that there must be something wrong with your Logos installation or the PC.

I can offer some relief until you resolve the problem! Go into Program Settings and turn-off automatic downloads. That will allow you to control the download of resources and subsequent indexing, so you can do those processes when the PC isn’t being used.

Out of curiosity, what processor and how much ram is in your new PC?

Rod Bergen

https://community.logos.com/discussion/comment/545330#Comment_545330

Here are my basic system specs:

Microsoft Windows 7 Professional - Version 6.1.7601 Service Pack 1 Build 7601
System Model - Dell Latitude E6530
Processor Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz, 2701 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date Dell Inc. A11, 12/03/2013
Installed Physical Memory (RAM) 4.00
500Gb GB Hard Drive (not SSD)

I will try the other suggestions and let you know how they work.

Rod Bergen

https://community.logos.com/discussion/comment/545145#Comment_545145

Hi Taxee. If you use Windows this little known option might help you.
When Logos is indexing open Task Manager.

Click on the "Processes" tab.

Locate Logos Indexer

Right click on Logos Indexer and set the priority to a lower level.

This will cause indexing to be slower but you should see a bit of an improvement on your computer's performance while the indexing is taking place.

I do appreciate the suggestion but I checked the priority on the Logos Indexer and it was already set on "Low".

[:(]

Rod Bergen

https://community.logos.com/discussion/comment/545276#Comment_545276

You're right, Todd! When I see Logos chewing up near 100%, I leave it alone and avoid downloads as much as possible. Sinaiticus became available but it missed the download. I don't want it indexing again.
...Back to the iPod.

I guess this just reinforces my point.

You have an i7 with an SSD and you avoid downloads because they slow your system down.

I have also had the indexer crash several times. Once was so bad that customer service said that my only option was to rebuild my entire index. With a library of over 7,000 books, it took over 8 hours to rebuild.

Maybe that is my whole problem, Logos is not really designed to operate well with large libraries.

Perhaps the best solution is to stop purchasing additional resources.

[:^)]

Lee

https://community.logos.com/discussion/comment/546748#Comment_546748

I suggested to Logos making indexer a low priority and throttling CPU usage to some figure like 85%, but the latter part is rather messy, and Logos has not found a need to do it yet. Having said that, I still don't understand how a low-priority indexer can hang otherwise healthy machines. Something's not quite right.

Don Awalt

https://community.logos.com/discussion/comment/546755#Comment_546755

I suggested to Logos making indexer a low priority and throttling CPU usage to some figure like 85%, but the latter part is rather messy, and Logos has not found a need to do it yet. Having said that, I still don't understand how a low-priority indexer can hang otherwise healthy machines. Something's not quite right.

Logos handles large libraries quite well, that's not your problem. The indexer on my laptop works extremely quickly and I barely know whether it's running or not. My library is over 17,000 Logos resources, the Resources folder is 48GB, the main Library Index is almost 13GB, I am a very long time user. My laptop is a 16GB Dell Precision 6700 with a NVIDIA Quadro K5000M video card, plus a disk flash card for very fast disk access. My system rebuilds the entire index in 3 hours - and I have no problem using the laptop for other things in the meantime.

Logos 4/5 bogs down if there is a weak link in CPU, memory size, graphics card, OR disk access - let no one convince you otherwise. This is a high performance application that runs a version of SQLServer on it; while users who run CAD/CAM, engineering, software development, or photography apps know the value of a suitably performing computer, for some reason people who run a similarly equipped database application with sophisticated display, functions, and calculations as a Bible app (Logos) still equate the app to reading a book on a Kindle, and wonder why a mid- to low-end computer doesn't perform well. Nothing could be further from the truth, and the sooner people realize that, the sooner they are able to take advantage of how good Logos really is.

My laptop is high end, of course - but Logos runs at light speed on it. There is a budget and a computer combination that will be the right price/performance match for you, as all combinations exist between running great and running poorly.

I hope this honest assessment helps.

DMB

https://community.logos.com/discussion/comment/546773#Comment_546773

Taxee ... recommend you do like us. We renovated our house to make room for the high powered super cooled indexer (garden hose). If you want performance, you need to face up to power. Tool-time power. Hi Don!

BillS

https://community.logos.com/discussion/comment/546755#Comment_546755

I suggested to Logos making indexer a low priority and throttling CPU usage to some figure like 85%, but the latter part is rather messy, and Logos has not found a need to do it yet. Having said that, I still don't understand how a low-priority indexer can hang otherwise healthy machines. Something's not quite right.

Au contraire! I'm running Logos Bible Software 5.2 SR-3 (5.2.0.0091), which sets the indexer to run by default at LOW priority. There isn't a lower priority, AFAIK. While there is still 15-60 seconds (depending on book) when Indexer has the laptop (specs below) so busy that the cursor won't respond, it no longer ties the system up for minutes or even hours as it did before Logos made the change. I had been manually changing it for some time before L5 started defaulting to the Low priority, so I noticed the beta when they changed it, & now it's in the SR. :-)

That said, it' better, but the OP is right that it's an index hog. I'm just adding, Lee, that they acted on your suggestion.

Blessings!

Lee

https://community.logos.com/discussion/comment/546800#Comment_546800

Low priority is something that was done very early on, I think, but with some implementation problems.

Anyway, the THEORY is with low prioritization, none of what some users are reporting would be happening, i.e. unresponsiveness. So there must be some small remaining problem preventing the theory from working out.It could be a problem with Logos code, or there could be some arcane software conflicts with, say, a video driver... I don't think Logos has isolated the problem yet.

My suggestion to throttle CPU usage was just a practical one. Some programs that work constantly in the backgroud, e.g. antivirus scanners, have a setting to throttle CPU usage, and even set it to some non-100% value by default.

Simon’s Brother

https://community.logos.com/discussion/comment/546747#Comment_546747

I have a 4-5 year old desktop machine running Win 7 Pro 32 bit with 4 GB RAM. I don't have any major problems with indexing stoping me do other things. Users with SSD report excellent performance. Is your Windows 32 or 64 bit. If 64 bit consider upgrading memory to 8GB RAM if possible. Logos recommends 6, but I have no problems with my current setup. My library would fall into the very large category. I've never owned a laptop so not sure if that has any bearing on performance.

Lee

https://community.logos.com/discussion/comment/546828#Comment_546828

Let's clarify the issue. It's not the performance per se, but the unresponsiveness. Users complain of hung screens, hung cursors... The reported problems should not be happening, even if the hardware is not quite up to par. On the Mac side, the reports seem more frequent.

Another issue which seems to be catching some users out is some catastrophic glitching of the indexer, probably because of some corrupted file somewhere. Right now, the indexer is not handling unexpected conditions gracefully (think SQL corruption and recovery).

Don Awalt

https://community.logos.com/discussion/comment/546830#Comment_546830

Freezes and hangs very much can be a performance issue. What I have seen in running Logos on under-performing hardware is the disk becomes the ultimate bottleneck when indexing. Think of it as grid lock in an intersection of a big city. The light keeps changing but cars are int he way of each other so nothing gets through. Tasks and processes "line up" to use the resources, and I/O takes a long time to disk, screen, etc. The same grid lock can occur, although of course it can also be due to a bug or other problem.

Lee

https://community.logos.com/discussion/comment/546852#Comment_546852

Prioritization of indexer should also lower prioritization in I/O.

To use the traffic gridlock metaphor, a lower priority for indexer means that if any vehicles need to get into the highway, indexer cars automatically get off the highway.

As far as I can tell, there are already plenty of user reports. Logos has responded by changing a few things, and approaching a solution. I can't understand why some users are so eager to characterise user reports of problems as sub-par hardware.

To help troubleshoot these non-isolated problems, Logos and users need to work together to find the factors responsible.

Dave Hooton

https://community.logos.com/discussion/comment/546830#Comment_546830

Right now, the indexer is not handling unexpected conditions gracefully (think SQL corruption and recovery).

Was that an encouragement for Logos to initiate recovery? Because v5.2x will re-index after an error is detected, but I'm not sure it correctly captures the conditions where automatic re-indexing (with messages in the logs) will be effective i.e. corrupt index files. If a resource is corrupt then automatic re-indexing is a waste of time.

Randy W. Sims

https://community.logos.com/discussion/comment/546747#Comment_546747

Here are my basic system specs:

Microsoft Windows 7 Professional - Version 6.1.7601 Service Pack 1 Build 7601
System Model - Dell Latitude E6530
Processor Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz, 2701 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date Dell Inc. A11, 12/03/2013
Installed Physical Memory (RAM) 4.00
500Gb GB Hard Drive (not SSD)

The only concerning thing I see there is memory. Ideally you should upgrade to 8GB to see a performance boost not just in Logos but overall. I'm not one of the ones that believes you should have to upgrade your computers to run Logos, but 4GB RAM is really minimal overall. I think Windows 8 handles low memory a bit better than previous versions.

One thing you might try is running Process Lasso (free to try). Run it. Hide some resources (eg a set of commentaries) in Logos, Unhide them, and let it re-index. See if system responsiveness improves while Process Lasso is running. If it does, Post the logs generated by Process Lasso.

Keep Smiling 4 Jesus :)

https://community.logos.com/discussion/comment/546747#Comment_546747

Here are my basic system specs:

Microsoft Windows 7 Professional - Version 6.1.7601 Service Pack 1 Build 7601
System Model - Dell Latitude E6530
Processor Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz, 2701 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date Dell Inc. A11, 12/03/2013
Installed Physical Memory (RAM) 4.00
500Gb GB Hard Drive (not SSD)

For Logos, a SATA III Solid State Disk (SSD) would noticeably improve responsiveness. With low prioritization of indexing process, an Intel 3rd Generation i7-3740QM @ 2.70 GHz has sufficient processing capacity to quickly read Logos resources and create index entries so Hard Drive speed becomes the bottleneck. Trying to use computer when resources are being indexed would be challenging since indexing disk read and writes are competing with all other disk access (similar to driving in rush hour traffic with many other cars that has many brake lights).

Noticed Dell Latitude E 6530 owner's manual => ftp://ftp.dell.com/Manuals/all-products/esuprt_laptop/esuprt_latitude_laptop/latitude-e6530_Owner%27s%20Manual_en-us.pdf shows optical drive can be replaced (and lacks internal mSata). Wonder about replacing optical drive with SSD => http://thessd.com/replace-laptop-hdd-with-ssd-then-move-hdd-to-2nd-drive-bay/ ? An alternative idea is replacing primary hard drive with SSD.

Keep Smiling [:)]

Lee

https://community.logos.com/discussion/comment/546871#Comment_546871

Right now, the indexer is not handling unexpected conditions gracefully (think SQL corruption and recovery).

Was that an encouragement for Logos to initiate recovery? Because v5.2x will re-index after an error is detected, but I'm not sure it correctly captures the conditions where automatic re-indexing (with messages in the logs) will be effective i.e. corrupt index files. If a resource is corrupt then automatic re-indexing is a waste of time.

The operative words in bold. Yes, I'm saying the indexer should attempt some kind of remediation of both index and database corruption, but it needs to do so correctly and with minimum fuss.

With low prioritization of indexing process, an Intel 3rd Generation i7-3740QM @ 2.70 GHz has sufficient processing capacity to quickly read Logos resources and create index entries so Hard Drive speed becomes the bottleneck. Trying to use computer when resources are being indexed would be challenging since indexing disk read and writes are competing with all other disk access (similar to driving in rush hour traffic with many other cars that has many brake lights).

Not really. When CPU prioritization is dropped in Windows (at least), IO priority is also dropped. I don't really know how to explain it anymore, the traffic analogy breaks down since there is a controller/scheduler at work, deploying traffic. But rest assured, if IO prioritization drops and everything works as documented, the indexer should not be holding up traffic in the way reported by some users.

Randy W. Sims

https://community.logos.com/discussion/comment/546915#Comment_546915

When CPU prioritization is dropped in Windows (at least), IO priority is also dropped.

This is what I thought also. Most references I found seem to indicate this is true. And it definitely seems true based on the behavior of my system. However, Process Explorer and Process Lasso do not show a lowered IO Priority for the indexer process. I almost think that they are reporting the requested priority level rather than the effective priority level. I'm not at all sure what to believe in this case.

Lee

https://community.logos.com/discussion/comment/546921#Comment_546921

As I had indicated, something is definitely wrong somewhere.

Keep Smiling 4 Jesus :)

https://community.logos.com/discussion/comment/546915#Comment_546915

I don't really know how to explain it anymore, the traffic analogy breaks down since there is a controller/scheduler at work, deploying traffic. But rest assured, if IO prioritization drops and everything works as documented, the indexer should not be holding up traffic in the way reported by some users.

Traffic analogy is scheduling small vehicles among many big buses, which tends to reduce overall throughput to match the buses. Indexing has four threads running in parallel while analyzing resource files. Looking at my resources folder, found Logos resource files average size is 260 MB. Hence four threads in parallel would request ~1 GB of files for reading then write out ~620 KB of indexing information.

Keep Smiling [:)]

NB.Mick

https://community.logos.com/discussion/comment/546989#Comment_546989

Looking at my resources folder, found Logos resource files average size is 260 MB.

You sure?

I think only a minimal number (lower than a dozen) of Logos resources even exist that go over 100 MB - I may be wrong, though, with the m.Ed and other things that include video files. But this average would mean that a library with 4k books need a Terabyte for resource files and another for indexes etc. I believe libraries this size were sold on DVDs, and I seem to remember that my >6k books installation including indexes and everything is smaller than 40 GB

Don Awalt

https://community.logos.com/discussion/comment/546915#Comment_546915

I don't really know how to explain it anymore, the traffic analogy breaks down since there is a controller/scheduler at work, deploying traffic.

This is totally incorrect. If there is not enough hardware capability, resources are lining up to get work done faster than they can get cleared, no matter how sophisticated the work controller is. Resources are timing out as well because it takes too much time for a completion to occur.If indexing is pumping out 1000 IO requests a second and they are taking 1/2 second to complete (just as an example), then responses will take longer, and longer, and longer - in a minute 60000 requests have been served, 120 a minute getting completed. How long until request #60,001 gets completed? No retry/error processing logic can handle under-performing hardware forever.

Lee

https://community.logos.com/discussion/comment/546996#Comment_546996

Suit yourself.

Note to Logos: please follow MS guidelines to ensure Scenario 4 in their list of I/O Access Patterns.

Note to Taxee: please contact Cust Service and tell them your problem. They should probably hook you up with someone who can troubleshoot. Logos may need to mine specific information about your setup, which is beyond the information that logging would capture.

Dave Hooton

https://community.logos.com/discussion/comment/546994#Comment_546994

Looking at my resources folder, found Logos resource files average size is 260 MB.

You sure?

The range of indexable** resources is 8 KB to 213 MB in my library with an average size of 6.8 MB (the largest non-indexable resource is 404 MB and the average of all resources is 8.2 MB).

** datasets are not indexed.

BillS

https://community.logos.com/discussion/comment/546830#Comment_546830

Let's clarify the issue. It's not the performance per se, but the unresponsiveness. Users complain of hung screens, hung cursors... The reported problems should not be happening, even if the hardware is not quite up to par. On the Mac side, the reports seem more frequent.

Another issue which seems to be catching some users out is some catastrophic glitching of the indexer, probably because of some corrupted file somewhere. Right now, the indexer is not handling unexpected conditions gracefully (think SQL corruption and recovery).

Clarify the issue: agreed. In that spirit, I'd offer that on my laptop when the screen (in reality the cursor) is unresponsive, I can still use alt-tab & cursor keys to navigate menus. And when I get to my performance data on home page (utilizations of CPU, network, & disk), it's always the disk that's saturated. Not the CPU. Even when the CPU is at 100%, I can still move the cursor. System (cursor) doesn't become unresponsive until the disk is maxed out. My disk is a SATA3 hybrid 7200rpm disk with on-board 32Gb SSD.

What's going on in others' systems when the pc becomes unresponsive?

Simon’s Brother

https://community.logos.com/discussion/comment/546830#Comment_546830

Lee assuming you are equating the word performance with speed: in using the word performance I take that to be inclusive of what you are saying and don't mean speed of indexing alone. If the cursor is freezing then that to me is poor performance. Speed is only one component of system performance.

If you are equating the word performance to something else then please clarify what you mean by the word.

DMB

https://community.logos.com/discussion/comment/547110#Comment_547110

Dave ... you don't have Ramsay? His cities of Paul top out at 472M and his top 3 almost equal a gig.

EDIT: I'll take that back. Ramsay's Cities of St Paul is 20M; the Cities of Paul is 470M. So Ramsay's not the big boy.

Dave Hooton

https://community.logos.com/discussion/comment/547121#Comment_547121

Dave ... you don't have Ramsay? His cities of Paul top out at 472M and his top 3 almost equal a gig.

Correct! But what is your average resource size?

Keep Smiling 4 Jesus :)

https://community.logos.com/discussion/comment/546994#Comment_546994

Looking at my resources folder, found Logos resource files average size is 260 MB.

You sure?

[:$] Oops, overall average size of my resource files is 2.66 MB. Cities of Paul is a media collection with over 900 photos.

Average size of *.lbxlls resource files is 2.403 MB. Average size of *.logos4 files is 2.406 MB (with size ranging from 19 KB to 472,095 KB).

Micrososft has an I/O Prioritization Document from 10 May 2006 => http://download.microsoft.com/download/a/f/7/af7777e5-7dcd-4800-8a0a-b18336565f5b/Priorityio.doc with scenario 4. Learned Vista was the first Windows platform with I/O prioritization, which is also the minimum supported OS for Logos 5. Note: Microsoft does not support .NET Framework 4.5 on Windows XP.

Thankful for a Dell 17" laptop (Inspiron 17R with i7-3610QM @ 2.30 GHz) that has a primary hard drive for Windows 7 and a secondary SATA III Solid State Drive (SSD) dedicated for Logos 5 use. With 8 GB of RAM have set aside 512 MB in a RAM disk for temporary file use, which works fine for incremental indexing. When completely rebuilding library index, need to use hard drive for temporary files since 512 MB is not enough. Thread => It's become so sluggish! includes more RAM Disk information.

Keep Smiling [:)]

Lee

https://community.logos.com/discussion/comment/547117#Comment_547117

What's going on in others' systems when the pc becomes unresponsive?

That is precisely what Logos, with the help of affected users, should try to track down or work around. It could be anything: a rogue driver, a particular SATA controller, another company's application, a line of sloppy code somewhere... The answer PROBABLY isn't: because the user does not have an SSD or some latest piece of hardware (which in some cases, they actually DO HAVE). But if it WERE, then Logos would do well to mitigate the IO saturation, the IP stack logjam, or whatever.

As far as I can tell, on the Mac side Logos is now actively addressing user complaints when OSX 10.9 is involved. They're trying to track the problem(s) down.

On the Windows side the indexer problems could be related, or entirely different. But the problems seem less common.

Users (Taxee in this case, not me) come on to report unexpected, unexplained behaviour. Cheerful "expert" analogies and advice like "Get a SSD", "Logos is a powerful piece of software" are well-intended but could get really tiresome because not only does it help to get to the underlying problem, the affected user can get turned off. The forum already has quite a few threads that drag on like this.

Better to refer Taxee to Logos.

Keep Smiling 4 Jesus :)

https://community.logos.com/discussion/comment/547253#Comment_547253

Better to refer Taxee to Logos.

=> https://www.logos.com/contact

Thankful for a number of Logos employees who interact with forum threads: e.g. 2nd reply in this thread:

Hi Taxee. I'm not a decision maker, so I can't speak to your request regarding the Indexer taking less processing power. You could request the option over at UserVoice though. After updates if you refrained from immediately installing them by re-starting the application, and instead installed them in the evening when you don't need the machine, then you'll be able to allow the install and index occur while you are asleep or otherwise occupied.

Logos User Voice Suggestion => Improve Logos Indexer performance (allow it to index in background) currently has 3 votes. Every Logos user has 10 votes for suggestions; up to 3 can be placed on one item.

Concur with thread title: Logos Indexer is a resource hog. When indexer is running, computer responsiveness slows down so Tommy's suggestion to schedule indexing when not using computer is prudent plus applicable on Windows and OS X.

Keep Smiling [:)]

Bradley Grainger (Logos)

https://community.logos.com/discussion/comment/547029#Comment_547029

Note to Logos: please follow MS guidelines to ensure Scenario 4 in their list of I/O Access Patterns.

Tried it; didn't work.

An indexing scenario was run while a Windows 8 computer was otherwise completely idle; there was no other competing I/O (unless the OS itself is performing background tasks). Indexing was performed on a 1TB 7200rpm hard disk.

With "normal" I/O, indexing (of 1,827 resources) took 38 minutes (to build the entire index from scratch). With "background" I/O, indexing took over 17 hours for the same task.

If "background I/O" did what the documentation claims it does (i.e., service high priority I/O requests first, but otherwise let I/O run normally), we would use it in a heartbeat. But empirical testing shows that it slows down I/O by over 25x even on an idle machine (and, presumably, much more if the machine is actually in use, but we didn't write an automated test to simulate an active environment). This means that what used to take hours would now literally take days, and typical indexing wouldn't even complete if the customer left their machine running overnight.

Again, if Windows "background I/O" would slow the indexer down during the day, but let it run at full speed overnight, we would use it. However, I haven't seen any evidence that this is the case.

Additionally, I have seen very few backup, virus scan, or desktop search products that use "background I/O" (maybe for the same reasons?). That means, if you're the only process on the box that is doing the right thing (by using background I/O), you get unfairly penalised (because no one else is "playing nice"). But now they can't reduce their I/O priority because programs like ours haven't. And so the circle of normal I/O priority perpetuates itself...

Lee

https://community.logos.com/discussion/comment/547274#Comment_547274

Well, one wonders about the exact methodology of that trial and how valid those results are, but it wouldn't be the first time that Windows is broken. It could be a driver thing, or MS's APIs don't follow their own documentation.

Work around it? Throttle CPU? Throttle your IO requests (e.g. make 1-2% IO requests low priority, or certain handles)? Submit a dev inquiry to MS?

The indexer must learn to play nice.

Dave Hooton

https://community.logos.com/discussion/comment/547296#Comment_547296

Work around it? Throttle CPU? Throttle your IO requests (e.g. make 1-2% IO requests low priority, or certain handles)? Submit a dev inquiry to MS?

The indexer must learn to play nice.

But it can't be universal**. There would have to be at least two Settings (CPU, I/O) for users to play around with and their defaults would depend on the number of CPU cores/hyperthreading, the type of main drive (HDD, SSD, MMC) and available RAM.

** I have a first generation laptop Core i5, 3.75 GB RAM and an SSD restricted by the computer's SATA 2 interface, but the indexer barely affects multi-tasking. Ditto for my second gen desktop Core i7, 8 GB RAM and 10000 RPM, 2.5" HDD also restricted by a SATA 2 interface.

DMB

https://community.logos.com/discussion/comment/547263#Comment_547263

Hmmm ... looks like my Logos resources are pretty tiny. 2.9M average or total 10g. Mine aren't even remotely close to Dave's average. Even his non-resource group is far larger. Part of this might have to do with excising the parts of books I don't like. Plus not using Logos.

On my Libronix, where the books are far better, my average is 4.7M. Looks like there's 2g of fluff over on the Logos side (doesn't surprise me). After a career of manipulating stats to insure bad decisions, it's almost second nature.

Lee

https://community.logos.com/discussion/comment/547324#Comment_547324

But it can't be universal

Much will depend on the creativity of the Logos dev team, which I'm sure there is no lack of. [Y] Those guys just need to dig around a little more.

Keep Smiling 4 Jesus :)

https://community.logos.com/discussion/comment/547324#Comment_547324

Work around it? Throttle CPU? Throttle your IO requests (e.g. make 1-2% IO requests low priority, or certain handles)? Submit a dev inquiry to MS?

The indexer must learn to play nice.

But it can't be universal**. There would have to be at least two Settings (CPU, I/O) for users to play around with and their defaults would depend on the number of CPU cores/hyperthreading, the type of main drive (HDD, SSD, MMC) and available RAM.

.

.

** I have a first generation laptop Core i5, 3.75 GB RAM and an SSD restricted by the computer's SATA 2 interface, but the indexer barely affects multi-tasking. Ditto for my second gen desktop Core i7, 8 GB RAM and 10000 RPM, 2.5" HDD also restricted by a SATA 2 interface.

Concur about nice not being universally desired, personally would appreciate option for Logos Indexing to go fast since have Logos 5 installed on several computers and can schedule when indexing starts.

Keep Smiling [:)]

Lee

https://community.logos.com/discussion/comment/547392#Comment_547392

[:^)] Aren't we forgetting that the indexer can be programmed with an algorithm to settle on how aggressive it needs to be? This is afterall the whole premise of Scenario 4.

But more than that, could we put an end to threads answering bug or pseudo-bug reports with "I've got this-and-that and it works fine, you should try it too". Or dial it down a bit? It's a fine line between assistance and condescension, and misinterpretation is all too rife in a text forum.

Rod Bergen

https://community.logos.com/discussion/comment/547413#Comment_547413

Aren't we forgetting that the indexer can be programmed with an algorithm to settle on how aggressive it needs to be? This is afterall the whole premise of Scenario 4.

This is really the point of my original post. If Logos says that my machine meets minimum requirements, then it shouldn't cause my system to grind to a halt every time it indexes. I appreciate that if I want to go out and buy a supercomputer it will run everything at lightning speed and I can look down my nose at all those that can't afford the latest and greatest tech.

But Logos is used by a lot of pastors, students and others who are on limited budgets or whose organizations are missional and so must watch every penny. It should not interfere with the normal operation of a machine that meets minimum standards.

Keep Smiling 4 Jesus :)

https://community.logos.com/discussion/comment/547423#Comment_547423

This is really the point of my original post. If Logos says that my machine meets minimum requirements, then it shouldn't cause my system to grind to a halt every time it indexes.

Apologies: looking at Logos Support web pages, have not found a reference that indexing will not cause a computer to slow down.

Noticed Windows 7 minimum hardware => http://windows.microsoft.com/en-us/windows7/products/system-requirements is close to Logos minimum => https://www.logos.com/support/tech-faq

Logos recommended hardware page => https://www.logos.com/support/logos5/hardware includes tip: "Logos 5 is a very large application, so meeting or exceeding the recommended requirements will drastically improve performance. If you want the best possible performance, think of these as a starting point." Caveat: needed to click several links to find Logos performance tip.

Logos installation page => https://www.logos.com/installation Step 4 Preparing your Library links to => https://www.logos.com/indexing with link to a blog about indexing => http://blog.logos.com/2009/11/the_power_of_indexing_explained/

Observation: have experienced indexing taking many hours so library preparation "extra time" on Logos support page is more than a bit understated, especially for OS X where indexing takes longer. In many sporting events, a time out tends to be quick so "extra time" fails to convey that hours may be needed for indexing. A friend using a MacBook Air has set Logos automatic downloads to No and chooses when to download resource(s) followed by indexing.

Personally Thankful for 567 free Vyrso eBooks since Thanksgiving 2011, which has provided many indexing opportunities. Thankful for program setting so can turn off automatic downloads and choose when to index.

But Logos is used by a lot of pastors, students and others who are on limited budgets or whose organizations are missional and so must watch every penny.

For technology, personally look at performance per penny for purchases. Thankful for a Dell Outlet hot deal last year so paid $ 722.80 for an Inspiron 17R Special Edition (i7-3610QM @ 2.30 GHz with 8 GB Ram, 1 TB hard drive at 5400 RPM), then added a 256 GB SATA III Solid State Disk (SSD) for $ 159.99 (Black Friday 2012) so my primary computer for running Logos 5 and Verbum has a hardware cost of $ 882.79 with decent performance.

My favorite Logos feature is visual filter highlighting that combines hundreds of search results for simultaneous display => http://wiki.logos.com/Extended_Tips_for_Highlighting_and_Visual_Filters#Examples_of_visual_filters Caveat: opening a resource with many visual filters enabled can take "extra time" to display all visual filter search results. Hence have learned at times to click once then wait while dreaming of sub-second response.

By the way, a new Verbum install with offline scan to copy licensed resources from Logos 5 then index took 31 minutes to create a Bible Index for 1,606 resources (plus 2 minutes to merge) and 159 minutes to create a library index for 11,908 resources (plus 7 minutes to merge). Overall indexing for initial library preparation took nearly four hours.

Keep Smiling [:)]

mike

https://community.logos.com/discussion/comment/547423#Comment_547423

This is really the point of my original post. If Logos says that my machine meets minimum requirements, then it shouldn't cause my system to grind to a halt every time it indexes. I appreciate that if I want to go out and buy a supercomputer it will run everything at lightning speed and I can look down my nose at all those that can't afford the latest and greatest tech.

But Logos is used by a lot of pastors, students and others who are on limited budgets or whose organizations are missional and so must watch every penny. It should not interfere with the normal operation of a machine that meets minimum standards.

This guy really knows what I'm thinking.

Logos shouldn't be really targeting towards people with high end computers. Consider those who needed the resources in a different environment. Not all pastors, students and missionaries on the field have i7 with 10k rpm hard drive or SSD.

Logos' logic: the more books you have, the more required you need to have a high end computer.

Get real ..

Lee

https://community.logos.com/discussion/comment/547500#Comment_547500

For technology, personally look at performance per penny for purchases. Thankful for a Dell Outlet hot deal last year so paid $ 722.80 for an Inspiron 17R Special Edition (i7-3610QM @ 2.30 GHz with 8 GB Ram, 1 TB hard drive at 5400 RPM), then added a 256 GB SATA III Solid State Disk (SSD) for $ 159.99 (Black Friday 2012) so my primary computer for running Logos 5 and Verbum has a hardware cost of $ 882.79 with decent performance.

<world-awareness>

1. In many regions, computer hardware can be 1.5 to 3 times as expensive as in America.

2. You would not want to be stuck in the middle of nowhere, with a souped up SSD that goes from 500 MB/s to 0 MB/s without any signs of impending failure.

3. Vista was panned worldwide. Three major factors: strange problems with drivers, speed improvements that only materialized on newest hardware, and premium prices. Sounds familiar?

</world-awareness>