8.3 Beta 2. Awful indexing performance on my computer

Page 1 of 1 (7 items)
This post has 6 Replies | 1 Follower

Posts 411
George | Forum Activity | Posted: Fri, Feb 8 2019 12:53 PM

I added the free book from Faithlife and the indexing for the book took over 40 minutes.  It is true that my computer is not a top of the line computer with a fast SSD drive.  It is an older 8 core AMD 3.5 GHz CPU, 8 GB of RAM and a 2TB 7200 rpm hardrive. Still, over 40 minutes is pretty bad.

1351.LogosIndexer.zip

Posts 24572
Forum MVP
Dave Hooton | Forum Activity | Replied: Sat, Feb 9 2019 7:52 PM

Indexing took 45 min overall. Indexing for the book took 3 min. Merging the index work for that book with the rest of the index took 34 min, and the many SQLite warnings indicate that performance is slower than anticipated. The other important part,  Building the term/document info index took  6 min 36s.

The merge and the term/document info index (40 min combined) will take roughly the same time, no matter how many books are indexed. I think the badness is a reflection of your old CPU. You can check the consistency by re-indexing another resource. In the GO Box, type reindex ESV (or a similar resource). It should take longer, but only by a few minutes. Upload the log.

Dave
===

Windows & Android

Posts 13226
Forum MVP
Mark Barnes | Forum Activity | Replied: Sun, Feb 10 2019 2:02 AM

Dave Hooton:
I think the badness is a reflection of your old CPU

You're right that normally far more time is taken merging than indexing, and therefore it doesn't make a huge time difference whether you're adding 1 book or 20.

The biggest factor in the merge is the size of your library. Big libraries take longer. Nothing can be done about that. But I don't agree with you about CPU. The next biggest factor in this case will be drive speed. I have an old 8-core CPU, but a fairly fast SSD, and adding a resource to my index only takes about 15 minutes — even with a very large library.

Posts 24572
Forum MVP
Dave Hooton | Forum Activity | Replied: Sun, Feb 10 2019 11:52 AM

Mark Barnes:
But I don't agree with you about CPU.

Yes, I should also have mentioned the HDD (even though 7200RPM is better than the usual 5400 RPM).

Dave
===

Windows & Android

Posts 7923
LogosEmployee
Bradley Grainger (Faithlife) | Forum Activity | Replied: Mon, Feb 11 2019 11:35 PM

Adding a SSD to your system is probably the most significant upgrade you can make to improve the performance of Logos.

Your computer: (1m 16.9s) Indexing LLS:9780830878291
My computer: (453ms) Indexing LLS:9780830878291

Your computer: (34m 4.6s) Merging postings...
My computer: (11m 9.1s) Merging postings... [Note: I probably have a lot more resources than you do]

Your computer: (45m 10.5s) Indexing.
My computer: (18m 14.5s) Indexing. [You could probably expect to see 10 minutes (or less) with an SSD and a smaller library than I'm testing with]

Posts 411
George | Forum Activity | Replied: Tue, Feb 12 2019 2:52 PM

Bradley Grainger (Faithlife):
Adding a SSD to your system is probably the most significant upgrade you can make to improve the performance of Logos.

I've added a SSD just for Logos and, not surprisingly, indexing is faster.

Posts 24572
Forum MVP
Dave Hooton | Forum Activity | Replied: Tue, Feb 12 2019 3:02 PM

George:
not surprisingly, indexing is faster.

Smile

Dave
===

Windows & Android

Page 1 of 1 (7 items) | RSS