Beta 5.3: Search improvement
The comparison comes from different computers (both 8 MB RAM) but the improvement obtained in 5.3 64-bit transcends the hardware differences:-
Basic Search for <Gen-Rev> in Bible Knowledge Commentary
64-bit 5.3 ---> 87304 results in 11s + c. 10s to display results. About 306 MB memory
32-bit 5.2b --> 87304 results in 210s + c. 3m to display results. About 395 MB memory
I don't think there has been any code optimization, and there was no re-indexing with 5.3. So this may well be attributed to the new SQLite Memory-mapped I/O!?
Dave
===
Windows 11 & Android 13
Comments
-
Searching my library is considerably faster here. 72,276 results in 25,468 articles and 1,827 resources in 0.70 seconds (a search for Qumran).
Showing the search results however, is slower. When, for example, I move to the next page in the search results, it takes 2.5 seconds to show the results of the next page.
0 -
Dave Hooton said:
So this may well be attributed to the new SQLite Memory-mapped I/O!?
That may help a little, but there are two major differences in search in 64-bits:
- Memory-map the postings file (the index) instead of using regular file I/O.
- Use RAM for temporary storage of results, instead of temp files. (We had to use temp files in 32-bit to avoid running out of memory if you were running lots of searches simultaneously.)
0 -
Would be interested to know the system spec differences any way.
Dave - would you mind doing a basic search across all resources for "the" on both? Its a pretty massive search.
L2 lvl4 (...) WORDsearch, all the way through L10,
0 -
FWIW, here's mine. MBP 2010 build, OS 10.9.4, 8 GB 1067 MHz DDR3, SSD 1 TB
Nearly 17.000 books in library.
MacOS Sierra / Logos 7 Collector's Edition & All Base Packages / Logos Now
0 -
abondservant said:
Dave - would you mind doing a basic search across all resources for "the" on both? Its a pretty massive search.
It would be meaningless as I don't have anywhere near 5900 resources! That's why I did the test on a single book that many users would have.
I'll do a comparison run after Logos let me know if 5.2b can run on the same machine as 64-bit 5.3.
Dave
===Windows 11 & Android 13
0 -
abondservant said:
Would be interested to know the system spec differences any way.
5.2b on first generation laptop Core i5 2.5 GHz, 180 GB SSD with 8 GB RAM
5.3 on second gen desktop Core i7 2.80 GHz, 250 GB 10,000 rpm HDD with 8 GB RAM.
Dave
===Windows 11 & Android 13
0 -
One could still generate a per resource average, which would be an interesting baseline.
L2 lvl4 (...) WORDsearch, all the way through L10,
0 -
Any way, searching simply what you did took nearly as much time as searching my entire library.
L2 lvl4 (...) WORDsearch, all the way through L10,
0 -
abondservant said:
One could still generate a per resource average, which would be an interesting baseline.
In 64-bit 5.3 the <Gen-Rev> search took 19s for 59 commentaries and 377686 results that included BKC, whereas BKC alone took 11s for 87304 results. The 2-volume EBC (also included in the 59 commentaries) took 9s for 76463 results. What is the meaningful average?
'the' on 5.2b took 0.17s for 129726 results in BKC. ---> laptop
'the' on 5.3 took 0.21s in BKC! ----> desktop
UPDATE:
'the' on 5.2b took 0.18s for 129726 results in BKC. ---> desktop!
Essentially the same time as the laptop.
Dave
===Windows 11 & Android 13
0 -
Dave Hooton said:
The comparison comes from different computers (both 8 MB RAM) but the improvement obtained in 5.3 64-bit transcends the hardware differences:-
Basic Search for <Gen-Rev> in Bible Knowledge Commentary
64-bit 5.3 ---> 87304 results in 11s + c. 10s to display results. About 306 MB memory ==> desktop 2.80 GHz Core i7 with 10000rpm HDD
32-bit 5.2b --> 87304 results in 210s + c. 3m to display results. About 395 MB memory ==> laptop 2.53 GHz Core i5 with SSD
On the same machine as Logos 5.3:-
32-bit 5.2b --> 87304 results in 242s
The laptop with SSD (Solid state Drive) beats the faster Core i7 desktop with a very fast HDD in this test.
Dave
===Windows 11 & Android 13
0 -
The laptop - also running 5.3 beta 1. was twice as slow.
1st gen i5 chip, 4gb ram, ssd87304 results, in 59 seconds.
Searching entire library for "the" required 37 seconds.
the <gen-rev> search is a bit more demanding. I ran <gen-rev> on my entire library on my desktop and after 10 minutes or so Logos crashed.L2 lvl4 (...) WORDsearch, all the way through L10,
0
