Glacial Search Speed

If someone else could run these searches and time them on their computer and post the results with their computer specs it might help. Thanks.

https://community.logos.com/discussion/comment/17202#Comment_17202

If someone else could run these searches and time them on their computer and post the results with their computer specs it might help. Thanks.

I'll check back later, when L4 is finished indexing (again). Since the search function directly uses the index, I wouldn't expect it to work very well until the indexing is done.

BTW, is your system finished indexing? If not, maybe that's the culprit.

Rick Brannan (Logos)

https://community.logos.com/discussion/comment/17202#Comment_17202

Hi Mark.

Beta 5, Bible Search, OT in NASB95, Grid results view (while indexing): 69 results in 69 verses, 21.43 sec.

Vista Business, 2Gigs RAM (thought I had more here on the office box ...) 2CPUs, each 2.66 GHz (If I read the systems specs properly).

- Rick

https://community.logos.com/discussion/comment/17207#Comment_17207

BTW, is your system finished indexing? If not, maybe that's the culprit.

No I have not downloaded the update to Beta 5 yet. I have a long download going on in the background, but that would affect the searches in 3.0 and 4.0.

Fred Greco

https://community.logos.com/discussion/comment/17208#Comment_17208

21.11 seconds for me

Windows7RC 64bit

Intel Core2 Duo 2.4GHz, 4GB RAM

https://community.logos.com/discussion/comment/17208#Comment_17208

Hi Mark.

Beta 5, Bible Search, OT in NASB95, Grid results view (while indexing): 69 results in 69 verses, 21.43 sec.

Vista Business, 2Gigs RAM (thought I had more here on the office box ...) 2CPUs, each 2.66 GHz (If I read the systems specs properly).

- Rick

Rick,

Thanks. Can you rerun the same search in 3.0 to see what your results would be? I know there is no built in timer, but on my machine the results appeared within a second or two after hitting the start search button. I'd be curious to see what you and others are getting.

https://community.logos.com/discussion/comment/17216#Comment_17216

21.11 seconds for me

Windows7RC 64bit

Intel Core2 Duo 2.4GHz, 4GB RAM

Fred, can you do the same search in 3.0 and estimate the time it takes (or time it)? My issue is the difference in how long searches take between 3.0 and 4.0. I know there will be differences between one machine and another for the same search in 4.0. Thanks.

Melissa Snyder

It took 15.97 secs on my system to find 69 results in 69 verses, during Indexing.

Intel Core2 Duo E6550 2.33Ghz, 4G RAM, Vista 32 Business.

Fred Greco

https://community.logos.com/discussion/comment/17220#Comment_17220

Mark,

5.3 seconds in Logos3. I was not indexing during either search.

https://community.logos.com/discussion/comment/17215#Comment_17215

No I have not downloaded the update to Beta 5 yet. I have a long download going on in the background, but that would affect the searches in 3.0 and 4.0.

Possibly. It depends on your setup, how much RAM, and what else you have going on the background (e.g., Google Desktop used to really slow down my old system), and how your anti-virus handles downloads (does it scan every packet, or the full file only after your downloader finishes writing it)? There are a few other memory hogs that can really bog down a system.

You could try pausing your download (if possible) and pausing or exiting some of these other programs and see what happens.

https://community.logos.com/discussion/comment/17225#Comment_17225

Mark,

5.3 seconds in Logos3. I was not indexing during either search.

Thanks, Fred.

I timed the search in 3.0 and it was posting results in under 3 seconds on my machine. I reran the search in 4.0 and it took 50.11 seconds. I have a download running in the background.

The issue seems to be the HUGE difference in speed between 3.0 and 4.0. Even with your speed in 4.0 you are about 4 times FASTER in 3.0. My differential is much greater. I just wonder why this search is so much slower with software that has indexed resources.

https://community.logos.com/discussion/comment/17225#Comment_17225

Mark,

5.3 seconds in Logos3. I was not indexing during either search.

Thanks, Fred.

I timed the search in 3.0 and it was posting results in under 3 seconds on my machine. I reran the search in 4.0 and it took 50.11 seconds. I have a download running in the background.

https://community.logos.com/discussion/comment/17221#Comment_17221

It took 15.97 secs on my system to find 69 results in 69 verses, during Indexing.

Intel Core2 Duo E6550 2.33Ghz, 4G RAM, Vista 32 Business.

Are folks at Logos allowed to run this search in 3.0? If so, what time does it take for you to run it? (So far you have the best posted results in 4.0.)

Mark

https://community.logos.com/discussion/comment/17228#Comment_17228

No I have not downloaded the update to Beta 5 yet. I have a long download going on in the background, but that would affect the searches in 3.0 and 4.0.

Possibly. It depends on your setup, how much RAM, and what else you have going on the background (e.g., Google Desktop used to really slow down my old system), and how your anti-virus handles downloads (does it scan every packet, or the full file only after your downloader finishes writing it)? There are a few other memory hogs that can really bog down a system.

You could try pausing your download (if possible) and pausing or exiting some of these other programs and see what happens.

Rich,

I will rerun these searches after the download is complete.

Are you saying that there are situations in which a search in 4.0 should be expected to run slower than the same search in 3.0? To me the issue is how much slower (relatively) this search ran in 4.0 versus 3.0.

https://community.logos.com/discussion/comment/17234#Comment_17234

Are you saying that there are situations in which a search in 4.0 should be expected to run slower than the same search in 3.0? To me the issue is how much slower (relatively) this search ran in 4.0 versus 3.0.

No, I'm not saying that at all. I'm only saying that background activity is likely to slow down searches generally.

I can't think of any reason the search would run faster in L3 than L4.

Ebbe Andréasson

https://community.logos.com/discussion/comment/17237#Comment_17237

It took 8.11 secs on my system to find 69 results in 69 verses (not indexing) in Beta 5.

In v3 it took 4.6 secs.

But if I instead of searching only OT, search all passages same search took 0.39 secs in Beta 5 (find 185 results in 182 verses).

Intel Core Duo 2, 2.00GHz, 3,36GB RAM, PAE (Physical Adress Extension). XP SP3.

https://community.logos.com/discussion/comment/17234#Comment_17234

13 sec in L4 & maybe 10 sec in L3.

L4 is glacial with a bible range specified (OT, NT, Lam, Matt-Rev) - a known indexing problem

More later ...

https://community.logos.com/discussion/comment/17202#Comment_17202

If someone else could run these searches and time them on their computer and post the results with their computer specs it might help. Thanks.

It took about 20 seconds the first time searching the OT.

It took 1.8 seconds searching the entire Bible.

Repeated the search, restricting to the OT and it took 18.90 seconds.

It's just as true in L4 as it was in L3 that searching the whole Bible takes much less time than searching part of it. As I understand the reason for this is that Logos first searches the entire Bible, and then filters the search by the passage selection.

Interestingly searching the NIV took significantly longer than the NASB95, even though the latter is a rev. int.

https://community.logos.com/discussion/comment/17252#Comment_17252

It took about 20 seconds the first time searching the OT.

It took 1.8 seconds searching the entire Bible.

Repeated the search, restricting to the OT and it took 18.90 seconds.

It's just as true in L4 as it was in L3 that searching the whole Bible takes much less time than searching part of it.

Funny thing is the search I did in the OT in 4.0 took less time than a search in the whole Bible.

I'm still looking for comparative numbers between 3.0 and 4.0. Seems we should see an increase in speed which I am not seeing. So far those who've run searches in both versions are getting faster searches in 3.0.

I'm downloading Beta 5 now and will have to re-index, then I'll check my speeds again.

https://community.logos.com/discussion/comment/17251#Comment_17251

13 sec in L4 & maybe 10 sec in L3.

L4 is glacial with a bible range specified (OT, NT, Lam, Matt-Rev) - a known indexing problem

Dave, thanks for the help. You have the record for fastest search time in 4.0 so far, but your unindexed 3.0 is faster. Do you have a delay in seeing the results in 4.0 that adds significant time to the overall search process? I seem to.

Also, what are your machine and software specs?

https://community.logos.com/discussion/comment/17232#Comment_17232

Are folks at Logos allowed to run this search in 3.0?

HMMM. Strangely silent in Bellingham. Melissa? Rick? Are can you run this search for comparison in 3.0?

https://community.logos.com/discussion/comment/17254#Comment_17254

It took about 20 seconds the first time searching the OT.

It took 1.8 seconds searching the entire Bible.

Repeated the search, restricting to the OT and it took 18.90 seconds.

It's just as true in L4 as it was in L3 that searching the whole Bible takes much less time than searching part of it.

Funny thing is the search I did in the OT in 4.0 took less time than a search in the whole Bible.

I'm still looking for comparative numbers between 3.0 and 4.0. Seems we should see an increase in speed which I am not seeing. So far those who've run searches in both versions are getting faster searches in 3.0.

I'm downloading Beta 5 now and will have to re-index, then I'll check my speeds again.

As Rich says, searching part of a bible can take longer than searching the whole. But the problem is exacerbated in L4 with indexing significantly speeding library searches whilst their algorithm for partial searches is (now?) proving to be particularly inefficient. Some partial searches have taken 5-15 minutes on my laptop, with Grid searches significantly slower than Verses - see http://community.logos.com/forums/p/1832/14028.aspx#14028 (I just ran it in B4 - Verses took 8 sec and Grid took 7.5 min).

The difference in results with your phrase search is because:-

v4 has 2 extra results in Lam1:20 & 2:11 because the footnote characters upset v3's separation algorithm whilst v4 works with actual word separation.

v3 has 1 extra result at Dan 5:14 "spirit of the gods" because of v3's separation algorithm (picking up the) AND because it defaults to a stemming search whilst v4 defaults to a nostem search (you ask for "god" and you won't get "gods" unless you check Match all word forms in the dropdown under the Search icon!).

https://community.logos.com/discussion/comment/17256#Comment_17256

13 sec in L4 & maybe 10 sec in L3.

L4 is glacial with a bible range specified (OT, NT, Lam, Matt-Rev) - a known indexing problem

Dave, thanks for the help. You have the record for fastest search time in 4.0 so far, but your unindexed 3.0 is faster. Do you have a delay in seeing the results in 4.0 that adds significant time to the overall search process? I seem to.

Also, what are your machine and software specs?

The v3 time includes displaying the results and it just did it in under 4 sec after a restart, whilst v4 took about 12 sec to display results after a restart.

Machine specs are with my signature!

https://community.logos.com/discussion/comment/17232#Comment_17232

Are folks at Logos allowed to run this search in 3.0?

No, it's against company policy. But if you don't tell Bob...

For your original search (in an index of Scholar's Gold), my computer took 23.1s. As mentioned elsewhere on the forums, limiting to a verse range (e.g., Old Testament) is (much) slower than searching All Passages. An All Passages search took 5.95s.

The search engine also hasn't been optimised yet for searches that involve very frequent words. To show the difference, I rephrased the query to remove "of" and "the":
"My Spirit" OR (Spirit BEFORE 2-2 WORDS God) OR (Spirit BEFORE 3-3 WORDS Lord) OR "Holy Spirit"
This returns some unwanted hits (e.g., "spirit from God") but runs faster: 15.75s for "Old Testament" and 0.68s for "All Passages".

The original search in LDLS3 runs in under a second (I'd guess about 0.85s from my stopwatch). So, we have some work to do to match that, but you can see that fast searches are achievable in Logos 4 (by choosing the right settings and tweaking the query phrasing). Our goal is to get to that speed by default, without having to make you jump through hoops. Thanks for helping identify the current bottlenecks during the beta test. You might remember that it took three major releases of Libronix DLS to bring you "Bible Speed Search"; we hope it won't take quite as long with Logos 4... [;)]

https://community.logos.com/discussion/comment/17307#Comment_17307

No, it's against company policy. But if you don't tell Bob...

Ah, the veil has been lifted.

I have indexing going on right now so can't verify what you and others have found in searching the whole Bible vs the OT. My initial results were in the range of 1.5 times longer to search the whole Bible. But with a large download going on at the same time, I can't assume that was accurate.

Anyway, the differences between 3.0 and 4.0 right now are dramatic. I thought that I'd see fractions of a second in 4.0 with all the time spent on indexing. Rendering the results also takes more time in 4.0 than 3.0. I will redo my tests tomorrow once everything is re-indexed and I'm on Beta 5.

I do hope that search speed for sections or selections can be brought into line with 3.0 times. I do these far more than any other search and quite often want to see the hit counts as much as the results. So not be able to do that as quickly as in 3.0 would be a definite downer for me.

Perhaps my set-up is not robust enough to get the best times in 4.0.

Thanks, as always, for informing us of what is going on and what to look forward to.

https://community.logos.com/discussion/comment/17298#Comment_17298

Machine specs are with my signature!

Sorry, Dave. I needed to put my specs on to see your specs. Other than the difference in operating systems our two computers' specifications are identical.

Thanks for the comparison data. Your v4 data are definitely faster than mine. I'll retest tomorrow and see how I do.

https://community.logos.com/discussion/comment/17307#Comment_17307

For your original search (in an index of Scholar's Gold), my computer took 23.1s. As mentioned elsewhere on the forums, limiting to a verse range (e.g., Old Testament) is (much) slower than searching All Passages. An All Passages search took 5.95s.

The latest code on my computer now executes your original search in 6.10s (for the "Old Testament" range)

"My Spirit" OR (Spirit BEFORE 2-2 WORDS God) OR (Spirit BEFORE 3-3 WORDS Lord) OR "Holy Spirit"
This returns some unwanted hits (e.g., "spirit from God") but runs faster: 15.75s for "Old Testament" and 0.68s for "All Passages".

This now runs in 0.72s (again, for "Old Testament").

https://community.logos.com/discussion/comment/17332#Comment_17332

Note that there's still some more optimisation work to be done: "spirit" in Top Bibles takes 0.21s for a "All Passages" search, but 1.67s for "Bible" (with the new code).

https://community.logos.com/discussion/comment/17332#Comment_17332

The latest code on my computer now executes your original search in 6.10s (for the "Old Testament" range)

That would be much better. Here are my results now that I have Beta 5 and have finished reindexing. They are still considerably slower than yours and those of most everyone else who's reported:

38.18 seconds. Much slower than what others are reporting. I timed in 3.0 and it was about 6 seconds. BTW: a second run in 3.0 brought the time to about 1 second. Perhaps I had some overhead loading the search software on the first try. The difference in speed between 3.0 and 4.0 is still substantial on my machine.

3.19 seconds. Pretty fast but as I said, I want the hit counts in just the OT so wouldn't have chosen this search.

'Bible' search:

40.41 seconds for just 'Bible'. Doing this in 3.0 took less than 2 seconds.

Finally:

This is shorter than either OT or Bible search and longer than All Passages. I just wonder why an All Passages search should be faster. Isn't All Passages the same as Bible and Apochrypha?

So, my speeds seem to be quite a bit slower than some others here, including Dave Hooton's whose computer specs are very much like mine. In any case the real issue remains the inordinate increase in search time for 4.0 over 3.0 for identical searches. Really have to say I hope this is ironed out.

https://community.logos.com/discussion/comment/17455#Comment_17455

3.19 seconds. Pretty fast but as I said, I want the hit counts in just the OT so wouldn't have chosen this search.

Why not?

In L3 I have often searched the whole Bible, even when I only wanted results in one book, because it was faster to do the full search and then scroll down to my desired location.

I just wonder why an All Passages search should be faster. Isn't All Passages the same as Bible and Apochrypha?

Yes, kind of curious. I suppose it's because there are passages in the LXX that don't appear in the accepted "Apocrypha." Is that right? Or perhaps there's room for future expansion for books included in some canons that are excluded from both the Roman Catholic and the Protestant cannon.

So, my speeds seem to be quite a bit slower than some others here, including Dave Hooton's whose computer specs are very much like mine.

This can be due to a number of factors, including whether or not you regularly defrag your hard drive, if you have Google Desktop (or MS indexing service) running in the background, how your anti-virus deals with file accessing, as well as a host of more esoteric hardware specifications, and background system services that may be running. These can be notoriously subtle to track down, let alone resolve.

But you're right. While L4 does much better at searching our full library, it still struggles in doing searches in the Bible.

https://community.logos.com/discussion/comment/17455#Comment_17455

3.19 seconds. Pretty fast but as I said, I want the hit counts in just the OT so wouldn't have chosen this search.

Beta 6 should make an "Old Testament" or "Bible" search almost as fast as "All Passages". I would expect your 38.18 second search "Old Testament" to take around 3.5s with Beta 6. (The "Bible", and "Bible and Apocrypha" searches should run in this kind of time, too.)

https://community.logos.com/discussion/comment/17465#Comment_17465

Why not?

In L3 I have often searched the whole Bible, even when I only wanted results in one book, because it was faster to do the full search and then scroll down to my desired location.

I have, too. Sometimes I want the hit counts. I want the number of times the word or phrase appears in whatever section I am interested in. I don't want to have to count them manually. That's why I do searches on specific sections, books, or even chapters.

This can be due to a number of factors, including whether or not you regularly defrag your hard drive, if you have Google Desktop (or MS indexing service) running in the background, how your anti-virus deals with file accessing, as well as a host of more esoteric hardware specifications, and background system services that may be running. These can be notoriously subtle to track down, let alone resolve.

You've given me some things to check. Sure would like to at least duplicate the speeds others are reporting in 4.0. Thanks, Rich.

https://community.logos.com/discussion/comment/17500#Comment_17500

Beta 6 should make an "Old Testament" or "Bible" search almost as fast as "All Passages". I would expect your 38.18 second search "Old Testament" to take around 3.5s with Beta 6. (The "Bible", and "Bible and Apocrypha" searches should run in this kind of time, too.)

Great. That is welcome news. Thanks for letting us know what's coming up.

Fred Greco

https://community.logos.com/discussion/comment/17500#Comment_17500

3.19 seconds. Pretty fast but as I said, I want the hit counts in just the OT so wouldn't have chosen this search.

Beta 6 should make an "Old Testament" or "Bible" search almost as fast as "All Passages". I would expect your 38.18 second search "Old Testament" to take around 3.5s with Beta 6. (The "Bible", and "Bible and Apocrypha" searches should run in this kind of time, too.)

[Y] Great news!

Ebbe Andréasson

https://community.logos.com/discussion/comment/17455#Comment_17455

3.19 seconds. Pretty fast but as I said, I want the hit counts in just the OT so wouldn't have chosen this search.

My search is faster but how come we have different search results in same resource?

https://community.logos.com/discussion/comment/17747#Comment_17747

My search is faster but how come we have different search results in same resource?

Ebbe,

I guess it takes an extra 2.24 seconds to dig up those last 3 hits.

I'm kidding.

Very curious. Have to say I like your search speed results better than mine, though.

Wonder if Logos has an idea how the results could differ?

https://community.logos.com/discussion/comment/17753#Comment_17753

Very curious. Have to say I like your search speed results better than mine, though.

Wonder if Logos has an idea how the results could differ?

It depends how you configure these options from the little arrow under the Search icon!

The tick gets 188. Remove it and you get 185!

Tom Reynolds

https://community.logos.com/discussion/comment/17254#Comment_17254

Sorry it took so long for me to post - I was waiting to finish indexing [:)].

It took L4 a whopping 25.67 seconds to find 69 results.

Core 2 Duo P7570 2.0 GHz with 3GB RAM and Vista Business.

In Logos3 it took less than three seconds from the time I pushed the button but it only found 68 occurrences. This difference stems from the fact that

L4 found two occurrences in Lamentations (1:20; 2:11) that L3 did not.

L3 found Daniel 5:14 but L4 did not.

I think these differences in results is more disturbing than the speed with which it finds them, although why L4 is glacially slow needs to be addressed, of course. This is a good reason to keep both versions and run searches twice to be certain!