Is There A Better Way

There have been a few threads where we have discussed finding text within search fields or generating verse lists as inputs to visual filters where the an * has been used as a search term resulting in searches that take a long time.

I have been reading the help files and the wiki to see if there is a better way to locate whole verses where this is the requirement i.e.

Find all the verses that are Words of Christ that contain QT Quotes

Apply Visual Filter to all verses in This Verse List

I have concluded that there is not a better way because both search and filters are designed to work at the text level and not the verse level, hence we need the *, but given that I could be wrong figured it was worth asking.

Find more posts tagged with

Comments

Kevin Becker

No there isn't, you've highlighted one of the basic limitations to Logos' functionality. Bradley has joked in the past that he might make this search even slower so we would stop doing it. Logos is aware of this issue, but I don't know what they are planning regarding enhancing/redesigning wildcard searches.

It's clear that users want a way to search for fields quickly. I know I do.

fgh

https://community.logos.com/discussion/comment/193248#Comment_193248

I don't know what they are planning regarding enhancing/redesigning wildcard searches

I don't know the first thing about software development, but surely it should be possible to at least cache search results? It's one thing to wait for a search the first time, it's quite another to have to wait for the same search every time you apply a filter, or open a resource with a filter applied. Right now you simply can't use filters with wildcards in them.

Also, it must be possible to speed up at least some wildcard searches considerably. I made a filter for Gen 1-2 which was something like "the * day". It took forever, even though the range was so small. Now, obviously. if Logos had first searched for 'day', and only then checked which instances of 'day' fulfilled all search criteria, that search would have taken fewer seconds than it now took minutes. So I'm pretty sure there's lots of time that could be saved here.

Mark Barnes

https://community.logos.com/discussion/comment/193272#Comment_193272

Logos caches the database/index files in memory, rather than caching results (as memory is limited, you can't do both). Their strategy will work best if you do lots of similar but not identical searches, which I suppose most of the time we do. Caching results only really works if you're constantly performing the same search, which is unusual, (except in visual filters perhaps).

That said, I would like to see a workaround implement that would speed up searches like "the * day", as that surely could be optimised logically, rather than by caching.

Graham Criddle

https://community.logos.com/discussion/comment/193272#Comment_193272

Also, it must be possible to speed up at least some wildcard searches considerably. I made a filter for Gen 1-2 which was something like "the * day". It took forever, even though the range was so small. Now, obviously. if Logos had first searched for 'day', and only then checked which instances of 'day' fulfilled all search criteria, that search would have taken fewer seconds than it now took minutes. So I'm pretty sure there's lots of time that could be saved here.

In this particular case, a different construct might be worth considering.

The search string "the BEFORE 2 WORDS day" gives a slightly different set of results when compared with the search string "the * day" due to it matching cases where there is no intervening word between "the" and "day" such as 1:18. However, it does return in under 2 seconds as opposed to for the wildcard search.

Even with the wildcard search you get a result which you probably didn't want (Gen 1:5) so in either case there is some filtering to do.

Not suggesting this is a general solution but it may help in some cases.

Graham

Dave Hooton

https://community.logos.com/discussion/comment/193281#Comment_193281

The search string "the BEFORE 2 WORDS day" gives a slightly different set of results when compared with the search string "the * day" due to it matching cases where there is no intervening word between "the" and "day" such as 1:18. However, it does return in under 2 seconds as opposed to for the wildcard search.

Even with the wildcard search you get a result which you probably didn't want (Gen 1:5) so in either case there is some filtering to do.

You get a better result with the BEFORE 2-2 words day (2-2 = exactly 2 words) but no highlighting of the intervening word. You get the same results with "the ?* day" (? ensures an intervening word) and the intervening word is highlighted, but it takes significantly longer!

Graham Owen

https://community.logos.com/discussion/comment/193272#Comment_193272

but surely it should be possible to at least cache search results

As I understand it Logos avoid storing/caching search results because of the dynamic nature of the library and the resources that it includes. I can understand this.

For Bible searches we can effectively store search results by saving the details as a Passage List but, to the best of my knowledge, one of the things that we are unable to do is use the passage list to directly define a visual filter hence the technique of searching for * using the passage list to define the search range. I guess in simple terms what would be useful is the option define a visual filter that applies to all of the text of the verses in a passage list.

fgh

https://community.logos.com/discussion/comment/193277#Comment_193277

(except in visual filters perhaps)

Exactly! (I was only really talking about filters.)

"the BEFORE 2 WORDS day"

You get a better result with the BEFORE 2-2 words day ... but no highlighting of the intervening word.

These strings are, unfortunately, useless for filters, for precisely that reason. [:(]

Even with the wildcard search you get a result which you probably didn't want (Gen 1:5) so in either case there is some filtering to do.

Not a problem in Swedish. [:)] (Not in Hebrew or Greek either, I believe.)

to the best of my knowledge, one of the things that we are unable to do is use the passage list to directly define a visual filter

Actually we can, and I thought it was a great idea. I did a search for "den * dagen" (with quotes); created a Passage List out of the results; and then redid the search using that Passage List, expecting a much faster result with only 7 verses chosen than with 35 -- but for some reason the time only went down with about 29%. Anyone has any idea as to why?

Mark Barnes

https://community.logos.com/discussion/comment/193386#Comment_193386

Actually we can, and I thought it was a great idea. I did a search for "den * dagen" (with quotes); created a Passage List out of the results; and then redid the search using that Passage List, expecting a much faster result with only 7 verses chosen than with 35 -- but for some reason the time only went down with about 29%. Anyone has any idea as to why?

I think because that when you do a search, the whole Bible is searched in the first pass, and then verses that match the restriction are filtered out in a second pass. If true, this does seem inefficient, but I cannot even begin to imagine how it could be coded to be done in any other way given how complex a passage list could be.