NEW Search Engine for 7.3 Beta 3

Page 1 of 1 (19 items)
This post has 18 Replies | 2 Followers

Posts 2322
LogosEmployee
Philana R. Crouch | Forum Activity | Posted: Thu, Dec 1 2016 2:58 PM

New Search Engine

In 7.3 Beta 3, a new search engine that has been optimized for common searches has been added for testing.

How to Turn it On/Off

The new search engine is not enabled by default in 7.3.

  • Use the command Set Use New Search Engine to Yes to turn it on.
  • Use the command Set Use New Search Engine to No to turn it off.
  • Tip: creating shortcut for both commands will make switching back and forth easier.

What to Expect

  • A short period of indexing (possibly up to 10 minutes) will take place when Beta 3 is installed (to build an additional file that the new search engine needs).
  • It supports Basic Ranked, Basic By Resource, and Bible searches.
    • Basic By Count, Morph, Syntax and Clause searches are not supported by the new search engine.
    • Searching within Markup or Reference Ranges (Bible Search) is not supported.
  • Ranked search will not display full result counts for many searches.
    • Displaying full or approximate result counts may be added in a future release; it will not be in the initial implementation.
  • Only basic search operators (multiple terms, phrases, AND, comma) are supported.
    • Advanced search operators (proximity, INTERSECTS, ANDEQUALS) will use the old search engine.
    • Due to the way Chinese and Korean are indexed, the new search engine will not be used for CJK searches.
  • The new search engine will automatically fall back to the existing search engine if it can't support the current query or search type. Thus, no search functionality is removed by enabling the new search engine.
    • It's not always easy to tell whether the new search engine or the old one was used for a particular search.
  • Some Guide sections (that use simple searches as described above) will benefit from the new search engine if it's enabled.
  • The new search engine requires a 64-bit operating system (32-bit Windows users are not supported).

Known Issues

  • Cited By may show "-1 results" when "Sort results by search rank" is enabled in the Cited By panel menu.
Posts 13062
Forum MVP
Mark Barnes | Forum Activity | Replied: Thu, Dec 1 2016 5:32 PM

A quick test of this suggests that for simple searches, the reported performance is significantly quicker. As a minimum it's about three times quicker than the old search. For searches with millions of results, it might be 30 times quicker.

I notice small discrepancies in a "By Resource" count (158,789 results vs. 158,613 results in one test, 2,061,881 vs 2,061,665 in another, 12,903,925 vs 12,893,978 in a third).

I find it odd that we can't get counts with a Ranked new search. You can run the "Ranked" and the "By Resource" search quicker than the old "Ranked Search" on its own. When necessary, why not run a background "By Resource" search to get the counts?

When running searches with relatively few results (thousands, not millions) "Your books" can be slower under the new search engine (about half the speed of before).

Posts 7895
LogosEmployee

Mark Barnes:
I notice small discrepancies in a "By Resource" count (158,789 results vs. 158,613 results in one test, 2,061,881 vs 2,061,665 in another, 12,903,925 vs 12,893,978 in a third).

The new search engine will count multiple occurrences within one footnote as a single "hit", whereas the old search engine would count them multiple times. This should have been in the release notes; I'll update them.

(If you notice hit count discrepancies when searching just surface text, that could be a problem.)

Posts 145
Dustin Pearson | Forum Activity | Replied: Thu, Dec 1 2016 9:45 PM

Can an indication be made if the search feature is on? Maybe like the update channel status in the "About" box at least?

I really dislike "switches" with no indication as to status.

Posts 978
Richard Wardman | Forum Activity | Replied: Fri, Dec 2 2016 2:17 AM

Also spotted:

Or was that already a thing?

Posts 3591
Francis | Forum Activity | Replied: Fri, Dec 2 2016 2:45 AM

Can you, in a nutshell, explain what the new search engine will do as compared to the current one? Are we to expect (when it is fully developed and implemented) only search speed improvements or are there also other performance enhancements (e.g., a better ranking algorithm) or added/different functionalities?

Posts 24347
Forum MVP
Dave Hooton | Forum Activity | Replied: Fri, Dec 2 2016 3:02 AM

Bradley Grainger (Faithlife):
(If you notice hit count discrepancies when searching just surface text, that could be a problem.)

No problem there. For simple word searches ('the") with c. 30 millions results it is 2x-2.5x faster but see these results are greater!

Otherwise the difference in real world searches is not significant

EDIT: did not try Ranked searches.

Dave
===

Windows & Android

Posts 13062
Forum MVP
Mark Barnes | Forum Activity | Replied: Fri, Dec 2 2016 3:25 AM

Dave Hooton:
Otherwise the difference in real world searches is not significant

Ranked search is much more improved than "By Resource". Also, the more results, the greater the difference. A ranked search for "the" in my library gives 2.23s compared to 60.68s (x27). For "By Resource", it's 19.07 vs 82.48 (x4).

Posts 24347
Forum MVP
Dave Hooton | Forum Activity | Replied: Fri, Dec 2 2016 3:56 AM

Mark Barnes:
Also, the more results, the greater the difference.

Yes, but my "real world" simple searches were not better than 2x faster with old search engine taking no more than 1 seconds (377k results).

Dave
===

Windows & Android

Posts 7895
LogosEmployee

Francis:

Can you, in a nutshell, explain what the new search engine will do as compared to the current one? Are we to expect (when it is fully developed and implemented) only search speed improvements or are there also other performance enhancements (e.g., a better ranking algorithm) or added/different functionalities?

You can expect speed improvements for now. We'll announce other changes when they become available.

Posts 7895
LogosEmployee

Mark Barnes:
When running searches with relatively few results (thousands, not millions) "Your books" can be slower under the new search engine (about half the speed of before).

I haven't been able to reproduce this. 

What kind of slowdown are you seeing: 0.05s to 0.1s or 1s to 2s?

A screenshot might be helpful.

Posts 15805
Forum MVP
Keep Smiling 4 Jesus :) | Forum Activity | Replied: Mon, Jan 9 2017 10:33 AM

Philana R. Crouch:
Ranked search will not display full result counts for many searches.

Follow-up to thread => Suggestion: Bundle this with all Logos packages and updates

Replicated Rank showing 1-100 for search using Logos 7.4 Beta 1:

advil,ibuprofen,tylenol,acetaminophen

Changing to "By Resource" or "By Count" shows 337 results in 230 articles.

Changing back to Ranked and clicking next shows 307 results in 230 articles

Noted 230 articles is consistent while 307 and 337 results are a bit different.

Keep Smiling Smile

Posts 7895
LogosEmployee

Keep Smiling 4 Jesus :):

Replicated Rank showing 1-100 for search using Logos 7.4 Beta 1:

This is by design.

Keep Smiling 4 Jesus :):

Changing to "By Resource" or "By Count" shows 337 results in 230 articles.

Changing back to Ranked and clicking next shows 307 results in 230 articles

Noted 230 articles is consistent while 307 and 337 results are a bit different.

I can reproduce something similar and will file a bug.

Suggestion: add "paracetamol" to your list of drug synonyms Smile

Posts 7895
LogosEmployee

Bradley Grainger (Faithlife):
I can reproduce something similar and will file a bug.

This will be fixed in the next beta.

Posts 15805
Forum MVP
Keep Smiling 4 Jesus :) | Forum Activity | Replied: Mon, Jan 9 2017 11:57 AM

Bradley Grainger (Faithlife):

Bradley Grainger (Faithlife):
I can reproduce something similar and will file a bug.

This will be fixed in the next beta.

Thanks plus expanded search by another term:

advil,ibuprofen,tylenol,acetaminophen,paracetamol,naproxen

Keep Smiling Smile

Posts 2322
LogosEmployee
Philana R. Crouch | Forum Activity | Replied: Mon, Jan 16 2017 2:54 PM

Keep Smiling 4 Jesus :):

Changing to "By Resource" or "By Count" shows 337 results in 230 articles.

Changing back to Ranked and clicking next shows 307 results in 230 articles

Noted 230 articles is consistent while 307 and 337 results are a bit different.

Keep Smiling Smile

This should be fixed in 

7.4 Beta 3 (7.4.0.0014)

Posts 80
EX | Forum Activity | Replied: Mon, Jan 16 2017 6:05 PM

I love this new search engine. Thank you Logos!

PCA Church
L4 Platinum, L5 Reformed Platinum, L6 Reformed Diamond, Reformed Studies XL, Platinum, Logos Now

Posts 696
Armin | Forum Activity | Replied: Sun, Feb 19 2017 8:30 PM

Richard Wardman:

Also spotted:

Or was that already a thing?

I am also curious what the "new test editor" is. Does anybody know about it?

Posts 24347
Forum MVP
Dave Hooton | Forum Activity | Replied: Mon, Feb 20 2017 11:53 AM

Probably from testing the new Rich Text Editor introduced in v5.1.

Dave
===

Windows & Android

Page 1 of 1 (19 items) | RSS