Bible search is unreliable

Bradley,

Thanks for the heads up on beta ... look forward to that.

While I am at this, I noticed that text in "notes" of the CUV Bible are not being searched like in the English Bibles. An example here: I am searching for "定命" (command in English) here and notice that there is no hit.

Thanks!

Oops ... sorry,

I meant Basis search ... see below, I search for Bible Text, Footnote Text and Translator's Note.

• I have "Bible Text" selected just to show that the search is a valid search
• Turn on "Footnote" and "Translator's note" ... yields no result ... specifically, missed the "定命" in the Footnote in Dan 2:5.

I suppose, I should test it after 7.2 is release ... Just to complete the various permutation, so that your tester(s) can tests these ... thanks

• First test "定"was not pickup in the notes of Dan 2:5
• Second test "命" picks up Dan 2:5, however "定命" missed, sounds like 7.2 should resolve this
• Third test "定" AND "命" missed
• Fourth test "定" WITHIN 2 WORDS "命" missed

Hi Bradley,

Let me know if I should move this to the Beta forum.

A search query consisting of consecutive Han characters without spaces is treated as an AND search for all of the characters, regardless of order, with the added consideration that any consecutive pairs will merged into a single search hit. E.g. if A, B, and C are Han characters, then the search query ABC will return hits in articles or verses that contain A, B, AND C in any order, including consecutively.To restrict the result to just ABC you could specify the query as a phrase search “ABC”, although for ranked search results you should use ABC INTERSECTS “ABC”. This causes the search to use the better rankings of the search results for ABC, while limiting the results to the same set as the phrase search.

This new search behavior in Beta is extremely confusing for Chinese users. It swings the pendulum from missing a lot of hits to returning too many hits. E.g. A search for 和平 ("peace") also returns verses with 和平安祭 ("and peace offering"). These results are confusing and misleading.

I think as a start, for Chinese searches you should at least do the same as Korean - default the search for ABC as ABC INTERSECTS "ABC".  A search for ABC XYZ should be interpreted as (ABC INTERSECTS "ABC") AND (XYZ INTERSECTS "XYZ").

In addition, for the indexer to blindly index any two consecutive characters is a flawed approach, for two reasons:

1. Many Chinese terms (especially transliterated names) contain 3, 4, 5 characters.

2. Two consecutive characters may not necessarily imply they are related as one unit of meaning (as seen in the above 和平安祭 example).

You almost need to have a dictionary of multi-character terms (耶和華，和平，平安祭) for the indexer to know which consecutive characters form legitimate units of meaning, and which do not, in the absence of spaces in CJK languages.

Many Chinese input methods (輸入法) provide such a "dictionary" (詞典) of multi-character phrases, but I'm not sure if they can be leveraged for this purpose (technically and legally). I know Sougou Input Method 搜狗輸入法 has many user-created dictionaries 細胞詞典 and many users have created Christianity and Bible-related ones for other users to use.

My few cents. Thanks for trying to solve this hard problem with CJK languages. Prior to this I have avoided using Logos for Chinese Bible searches all these years because I know the search results are completely unreliable. Now with the Chinese Bronze version pending, this problem comes to the forefront.

Thanks,

Peter

Hi Lawrence,

Thanks for your prompt response and for your working for CJK users. I deeply appreciate Logos paying attention and investing time and resources to this longstanding search reliability issue.

Before you make the change, I think we should hear the preference from other users first.

I spent some time playing with the Beta again, and it actually is more usable than I first thought:

- In most cases I can just do "ABC" instead of ABC INTERSECTS "ABC" if I don't care about ranking.

- I can easily solve the problem of 和平 finding 和平安祭 by searching for “和平” -平安祭 (the search engine even recognizes the curly quotation marks that my Chinese input method uses by default, which is very cool).

- Searching for 耶穌福音 (Jesus Gospel) finds verses with 耶穌 and/or 福音 in whichever order, which is fine. (With the proposed change, users will have to search for 耶穌 福音 which I'm not sure everyone is OK with. To find the exact string in that exact order, user will have to use quotation marks.)

- Searching for 和平 without quotation marks will find MANY extraneous verses with 和 (and) and 平 (flat) in whichever order, which is confusing and seems wrong, but this can be easily solved by using quotation marks, and it only happens with bigrams or trigrams where each of the characters is also a common word that can be used in totally different contexts. Such phrases may be rarer than I originally think. I also tried 主的恩 (Lord's grace) which has the same issue but again putting quotation marks solves it.

I also tried doing similar searches on other Chinese Bible search iOS apps or engines (e.g. Bible Gateway) and compare results. Looks like 和平 trips up almost all of them.

Other CJK users, please chime in?

Thanks again!

Peter

Thank you Philip and Lawrence (and Logos!)  Either implementation will be acceptable to me. By using quotation marks, your search results are already better than the other apps/websites I've tested.

I'm eagerly looking forward to the Bronze package for a full Chinese Bible software experience!

Thank YOU for all of your diligence, professionalism, and service to the global Church!

Peter

Philip,

I have not participated in beta for a long time ... after hearing about 7.2 from Bradley above, I finally install the 7.2 RC1

Chinese Search is finally working now[:)]

See my input on 2 words search as a string if I initiate the search from the CUV text in the following post in beta.

http://community.logos.com/forums/p/132973/864140.aspx#864140 

I just did. Thanks for cross-referencing that thread here. I missed that thread.

Thanks,

Peter

Lawrence,

I was looking for use cases to proof my point ... instead, I found these case that shows that your proposed search results is better [:)] for these 2 examples

• “一载、二载、半载” ... time, times and half a time
• "金、银、铜" ... Gold, Silver, bronze