BUG: Search not working when question mark included
I was doing some searching last night and I discovered what appears to be a problem. I am looking at here, the book "The Jesus I Never Knew" by Philip Yancey. I was doing searches and not finding the verse I was looking for; actually I was doing this on my iPad. So I figured there must be a problem with the iPad app, and I tried this morning on my desktop (5.0b). If you notice the picture, the search finds no results even though it should have found at least 1! I tried typing the phrase, I tried copying it from the resource. Search is supposed to work in eBooks, correct? This is a Vyrso eBook.
Note if I search for just a word, like "missed" (no quotes), it works.
Comments
-
OK I just tried a few other eBooks and search works fine - it's just this one. Is the resource broken?
0 -
Don Awalt said:
Search is supposed to work in eBooks, correct?
Yes it should
Don Awalt said:I tried typing the phrase, I tried copying it from the resource.
What happens if you select the phrase in the resource, right-click and choose "search this resource"?
It sounds like it might be an indexing problem - but a strange one.
It will be interesting to see if others with the resource get the same problem. I don't have it so can't check
0 -
Graham Criddle said:
What happens if you select the phrase in the resource, right-click and choose "search this resource"?
Ah, excellent tip to try! I did, and it also comes up with 0 results! Something is definitely broken. I wonder if there is a way to figure out if it is a bug or a bad resource?
I recall that I have the same problem on my iPad, does that mean the resource is bad on the server? Or maybe a bug that is in both versions?
0 -
Some phrases work, some do not. I found one place where if I highlight the complete sentence, which includes a question mark, it fails; leave out the question mark, it succeeds.
0 -
Don Awalt said:
I recall that I have the same problem on my iPad, does that mean the resource is bad on the server? Or maybe a bug that is in both versions?
If it was only on the desktop, I'd have suspected a bad index or download (in this case, hiding the book, restarting Logos [possibly even verifying deletion of the *.logos4-file], unhiding the book thus forcing redownload and re-index), but since you saw the issue on the pad it may well be that the original resource file is somehow broken. Note that these are ePub-files Logos gets from the publisher and treats only in automated run with a modified version of the PB compiler.
To narrow this down, if you like, try to give less information to the search, such as leaving off the questionmark or searching for the words of the sentence, not the phrase. Maybe there's a non-printing character somewhere in the spaces of the sentence. Or the questionmark is another UNICODE-character. looking same but having another internal number than the one you type in (we had issues with straight and curved quote markers in the past).
Have joy in the Lord!
0 -
Don Awalt said:
I recall that I have the same problem on my iPad, does that mean the resource is bad on the server? Or maybe a bug that is in both versions?
At least it suggests it isn't an indexing problem in your Logos 5 environment[:)]
My guess is that it is a resource problem but I wouldn't know how to prove that.
Hopefully someone else with the resource can verify or Logos may comment later.
0 -
Don Awalt said:
Some phrases work, some do not. I found one place where if I highlight the complete sentence, which includes a question mark, it fails; leave out the question mark, it succeeds.
Your post wasn't there when I posted mine above. Seems the questionmarks are the culprit. Can you verify this against another resource - or two, such as a Vyrso book and a regular Logos resource?
Then we'd know if phrase search is broken everywhere for phrases with a questionmark in it (serious bug!) or only for specific representations of questionmarks such as in this resource or in the Vyrso books.
Have joy in the Lord!
0 -
Hi Mick
NB.Mick said:Your post wasn't there when I posted mine above. Seems the questionmarks are the culprit. Can you verify this against another resource - or two, such as a Vyrso book and a regular Logos resource?
Then we'd know if phrase search is broken everywhere for phrases with a questionmark in it (serious bug!) or only for specific representations of questionmarks such as in this resource or in the Vyrso books.
Good questions - and it looks like question mark searches don't work in either type of resource. The first screenshot is from a Vyrso book, the second is from a Logos book
0 -
Yes, good job! It appears the question mark breaks searches! I too tried about 5 Vyrso books and no search with a question mark works, search this resource without the question mark and it finds it.
And it DOES seem like ? breaks full Logos resources too! Yikes!!
0 -
Hang on - just worked out what is going on here[:)]
The question mark is being treated as a wildcard character
See http://wiki.logos.com/Detailed_Search_Help#Wildcards which states
"A ? stands for one character in a word. So s?n would find sin, son and sun. It cannot mean zero characters, so neighbo?r would find neighbour, but not neighbor."
So since there isn't a character where the "?" is the search doesn't match.
So it is working as designed - if not as expected
0 -
Graham Criddle said:
Hang on - just worked out what is going on here
The question mark is being treated as a wildcard character
See http://wiki.logos.com/Detailed_Search_Help#Wildcards which states
"A ? stands for one character in a word. So s?n would find sin, son and sun. It cannot mean zero characters, so neighbo?r would find neighbour, but not neighbor."
So since there isn't a character where the "?" is the search doesn't match.
So it is working as designed - if not as expected
No, that's not the reason, since wildcard characters don't (or at least should never ever) work inside phrases. And of course there is a character where the ? is in the search phrase: the questionmark.
I played a bit with the search in biblia, confirming what you found, too. Since I seemed to remember information from long ago that the Logos index doesn't include punctuation at all, I tried with periods. Actually Logos will find the sentence with a period. I will find it as well if you omit the period (or the questionmark) and have a search string where the punctuation is inside the string. But it won't act on the questionmark as if it was a wildcard and it will give zero hits always.
Have joy in the Lord!
0 -
NB.Mick said:
Since I seemed to remember information from long ago that the Logos index doesn't include punctuation at all, I tried with periods. Actually Logos will find the sentence with a period
I had seen this too - I did searches with semi-colons etc. and I noticed the result come back correctly, but the semi-colon was not highlighted, as if it was ignored in the search.
0 -
Don Awalt said:NB.Mick said:
Since I seemed to remember information from long ago that the Logos index doesn't include punctuation at all, I tried with periods. Actually Logos will find the sentence with a period
I had seen this too - I did searches with semi-colons etc. and I noticed the result come back correctly, but the semi-colon was not highlighted, as if it was ignored in the search.
You may omit the punctuation or have it inside - both ways find the hit. But if you include a questionmark it will give zero results. You should change the thread title to sart with "BUG:" to alert Bradley or another Logos-employee when they screens the forum posts.
Have joy in the Lord!
0 -
NB.Mick said:
No, that's not the reason, since wildcard characters don't (or at least should never ever) work inside phrases. And of course there is a character where the ? is in the search phrase: the questionmark.
But the question mark - as a wildcard character - does work inside phrases. See the example below (just taken at random from a book I am reading)
Here the "?" is matching the "n"
NB.Mick said:Since I seemed to remember information from long ago that the Logos index doesn't include punctuation at all,
Yes, this is key information.
NB.Mick said:But it won't act on the questionmark as if it was a wildcard and it will give zero hits always.
Because - as the wiki article states - " It cannot mean zero characters" and so because there isn't a character immediately after the rest of the phrase it won't find a hit.
I think it makes sense[:)]
0 -
Don Awalt said:
I had seen this too - I did searches with semi-colons etc. and I noticed the result come back correctly, but the semi-colon was not highlighted, as if it was ignored in the search.
Yes punctuation is not searchable - as discussed at http://community.logos.com/forums/t/22626.aspx (and probably elsewhere)
0 -
Graham Criddle said:NB.Mick said:
But it won't act on the questionmark as if it was a wildcard and it will give zero hits always.
Because - as the wiki article states - " It cannot mean zero characters" and so because there isn't a character immediately after the rest of the phrase it won't find a hit.
I think it makes sense
No it doesn't. If it means one character and ? is omitted, the questionmark should resolve the subsequent space as a hit.
Have joy in the Lord!
0 -
Okay.
From testing a bit more on biblia (I'm not at my Logos-PC right now) it seems, the bug might be described as
While the questionmark works as a wildcard to find one character (counterintuitively even within phrases), it doesn't work when the respective position is an omitted punctuation character (such as a questionmark!) i.e. would be the subsequent character after the omitted punctuation.
At least in biblia, it seems to be buggy regardless of phrase or non-phrase, and the omitted character may as well be another, such as a colon. Maybe someone can confirm this is implemented in L5 likewise. And we wait for Bradley to turn on the light.
Have joy in the Lord!
0 -
NB.Mick said:
While the questionmark works as a wildcard to find one character (counterintuitively even within phrases),
Logos does not index punctuation characters, and they are ignored during a search e.g. "lord jesus christ" will find "Lord, Jesus Christ" in 1 Cor 8:6 (ESV). If you try to search for "lord, jesus christ" the comma is also ignored and you get the same results as before. If you search for "lord? jesus christ" you get zero results because the search engine is looking for a single non-punctuation character immediately after "lord" e.g. "lords jesus christ".
If you search for "lord ? jesus christ" the engine is looking for a one character word (e.g. "a") following lord, and fails. But a search for "lord ??? christ" will find "lord and christ" because it is looking for a 3 character word following lord.
Now, whilst "lord? jesus christ" failed, a search for "lord* jesus christ" will succeed, giving the same results as "lord jesus christ" in this case. The * wildcard looks for a sequence of zero or more non-punctuation characters.
I trust that illustrates the purpose of a wildcard in a phrase (a wildcard character won't match punctuation!).
The engine does allow matches where punctuation is part of a word e.g. enter God and you will see a suggestion for God's. One will search strictly for "God", the latter will look only for "God's". Match all word forms will search for both (as well as "gods").
Finally, Lord BEFORE 3-3 CHARS “Jesus Christ” will find only “Lord, Jesus Christ” (in ESV) because the CHARACTER proximity (exactly 3 chars) includes the comma! Ergo, Lord BEFORE 2 CHARS “Jesus Christ” will find “Lord Jesus Christ” (without the comma).
EDIT: (subtle refinement)
Wildcard will search for apostrophe when part of a word e.g God?s will find God's
Dave
===Windows 11 & Android 13
0 -
I recently wanted to search the Sermon on the Mount for "?" to find every instance where Jesus asked a question. It did not work because of the role that "?" plays as a wildcard. Is there a way to perform the search for a question?
Thanks,
David
0 -
Hi David - and welcome to the forums
You can search for "interrogative pronouns" using a morph search
This might give you what you want - although it seems to give a couple of results which I wouldn't have expected.
Graham
0 -
David Scott said:
I recently wanted to search the Sermon on the Mount for "?" to find every instance where Jesus asked a question. It did not work because of the role that "?" plays as a wildcard. Is there a way to perform the search for a question?
I wouldn't trust translations to really find every instance where Jesus asked a question - probably Louw-Nida domains could cover this.
However, to just quickly finde very ? charcter in a resource, I'd use the Ctrl-F text search.
Have joy in the Lord!
0 -
Dave Hooton said:NB.Mick said:
While the questionmark works as a wildcard to find one character (counterintuitively even within phrases),
Logos does not index punctuation characters, and they are ignored during a search e.g. ( ....)
I trust that illustrates the purpose of a wildcard in a phrase (a wildcard character won't match punctuation!).
Dave,
I appreciate your explanation of wildcard functionality in searches - actually I think we should put some of this info into http://wiki.logos.com/Detailed_Search_Help under wildcards (and a remark under phrase search, that wildcards apply within phrases). The thread shows that not all of us immediately think of wildcards in the context of a phrase search for a long text containing a questionmark.
Howver, I still think the feature is buggy with respect to the interaction of a ? wildcard with omitted punctuation. Maybe even more:
Dave Hooton said:Now, whilst "lord? jesus christ" failed, a search for "lord* jesus christ" will succeed, giving the same results as "lord jesus christ" in this case.
okay, but what about "lord?jesus christ"? should find "Lord Jesus Christ" as well as "Lord, Jesus Christ". I expected the latter to fail (the bug from yesterday), but it gives zero results total.
EDIT:
maybe the bug is better explained with that wildcards ? and * refuse to resolve for space. "lord*jesus christ" also won't work. I can take any phrase that gives a search result and replace a space with a wildcard and it won't find anything. Or add a questionmark to a word that gets a match - the hit will drop out. That's not only counterintuitive, that's simply wrong.
Have joy in the Lord!
0 -
NB.Mick said:
maybe the bug is better explained with that wildcards ? and * refuse to resolve for space. "lord*jesus christ" also won't work. I can take any phrase that gives a search result and replace a space with a wildcard and it won't find anything. Or add a questionmark to a word that gets a match - the hit will drop out. That's not only counterintuitive, that's simply wrong.
Searches work by first breaking your query into "terms" and then finding articles in your resources that contain all those terms. If you've used Boolean operators or quotation marks to specify a phrase, then additional checks are made to make sure all the terms make a phrase, etc.
But the key insight into the process is that the terms you entered are first looked up in a global list of all terms in all resources. "lord*jesus" is considered one term (because it doesn't contain spaces), and it can only match individual terms in the term list. If "lordajesus" or "lordabcdefgjesus" were words, then they would match the wildcard. But since there aren't any words like that, the term matches nothing and the search fails.
The problem with allowing * to match any number of intervening characters is that "lord*jesus" would match from "Lord" in Gen 2:4 to "Jesus" in Rev 22:21, which seems fairly pointless. Or, if we only wanted it to match intervening punctuation characters, then that's already supported via a simpler syntax: "lord jesus".
0 -
As you've discovered, the problem is that a '?' at the end of a search phrase is being treated as a wildcard character, which doesn't match itself (because punctuation is not indexed). I can see how this is quite unexpected and counterintuitive.
It seems like the best way to still allow wildcards (for the very small number of people who use them) as well as allow selected text -> Search this resource to work would be to relax the restriction on ? matching exactly one character, and allow a trailing wildcard at the end of a word to match nothing. That is, "it?" would now match just "it" (fixing the bug) as well as "its", "ita", "ite", etc. (wildcard matches).
0 -
David Scott said:
I recently wanted to search the Sermon on the Mount for "?" to find every instance where Jesus asked a question. It did not work because of the role that "?" plays as a wildcard. Is there a way to perform the search for a question?
One option is searching for words often used in questions with Words of Christ tagging:
if, how, however, what, where, when, who, whoever, whom, why
Keep Smiling [:)]
0 -
Bradley Grainger (Logos) said:
the key insight into the process is that the terms you entered are first looked up in a global list of all terms in all resources. "lord*jesus" is considered one term (because it doesn't contain spaces), and it can only match individual terms in the term list.
Thank you, Bradley, and you're right, this is a key insight. Especially regarding phrases I wasn't aware that you break them up and run them against the index database. To clarify: his means that wildcards currently can't match spaces (or whatever breaks up a term) and they can't produce hits over term boundaries, correct?
Have joy in the Lord!
0 -
Bradley Grainger (Logos) said:
As you've discovered, the problem is that a '?' at the end of a search phrase is being treated as a wildcard character, which doesn't match itself (because punctuation is not indexed).
I understand the problem, since due to broken-up lookup in the term database it won't find anything to match, neither the space character nor the beginning of the next word.
Bradley Grainger (Logos) said:It seems like the best way to still allow wildcards (for the very small number of people who use them) as well as allow selected text -> Search this resource to work would be to relax the restriction on ? matching exactly one character, and allow a trailing wildcard at the end of a word to match nothing. That is, "it?" would now match just "it" (fixing the bug) as well as "its", "ita", "ite", etc. (wildcard matches).
Yes, please!
Have joy in the Lord!
0 -
NB.Mick said:
To clarify: his means that wildcards currently can't match spaces (or whatever breaks up a term)
Spaces separate words, as does punctuation and footnote markers, so wildcards won't match spaces. That is why "lord ??? christ" will find "lord and christ" If the wildcard matched spaces the expression would match "lord christ" (multiple spaces is the same as single space - it's just a separator).
NB.Mick said:and they can't produce hits over term boundaries, correct?
Yes. As explained above lord*jesus is a single term and won't match "lord jesus".
Dave
===Windows 11 & Android 13
0 -
Bradley Grainger (Logos) said:
It seems like the best way to still allow wildcards (for the very small number of people who use them) as well as allow selected text -> Search this resource to work would be to relax the restriction on ? matching exactly one character, and allow a trailing wildcard at the end of a word to match nothing
That will have consequences for when it is really wanted/intended. When picked up from selected text in a resource it should be parsed for question mark and discarded.
Dave
===Windows 11 & Android 13
0
