BUG: Search not working when question mark included

I was doing some searching last night and I discovered what appears to be a problem. I am looking at here, the book "The Jesus I Never Knew" by Philip Yancey. I was doing searches and not finding the verse I was looking for; actually I was doing this on my iPad. So I figured there must be a problem with the iPad app, and I tried this morning on my desktop (5.0b). If you notice the picture, the search finds no results even though it should have found at least 1! I tried typing the phrase, I tried copying it from the resource. Search is supposed to work in eBooks, correct? This is a Vyrso eBook.

Note if I search for just a word, like "missed" (no quotes), it works.

Find more posts tagged with

Comments

Don Awalt

OK I just tried a few other eBooks and search works fine - it's just this one. Is the resource broken?

Graham Criddle

Search is supposed to work in eBooks, correct?

Yes it should

I tried typing the phrase, I tried copying it from the resource.

What happens if you select the phrase in the resource, right-click and choose "search this resource"?

It sounds like it might be an indexing problem - but a strange one.

It will be interesting to see if others with the resource get the same problem. I don't have it so can't check

Don Awalt

What happens if you select the phrase in the resource, right-click and choose "search this resource"?

Ah, excellent tip to try! I did, and it also comes up with 0 results! Something is definitely broken. I wonder if there is a way to figure out if it is a bug or a bad resource?

I recall that I have the same problem on my iPad, does that mean the resource is bad on the server? Or maybe a bug that is in both versions?

Don Awalt

Some phrases work, some do not. I found one place where if I highlight the complete sentence, which includes a question mark, it fails; leave out the question mark, it succeeds.

NB.Mick

I recall that I have the same problem on my iPad, does that mean the resource is bad on the server? Or maybe a bug that is in both versions?

If it was only on the desktop, I'd have suspected a bad index or download (in this case, hiding the book, restarting Logos [possibly even verifying deletion of the *.logos4-file], unhiding the book thus forcing redownload and re-index), but since you saw the issue on the pad it may well be that the original resource file is somehow broken. Note that these are ePub-files Logos gets from the publisher and treats only in automated run with a modified version of the PB compiler.

To narrow this down, if you like, try to give less information to the search, such as leaving off the questionmark or searching for the words of the sentence, not the phrase. Maybe there's a non-printing character somewhere in the spaces of the sentence. Or the questionmark is another UNICODE-character. looking same but having another internal number than the one you type in (we had issues with straight and curved quote markers in the past).

Graham Criddle

I recall that I have the same problem on my iPad, does that mean the resource is bad on the server? Or maybe a bug that is in both versions?

At least it suggests it isn't an indexing problem in your Logos 5 environment[:)]

My guess is that it is a resource problem but I wouldn't know how to prove that.

Hopefully someone else with the resource can verify or Logos may comment later.

NB.Mick

Some phrases work, some do not. I found one place where if I highlight the complete sentence, which includes a question mark, it fails; leave out the question mark, it succeeds.

Your post wasn't there when I posted mine above. Seems the questionmarks are the culprit. Can you verify this against another resource - or two, such as a Vyrso book and a regular Logos resource?

Then we'd know if phrase search is broken everywhere for phrases with a questionmark in it (serious bug!) or only for specific representations of questionmarks such as in this resource or in the Vyrso books.

Bradley Grainger (Logos)

As you've discovered, the problem is that a '?' at the end of a search phrase is being treated as a wildcard character, which doesn't match itself (because punctuation is not indexed). I can see how this is quite unexpected and counterintuitive.

It seems like the best way to still allow wildcards (for the very small number of people who use them) as well as allow selected text -> Search this resource to work would be to relax the restriction on ? matching exactly one character, and allow a trailing wildcard at the end of a word to match nothing. That is, "it?" would now match just "it" (fixing the bug) as well as "its", "ita", "ite", etc. (wildcard matches).

NB.Mick

As you've discovered, the problem is that a '?' at the end of a search phrase is being treated as a wildcard character, which doesn't match itself (because punctuation is not indexed).

I understand the problem, since due to broken-up lookup in the term database it won't find anything to match, neither the space character nor the beginning of the next word.

It seems like the best way to still allow wildcards (for the very small number of people who use them) as well as allow selected text -> Search this resource to work would be to relax the restriction on ? matching exactly one character, and allow a trailing wildcard at the end of a word to match nothing. That is, "it?" would now match just "it" (fixing the bug) as well as "its", "ita", "ite", etc. (wildcard matches).

Yes, please!

Dave Hooton

It seems like the best way to still allow wildcards (for the very small number of people who use them) as well as allow selected text -> Search this resource to work would be to relax the restriction on ? matching exactly one character, and allow a trailing wildcard at the end of a word to match nothing

That will have consequences for when it is really wanted/intended. When picked up from selected text in a resource it should be parsed for question mark and discarded.

Angela Murashov

"it?" would now match just "it" (fixing the bug)

This will be fixed in 5.2 Beta 2.

Bradley Grainger (Logos)

That will have consequences for when it is really wanted/intended.

We considered the implication of making "it?" now match "it" and decided that the actual use cases for a wildcard at the end of a word are rare, and that there is a workaround for advanced users who know what wildcard searching is (e.g., it? NOTEQUALS [nostem] it).

In most cases, the prefix of a search term before the question mark doesn't actually form a new word; for example, the search "baptis?" is unaffected, because "baptis" isn't a separate word, so it won't hurt to attempt to match it.

When picked up from selected text in a resource it should be parsed for question mark and discarded.

The most common queries that contain question marks are (in order, according to our analytics)

Questions, such as: How did Zechariah die?
Morph queries (these don't really count as containing question marks internally, but do show a question mark in the UI)
Selected text from resources, such as: Who do they say that I am?
Typos/invalid queries, e.g., ? millenium
Actual wildcard queries

It wouldn't be sufficient to just strip question marks from selected text in resources that is sent to the search panel. Questions are typed in directly, or text is copied and pasted from web pages, etc.

This change fixes scenarios #1 and #3 above, while making #5 a little less convenient.