Searching with British or US spelling - different results!

I had naively assumed that I could search with British or US spelling and all the results from both queries would be returned. However, I was searching my library today for "empathise" and found just 26 results. When I search for empathize, it returns 718 (different) hits. (My search was more complex but when I got no meaningful results I tried just this word).

Is this the intended behaviour of the software? If so, is it worth going to uservoice and making a suggestion to return all results for a word whether it is printed by a British publisher or an American one?

Colin

Find more posts tagged with

Comments

JT (alabama24)

Is this the intended behaviour of the software?

Yes!

is it worth going to uservoice and making a suggestion to return all results for a word whether it is printed by a British publisher or an American one?

Probably not. I assume this would be very difficult to implement. How would you go about it?

Colin

https://community.logos.com/discussion/comment/890927#Comment_890927

Thanks Alabama for clearing that up!

I assume this would be very difficult to implement. How would you go about it?

No idea but Google manages it somehow ....

Colin

DominicM

https://community.logos.com/discussion/comment/890937#Comment_890937

Easiest way is using a wild card in these cases: emphasi*e would return both sets in same search

PetahChristian

https://community.logos.com/discussion/comment/890927#Comment_890927

I assume this would be very difficult to implement. How would you go about it?

Some degree of search normalization is already implemented in Logos. For example, naïve returns results that include naive (diacritic-insensitive), and shepherd returns results that include Shepherd (case-insensitive).

UK/US normalization involves the same manner of canonicalization, but the real dilemma is how far you take it:

Do you only canonicalize spellings (e.g., normalise -> normalize)?
What about words with different meanings, such as bonnet? Should it also match hood but not hat?
Do you include jelly in results for jam? Was that the right sense for the term?

The difficulty really lies not with what is technically possible, but what people expect. As soon as you start returning results that don't literally match what a user expected, are the results incorrect? Should the user have to then exclude any false positives? At that point, it's a no-win situation.

I think FL has chosen the most practical solution, which is to not surprise users and return exactly what the user searched for, while giving them the power to use grouping/lists/OR to specify alternate spellings

JT (alabama24)

https://community.logos.com/discussion/comment/890937#Comment_890937

No idea but Google manages it somehow ....

Google spends BILLIONS on thier search engine. Logos doesn't.
Google searches are done online, not locally. Would you be ok gaining this feature but losing offline (local) searches?
You can accomplish what you want now, with the right syntax.

Justin Gatlin

https://community.logos.com/discussion/comment/890948#Comment_890948

I assume this would be very difficult to implement. How would you go about it?

Some degree of search normalization is already implemented in Logos. For example, naïve returns results that include naive (diacritic-insensitive), and shepherd returns results that include Shepherd (case-insensitive).

UK/US normalization involves the same manner of canonicalization, but the real dilemma is how far you take it:

Do you only canonicalize spellings (e.g., normalise -> normalize)?

What about words with different meanings, such as bonnet? Should it also match hood but not hat?

Do you include jelly in results for jam? Was that the right sense for the term?

The difficulty really lies not with what is technically possible, but what people expect. As soon as you start returning results that don't literally match what a user expected, are the results incorrect? Should the user have to then exclude any false positives? At that point, it's a no-win situation.

I think FL has chosen the most practical solution, which is to not surprise users and return exactly what the user searched for, while giving them the power to use grouping/lists/OR to specify alternate spellings

The logical thing would be to include alternate spellings in Include All Word Forms, and not when that is unselected. That gives both power and ease of use.

Dave Hooton

https://community.logos.com/discussion/comment/890948#Comment_890948

The difficulty really lies not with what is technically possible, but what people expect. As soon as you start returning results that don't literally match what a user expected, are the results incorrect?

Match All Word Forms will return results that don't literally match what was entered (and it could be a user's default). But it doesn't normali[sz]e British/US spellings. So a solution would have to involve a new Search option Normali[sz]e British/US Spellings, and hope that FL can also change and use it to feed the algorithm used by Match All....

EDIT: If FL do the Brtish/US variants the algorithm could be given each variant when Match All... is selected.

JT (alabama24)

https://community.logos.com/discussion/comment/891014#Comment_891014

I wonder how that works. I assume that some basic rules are applied (prefix, suffix, etc.) which wouldn't apply here. If that is the case, each and every variant would have to be added manually. I could be wrong. I have been before. I agree, however, with your thinking!

Dave Hooton

https://community.logos.com/discussion/comment/891018#Comment_891018

I wonder how that works. I assume that some basic rules are applied (prefix, suffix, etc.) which wouldn't apply here. If that is the case, each and every variant would have to be added manually.

The algorithm for Match All.. is complex, but wouldn't have to be modified if FL gave it each British/US variant triggered by a separate British/US Spelling option.

JT (alabama24)

https://community.logos.com/discussion/comment/891022#Comment_891022

I wonder how that works. I assume that some basic rules are applied (prefix, suffix, etc.) which wouldn't apply here. If that is the case, each and every variant would have to be added manually.

The algorithm for Match All.. is complex, but wouldn't have to be modified if FL gave it each British/US variant triggered by a separate British/US Spelling option.

I agree... if this were possible, this is how it should be implemented. My question, however, is would there be a good way to create rules for this? Would there be many false positives?

Dave Hooton

https://community.logos.com/discussion/comment/891014#Comment_891014

The logical thing would be to include alternate spellings in Include All Word Forms, and not when that is unselected.

The most flexible approach is to have a separate option for British/US Spellings (see my other posts).

Dave Hooton

https://community.logos.com/discussion/comment/891027#Comment_891027

My question, however, is would there be a good way to create rules for this? Would there be many false positives?

I'd say a lexical approach would work much better than an algorithm based on letter usage e.g. 's' versus 'z'.

Colin

https://community.logos.com/discussion/comment/891031#Comment_891031

Thanks for carrying on to discuss this guys and for giving me a little hope at least. I want to make the point which I left unstated in my first post which is that this 'problem' affects all of us. There were results found using the British spelling query which did not show when using the American one.

It's in all our interest either to know how to construct a search which will discover both spellings or to see if the experts can figure out a way to modify the search engine so it will be done automatically.

Colin

Tom Reynolds

https://community.logos.com/discussion/comment/891065#Comment_891065

While I appreciate that this might be difficult it seems clear that people are missing out on results because of spelling variations or errors. e.g. if you are not even aware that there are different UK, etc. spellings for a word you are unlikely to ever find them. When you add in spelling mistakes or old forms of spelling you have quite a mess. For example, to find all the instances of counselling you would have to do at least 5 searches (BTW I entered typo reports for the first two variations).

couseling (7x in my library)

conseling (1x)

counsselling (1x)

counseling (26503x)

counselling (1709x)

David Ames

RE: search with British or US spelling

https://en.wikipedia.org/wiki/Wikipedia:List_of_spelling_variants

Has a list of 158 words that have different spellings

JT (alabama24)

https://community.logos.com/discussion/comment/891065#Comment_891065

It's in all our interest either to know how to construct a search which will discover both spellings

That I can help you with. A search for behaviour OR behavior will find both spellings of the word. The operator "OR" means that only one of the variants has to be true.

Dave Hooton

https://community.logos.com/discussion/comment/891080#Comment_891080

That I can help you with. A search for behaviour OR behavior will find both spellings of the word.

They will have different highlight colours in results. Use behaviour, behavior for the same color.

JT (alabama24)

https://community.logos.com/discussion/comment/891172#Comment_891172

[Y]

Suzy

https://community.logos.com/discussion/comment/891172#Comment_891172

That I can help you with. A search for behaviour OR behavior will find both spellings of the word.

They will have different highlight colours in results. Use behaviour, behavior for the same color.

I wonder if it would be helpful if the predictive text in the search box could offer these options as a reminder? Possibly even in a different colour? Or even color :-)

Dave Hooton

https://community.logos.com/discussion/comment/891195#Comment_891195

I wonder if it would be helpful if the predictive text in the search box could offer these options as a reminder? Possibly even in a different colour? Or even color :-)

It will suggest on a lexical basis if you type e.g. behavio, but you have to make the choice. What you suggest is possible but it implies the search term will be behaviour, behavior whereas a search option would enable British & US spellings or disable it for either word.