Brand new search engine ready to be tested
We've been working on a brand-new search engine that doesn't require special syntax to answer your queries. It's very experimental at this stage, but we'd love your feedback.
We're not yet ready to include the search engine in the desktop app, but you can access it on web here: https://beta.app.logos.com/search?kind=semantic
We'd love you to run real questions/queries there and use the thumbs-up/thumbs-down icons in the results to let us know how well the engine has answered your query.
Then come back to this thread and let us know your overall impressions.
To be clear, we don't intend to replace our existing search engine with all its advanced syntax. That won't be going away. And it's designed for searching your books, not the Bible. But we hope it will have a place in making search even simpler.
Comments
-
Well, I'm impressed. I did a super-simple 'symbols in jubilees'
- Listing went from best match, to substitution match, to most likely key area, to wild-match (good)
- I assume it's a FL library; I already saw several books that get close, I'd like to buy
- I didn't do the thumb thing; I'm just happy to get close!
I don't use the current search syntax (beyond NEAR, etc), so I can't tell exactly what is being beta'd.
But I just tried a similar simple query on Verbum 'symbols jubilees'; it was lost and poor sort. Frustrated,
This looks good.
EDIT: Searches where you're not quite sure how to get there (not just syntax; concept) are the most difficult. What would I pay for this (how valuable)? I'd go as high as $199 (Verbum 11; just this one new feature). And that's just limited use. Exciting.
0 -
My first reaction is obvious. I would like to just search my own library and not include results from the store (this could be a toggle option for those who do want to include the store). And I'd like to be able to sort and display the results in all the ways I can do in the Books tab.
Also, I like how the All tab gives you the top 3 results first and then starts listing the various 'cards'. Can this also list the top 3 results first before that Answers card?
0 -
Mark,
In our searches, I assume it is expected to search the library books we own. Would that be correct?
0 -
Mark said:
In our searches, I assume it is expected to search the library books we own. Would that be correct?
Part of the experiment is for us to discover if there's also value in you seeing results from books you don't own. To be honest, it's also easier for us to code it that way too, so that helped us get the experiment out more quickly. If we release this as a finished version, there would be options to filter results just to your own library.
0 -
Mark Barnes (Faithlife) said:Mark said:
In our searches, I assume it is expected to search the library books we own. Would that be correct?
Part of the experiment is for us to discover if there's also value in you seeing results from books you don't own. To be honest, it's also easier for us to code it that way too, so that helped us get the experiment out more quickly. If we release this as a finished version, there would be options to filter results just to your own library.
For this experimental version, could it list our owned resources first and then the store resources? It's hard for me to tell you if the search result is good if I don't own the book. The snippet of text is not long enough to give me the result in context.
0 -
Kiyah said:
The snippet of text is not long enough to give me the result in context.
That itself is useful information.
We will be bringing improvements to how we handle licensed vs unlicensed resources.
0 -
Here's some positive feedback:
I typed in the following search: logos christology epistemology sophia
It gave me this book by Elisabeth Schussler Fiorenza as one of the results. I gave this a thumbs up for feedback, because it's on topic AND it does not show up as a result when I type that exact same search in my regular search window in the desktop application. This is because it doesn't have the words epistemology and logos in the same article. So it would seem that we are not limited by article in this new search engine. Cool.
EDIT:
This result also hadn't shown up in any of the other searches I've tried for the topic I'm studying. So this new search engine helped me find a book that I hadn't yet found for the past couple of days of researching.
0 -
I really like this new search function. I like how the search provides the top 3 hits but wish it only searched my library. The thumb thing is tacky, but this is a nice function. The option to restrict searches to books from your personal logos library is needed. Why can't the snippets be larger?
0 -
Mark Barnes (Faithlife) said:
Part of the experiment is for us to discover if there's also value in you seeing results from books you don't own. To be honest, it's also easier for us to code it that way too, so that helped us get the experiment out more quickly. If we release this as a finished version, there would be options to filter results just to your own library.
Well that makes sense. I do like very much the idea in the final product of having the ability to search only our libraries, but also the ability to search all FL resources.
I would also find it useful if there was the ability to search along theological lines as well as historical lines. If I wanted resources (for example from the Eastern branch of Christianity) or I wanted to limit the search to resources from Medieval times or from the writings of the Church Fathers...just a thought.
0 -
I'd have to play with this for some time to retrain myself on how I state my searches, The ranking of the results appears to be nearly spot on.
- the law of prayer is the law of believing maxim with quotations works well in a basic search ... here the answer box is what I want but the rest results emphasize the law portion - I get law of fermentation as an example of how off topic it gets
- No English translation did well providing hits
- lex credendi lex orandi provided excellent results
- barren woman story type gave impressive results
- type-scenes Biblical also gave impressive results
- incarnation sermons Chrysostom gave very mixed results as it gave each word priority rather than treating them as equal i.e. some results ignored Chrysostom, some ignored incarnation ...
All in all quite impressive
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
This is very cool, Mark. I've only had time to play with it a bit, but I am impressed.
The first search I tried was: humor in the bible
The results from this experimental semantic search were much better than a typical Books search in Logos with the same text:
I took some time to click thumbs up on many of those as good results.
Then I tried another search related to a question that Christian Alexander asked recently: economic Trinity vs. immanent Trinity
Again, this new search gave better results.
My only suggestions for improvement are:
1. I expected the links from the headings in the search results to take me to the heading of the section in the book, like what happens in Book search in Logos Desktop. Instead, that link takes me to the context of the excerpt shown. In the Desktop app, I can click on the search hits that are highlighted in that excerpt and go there if I want to. But in many cases, I want to get the whole context, so I want to go to the beginning of the section.
2. When one of the search results is selected, the up and down thumb icons are darker grey even if neither has been clicked, making it look like both have been clicked, which is confusing. See if you can find a lighter shade of grey that can still be seen on that highlight-grey background. Or make them white in that case.
0 -
predictions of christ's return - first few results were ok and then the focus on his "return" was dropped.
when will Jesus return - was much better with more books from my library!
pretribulation rapture - was very good except i had to buy one of the books!
I'm impressed so far.
Dave
===Windows 11 & Android 13
0 -
Mark Barnes (Faithlife) said:
Part of the experiment is for us to discover if there's also value in you seeing results from books you don't own
I really like being able to see search results from resources that I do not own. Will there be a way filter or organize the results from those resources I do not own into group or at the end of my results?
0 -
I am clicking the little thumb buttons, but I just wanted to say here that I was very much helped in my research project after spending only a few minutes playing with a couple searches.
0 -
Very impressive! My searches found some results in my library that were actually very helpful and would have been difficult for me to ferret out using the current search capabilities in the desktop app. I agree with others that it would be helpful to either hide the results for resources not in our library or move them to the end.
Even though this is very experimental, I suggest rolling it out soon and updating it via periodic updates. Even as an early beta feature, I think this type of search would be very popular among many types of Logos users (even those who have been using it for many years). And, since it is not replacing any of the current search capabilities, it seems that adding this search feature (even in early form) has lots of upside.
0 -
- Do the thumbs up/thumbs down train the search or are they merely statistical?
- Assuming there is a ranking to the results, I would prefer the unlicensed resource continue to show where they fall but have some visual clue they are locked.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
I am impressed. This doesn't feel like a sales pitch for new material, but an opportunity to see what material is available that might be helpful to add to the library.
As someone else suggested, longer info from an unlicensed resource would be beneficial for me to be able to judge if it is helpful or not.
0 -
MJ. Smith said:
- Do the thumbs up/thumbs down train the search or are they merely statistical?
- Assuming there is a ranking to the results, I would prefer the unlicensed resource continue to show where they fall but have some visual clue they are locked.
They do.
0 -
Rosie Perera said:
They do.
Thanks, Rosie. When I'm glancing through, something on the far right is nearly invisible. I'll have to pay attention.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
Rosie Perera said:MJ. Smith said:
- Do the thumbs up/thumbs down train the search or are they merely statistical?
- Assuming there is a ranking to the results, I would prefer the unlicensed resource continue to show where they fall but have some visual clue they are locked.
They do.
Rosie you must have a massive library. Most of your search results are showing as unlocked. I don't have a small library (just under 9,200 resources) but most of my results are showing as locked.
0 -
Kiyah said:
Rosie you must have a massive library. Most of your search results are showing as unlocked. I don't have a small library (just under 9,200 resources) but most of my results are showing as locked.
Yeah, I've got ~29,000 resources. Not counting the tens of thousands that are hidden. Mostly those freebie collections -- Library of Early English History and Library of Early American History.
0 -
MJ. Smith said:
Do the thumbs up/thumbs down train the search or are they merely statistical?
Merely statistical at this stage. There are ways we can potentially tweak the engine, but before we do that, we want to get a sense of how people use it and what type of queries need improvements.
0 -
Very impressive start! I think this will really open up access to many resources that would otherwise be over looked in our studies. However, in the current form I am getting much more hits from the store than my own library. This makes sense given the number of books available in the store. I am not opposed to having the store included in the search, but would like a better way to filter the results to see what is in my own library.
Keep up the development of this.
0 -
Ran this search in the experimental search: difference between ἀγάπη vs φιλέω
ἀγάπη NEAR φιλέω in book search and the results were much more broad as expected. I probably could have constructed the search to be more specific, but it was helpful to not have to with the experimental search.
Quite impressed. It would be nice to be able to use the lemma:g:agape rather than type or copy the Greek (or Hebrew).
0 -
This search was much better from a books search:
"Jesus in the old testament"
The experimental search brought up titles of books that matched this search from the Store, but did not do as accurate a job of finding material in my library. I assume that " " is not used in the experimental search.
So, for searches that are easily created such as above, the book search is better. For broad questions where the search syntax for book search is difficult, the experimental search is a better start.
0 -
This search was much better from a books search:
"Jesus in the old testament"
The experimental search brought up titles of books that matched this search from the Store, but did not do as accurate a job of finding material in my library. I assume that " " is not used in the experimental search.
So, for searches that are easily created such as above, the book search is better. For broad questions where the search syntax for book search is difficult, the experimental search is a better start.
0 -
As others have said, this looks it could be really useful and a great addition to what Logos already provide.
I tried two searches side-by-side:
- were the seven days of creation seven 24-hour days?
- were the seven days of creation seven undefined periods?
It was interesting to see that while both searches started with the same article, they then diverged although there was comminality in the sets of results.
as the two strings are effectively asking the same question - but from a different starting point - it's interesting to see how there are answers "on both sides" in each set of responses.
I was also impressed with the articles suggested by the query "on what day of the week did the last supper take place?"
0 -
John Fidel said:
The experimental search brought up titles of books that matched this search from the Store, but did not do as accurate a job of finding material in my library. I assume that " " is not used in the experimental search.
When I tried this search I got mainly "section headings" as opposed to book titles.
Would that explain some of what you saw?
0 -
John Fidel said:
For broad questions where the search syntax for book search is difficult, the experimental search is a better start.
Obviously, they'll tweak this and that. Especially your OL transliteration suggestion. Definitely.
But the 'boundary' between this and the existing search is hard to evaluate. I was trying to find sprinkling as purification in 2nd Temple writings. I read it last week. The new search split the difference, jewish/law/messianic vs Christian doctrinal discussions. The older search, I'd force it using various limiters.
0 -
I am getting better results. That's good.
In some searches there is a box at the top of the listings with the word Answers. What is this?
I also would like the option to search my books only, and store books only, or the combination.
Pastor, North Park Baptist Church
Bridgeport, CT USA
0 -
Mark Smith said:
In some searches there is a box at the top of the listings with the word Answers. What is this?
I agree this section seems weird. The best results are not usually there.
I'm finding good hits in books that I don't own. This is information that is very valuable. I can see why someone would want to toggle such results on/off, but for my purposes I definitely want hits in books I don't own.
0 -
John Fidel said:
This search was much better from a books search:
"Jesus in the old testament"
The new search engine is tuned to try and provide broad results for broad questions. Jesus in the Old Testament is a pretty broad search, and therefore you're mostly getting results in articles that deal exclusively with that topic.
Try tightening your query to get more specific results. For example, do you mean messianic prophecies in the Old Testament? Or do you mean Christophanies in the Old Testament?
Double-quotes are taken into consideration, by the way, but that doesn't necessarily exclude a result that talks about Jesus and the Old Testament or Jesus in the OT being surfaced.
0 -
Mark Smith said:
In some searches there is a box at the top of the listings with the word Answers. What is this?
We're using AI to try and identify snippets that directly answer the question that you've asked, and move them to the top of the results and inside the box. You'll likely find that box appearing when you ask natural language questions like How far is Bethlehem from Jerusalem? or What is glossolalia?. Keyword searches are much less likely to show the box.
In the answer box, the bold part of the snippet should answer your question directly. In the normal search results, we tend to bold keywords, instead of the "answer".
However, like most things AI, it isn't perfect, so your feedback on these "Answers" is very valuable.
0 -
DMB said:
But the 'boundary' between this and the existing search is hard to evaluate. I was trying to find sprinkling as purification in 2nd Temple writings. I read it last week. The new search split the difference, jewish/law/messianic vs Christian doctrinal discussions. The older search, I'd force it using various limiters.
The new search can generally give pretty good results without too much thought, even for natural language queries. But the existing search offers much more precision and control. That's why the new search won't replace the current one.
0 -
I like the natural language aspect of the new search; in testing it, I have already found information that had somehow eluded me with the “old” search. That being said, I like the precision of the old search for some search needs. Thank you for your plan to allow both types of searches in Logos. I have found the “chat” style of search being forced upon me by a major search engine not to my liking and appreciate Logos continuing to give me a choice.
Above all these things, walk in love, which is the bond of perfection. - Colossians 3:14
0 -
One of my first thoughts when I did a couple of searches was, "This feels more like doing a google search."
0 -
Hi Mark
maybe stating the obvious but just wanted to check.
it looks like the purpose of the new search engine is to search our resources (either ones we own or ones in the Logos catalog) as opposed to giving definitive answers to questions
is that correct?
Thanks, Graham
0 -
Mark Barnes (Faithlife) said:Mark Smith said:
In some searches there is a box at the top of the listings with the word Answers. What is this?
We're using AI to try and identify snippets that directly answer the question that you've asked, and move them to the top of the results and inside the box. You'll likely find that box appearing when you ask natural language questions like How far is Bethlehem from Jerusalem? or What is glossolalia?. Keyword searches are much less likely to show the box.
In the answer box, the bold part of the snippet should answer your question directly. In the normal search results, we tend to bold keywords, instead of the "answer".
However, like most things AI, it isn't perfect, so your feedback on these "Answers" is very valuable.
OK. I'll examine those results with that information in mind.
Pastor, North Park Baptist Church
Bridgeport, CT USA
0 -
I love this so far! My feedback:
- The results are much more usable than the current implementation of the All Search.
- Helps me better discover the resources I have (or don't have) pertaining to the topic/content I'm searching for.
- Hope the final version will have:
a) the ability to filter by books I own or not own;
b) the Group By options of the current Books search;
c) the "cards" in the results of the current All search.
- Can this be implemented for Bible Search as well? Or is Fuzzy Search already it?
- I tried running a couple of searches in Traditional Chinese. I can't tell whether the Experimental Search is vastly better than the current All Search or Books Search. But at least the Experimental Search doesn't seem to just searching for one of the characters in a phrase when I was trying to search for the phrase.
E.g. I was searching for 天堂 (heaven); the existing searches would return results with just the character 天 (sky). Same with 地獄 (hell) returning results with 地 (earth) only. The new search doesn't appear to do that. I didn't do extensive testing so this assertion needs to be further tested.
0 -
Mark Barnes (Faithlife) said:Kiyah said:
The snippet of text is not long enough to give me the result in context.
That itself is useful information.
We will be bringing improvements to how we handle licensed vs unlicensed resources.
Does the length of text the search engine is taking into account have a character limit? I'm not talking about what's displaying in the search window, I'm talking about the length of the text the search engine is counting as a hit. Or are those two things the same?
0 -
Graham Criddle said:
it looks like the purpose of the new search engine is to search our resources (either ones we own or ones in the Logos catalog) as opposed to giving definitive answers to questions
It's both. We never use AI to generate text to give you a definitive answer (assuming one exists!), but we do use AI to try and point you to a resource that gives you a definitive answer, perhaps even in the snippet.
0 -
As this search is not Bible oriented, I would like to see the Bible Browser updated with the newer label facets, its facets provided a more hierarchical structure, and the opening of the browser integrated into the Search panel. This new search plus the Bible Browser would give solid search results for the new users (in addition to their use for more experienced users).
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
Mark Barnes (Faithlife) said:
It's both. We never use AI to generate text to give you a definitive answer (assuming one exists!), but we do use AI to try and point you to a resource that gives you a definitive answer, perhaps even in the snippet.
Thanks Mark - that's helpful
0 -
I like the option of a semantic search alongside the traditional search.
I entered "significant words in John 7:37-41", and noticed that a lot of the early results came from a single resource.
Here are a couple of things to consider:
1. Group hits by resource. A given resource might be either a goldmine or a mirage. Grouping by resource (with a collapse/expand control) allows the user to either drill deeper or to bypass it.
2. Have a 1-2-3-4-5 set of buttons next to each hit so the user can do a quick first-pass categorisation through the hits.
0 -
I really like the "experimental" search!
- It's quick and simple, no formulas or special characters to remember
- Just type in a word, phrase, person, place or a question in plain English and it works!
- I like that it found hits even in books I would not have thought to look in, some helpful, some not so much.
- As others have noted, I like that it shows hits from the store, but the text from the store books does not contain enough information to determine if I want to purchase it. Most of the time it doesn't even mention the subject I searched.
- Also as other have noted, I would like store books sorted separately below my library books
- This simpler search tool would satisfy many (not all) of my search needs. I would still want the current search tool for more refined searching
Excellent job Mark and FaithLife or Logos of FaithLife or...[:D]!!
Too soon old. Too late smart.
0 -
Question?
Assuming this would eventually roll out to the desktop app, how would the search list and display cloud books?
Too soon old. Too late smart.
0 -
One other question:
How are the responses under (not in) the Answers box sorted? Is there some sort of ranking going on?
Some wishes:
Grouping hits by book or author.
Ability to search a set of resources rather than all of them.
Pastor, North Park Baptist Church
Bridgeport, CT USA
0 -
Bill said:
Assuming this would eventually roll out to the desktop app, how would the search list and display cloud books?
This is still very early stages, but I'd assume the display would be similar to All Search, with icons for Cloud Books and Print Library books.
0 -
Mark Smith said:
How are the responses under (not in) the Answers box sorted? Is there some sort of ranking going on?
Yes, there's a relevance ranking. It's a different algorithm to the one used in the current engine. It uses machine learning to measure the similarity of words/phrases in your query with words/phrases in the article. So, for example, if you searched for something to do with glossolalia, the ranking mechanism would promote articles that frequently mentioned glossolalia (the same as the old engine). But it would also rank fairly highly articles that only mentioned glossolalia once but also had several references to "speaking in tongues" – because the engine knows that "glossolalia" and "speaking in tongues" are very similar in meaning.
This process seems to work pretty well most of the time but can become unstuck, especially when you're interested in something very specific and don't want similar words to be found as well. In these cases, the current search engine tends to perform better.
0 -
Thanks for giving us the opportunity to play around and give some feedback on this Mark. It's very much appreciated!
Like everyone else has said, the ability to sort or limit answers would be good - e.g. All resources in my library. Only print books. Only this specific collection. Only this author but include both owned and unowned resources.
I love the simplicity of it. However, I think when you roll it out you will need to illustrate (perhaps like you have done with the other search functions) how one might input a question and what answers to expect. As the search function is presented right now, it looks and feels just like google or any other A.I. input. This is good, it makes users feel comfortable, and that I can search without having to have a PhD in boolean. Lowering the barriers to access/learning curve is probably the best thing logos can be doing right now. However, the answers don't match my expectation. For example, compare this search query performed in logos and OpenAI
I suspect my discomfort arises from some expectation that when I ask a question using natural language, I expect an answer using natural language - not a catalogue list of resources I don't own. I feel like the way around this is to just set clear expectations for the user right up front, maybe something in the search field like 'Enter anything - no special syntax required, and Logos will find the closet matches in your resources.'
Connected to that, it's not clear at this stage when you expect me to use this search engine verses the others. I understand that the others allow me to be much more specific, but where is the line? When should I use one over the other? The focus seems to be in matching questions to keywords in resources, not the bible. Am I using it wrong when I perform searches like the one below?
Additionally, what is the relationship between the search results and the prioritization feature within our libraries? I'm assuming that at this stage the engine is trying to find the closest match to our search from the entire logos library, regardless of our personal prioritizations?
Current MDiv student at Trinity Theological College - Perth, Western Australia
0
