Thoughts on Smart Search
I have a Logos Pro subscription, though up until now I have not tried any of the new AI features. I thought I would give it a try today and had some mixed results. My Test was using a book search with the question: "When did Job live?"
Things I liked in the results:
1) The Summary was helpful and relevant. The references in the summary make it reasonably trustworthy.
2) Search Results: The ability to click on a title in the results and open the resource to the quoted area was very helpful and convenient
3) The ability to summarize was useful in some longer articles to know if I should select those for later reading.
4) Bolding relevant terms in the results helped quickly show relevancy
Things that did not go as well
1) Many good resources were missed and were not presented in the first several pages of results (if at all). This included some of my preferred prioritized commentaries that clearly deal with this specific topic that I would always want included in such results. Part of trusting Smart Search is knowing that it is not only factually accurate, but also comprehensive in that it will not miss quality resources that I already have. If I just have to go back and do a manual search later to make sure important resources were not missed, then the whole purpose of the Smart Search is lost. I want Smart Search to better and more easily utilize the resources I already have that have historically required a lot of manual work to find and use. This test result did not accomplish this well.
2) The list within the first page contained non-relevant answers to my question and some not even about the Job in any way. Again, this is an issue because of many high quality resources in my library that should have been included in the results, weren't. Hopefully this will improve with time.
3) Some wacky and irrelevant results such as this one on the search term did:
Entries like this just make the list of results to go through longer and take unnecessary time. Hopefully these will be areas of improvement in the future. So I do see how this could become very useful with some of these issues being addressed, but for now I am still probably looking at using more manual searches than smart ones.
Comments
-
1) Many good resources were missed and were not presented in the first several pages of results (if at all).
How did you establish that they clearly deal with this specific topic?
In essence, prioritized commentaries is not a consideration as results may come from other resources. My Search provided the most relevant results from lower rated, barely used, commentaries via headings that contained "when job lived".2) The list within the first page contained non-relevant answers to my question and some not even about the Job in any way.
That is unfortunate as you can see that the emphasis can be on other words in the query. But the Synopsis was relevant to Job. So I often run an ALL Search for a broader selection of books and trust the Synopsis.
Dave
===Windows 11 & Android 13
0 -
Hi Dave.
I determined they dealt with the topic by verifying them manually. Pretty much all commentaries on Job include information regarding dating both the life of Job and the writing of the book even if their conclusion was that the when is unknown (which is still helpful in finding what a general informed consensus is on a topic). The commentaries that were not referenced have sections that clearly dealt with this topic (often extensively at that). One good example was my prioritized NICOT "The Book of Job". It has a very helpful section on dating Job, but it was not referenced at all in the results, the second half of which were not even related to Job. I ran several test searches with both 'all' and 'books' and the results were similar.0 -
If you narrow the query to NICOT OT, what is the synopsis? Is it different to your original query? Was NICOT Job present?
Dave
===Windows 11 & Android 13
0 -
If I understand what you are asking, I did a smart search on: NICOT OT.
The first page of results did not include any references to NICOT Job using an ALL smart search or Books smart search. Did I understand correctly what you were asking?
0 -
I do wonder, and I don't know if this has been addressed elsewhere, does the Smart Search of your own library consider prioritization at all?
0 -
Smart search does not take a user's prioritization into consideration.
1 -
I would REALLY like this to happen someday. It would make Smart Search more useful for me and I imagine others would like it too.
As an aside, I am hoping that someday, Logos will re-do prioritization as a standalone tool with categories. I know I won't get a Christmas miracle this year, but maybe someday.iMac (2019 model), 3Ghz 6 Core Intel i5, 16gb Ram, Radeon Pro Graphics. 500GB SSD.
2 -
You can only search a specific book in Books (modify
Your Books).Dave
===Windows 11 & Android 13
0 -
What do you understand by Prioritization as it currently only applies to books with an Index (excluding Page) e.g. in Bible Search?
Do you want categories of books to appear in the order you specify in the Prioritization List e.g. English Bibles, Greek Bibles, Hebrew Bibles, LXX Bibles, Bible Dictionaries (a mixture of Encyclopedias and Bible Concordances), English Dictionaries, Greek Lexicons, Hebrew Lexicons, Aramaic Lexicons. If you had non-indexed books in separate categories, I think you would need User Tags to identify them e.g. for monographs. And Logos' default prioritization would cover the books you did not list.
Dave
===Windows 11 & Android 13
0 -
my thinking is that whenever I am using ai for a search I want it to include any relevant resources that I have prioritized when it builds its summary. Does that answer your question?
0 -
my thinking is that whenever I am using ai for a search I want it to include any relevant resources that I have prioritized when it builds its summary. Does that answer your question?
Prioritization would have to include books without an index, and it seems you are not concerned about order as long as prioritized books are in the Smart Search?a
Dave
===Windows 11 & Android 13
0 -
As long as the generated summary includes the prioritized resource if it touches on the query. Yes.
0 -
As long as the generated summary includes the prioritized resource if it touches on the query. Yes.
I prefer that the first criteria be that the resource adds significant information to the topic and secondly, only if they provide essentially equivalent information my prioritized item be chosen over unprioritized items.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
1 -
That is a different to what was requested. It could be an option, but I'm not convinced that Smart Search differentiates "significant information" reliably. e.g. did Joseph's coat of many colors cause his brothers to be angry?
- The best answer (highlighted) from a Books Search was #3, yet it was not referenced by Synopsis for this specific.
- #1 had as good an answer, one paragraph away from the highlighted text, but referenced by Synopsis for this specific.
- ALL Search had #1 above listed as #2 and also referenced by Synopsis.
- #3 above required at least 4 expansions of results to locate; results that were inferior as they focused on the robe and/or its colors.
In ALL Search, my Library book #3 provided "equivalent information" to the #1 catalog book yet was buried underneath other "inferior" results from the Logos catalog.
Dave
===Windows 11 & Android 13
0 -
I have lots of AI credits, so I am giving this Job question a go. I am certainly a novice with AI. I found that in spite of the catalog of extraneous hits, the AI synopsis put together a reasonable answer. Perhaps I am asking too much but couldn't the search use Boolean logic to avoid a search for each word in the question? Clearly if this logic were used all results would be on target, e.g. when+did+Job+live.
0 -
I again ran smart search for the second question. This question regarding the coat had more responses on target. The synopsis was still on target with many resources from my library. I asked again in an ALL search. This would be of value if my library had not had the depth of support. Three top references were provided with the ALL Serach.
It occurred to me that it would be GREAT if the smart search feature would go one step further and identify which of the referenced books were in my library… that would be significant for me
0 -
I again ran smart search for the second question.
What was the 'second' question 'regarding the coat'?
it would be GREAT if the smart search feature would go one step further and identify which of the referenced books were in my library…
In ALL Search there is a lock icon against the title of books not in your library…
Dave
===Windows 11 & Android 13
0 -
Perhaps I have some setting turned off.
0 -
Dave
===Windows 11 & Android 13
0
