Thoughts on Smart Search

danwdoo

I have a Logos Pro subscription, though up until now I have not tried any of the new AI features. I thought I would give it a try today and had some mixed results. My Test was using a book search with the question: "When did Job live?"

Things I liked in the results:
1) The Summary was helpful and relevant. The references in the summary make it reasonably trustworthy.

2) Search Results: The ability to click on a title in the results and open the resource to the quoted area was very helpful and convenient

3) The ability to summarize was useful in some longer articles to know if I should select those for later reading.

4) Bolding relevant terms in the results helped quickly show relevancy

Things that did not go as well
1) Many good resources were missed and were not presented in the first several pages of results (if at all). This included some of my preferred prioritized commentaries that clearly deal with this specific topic that I would always want included in such results. Part of trusting Smart Search is knowing that it is not only factually accurate, but also comprehensive in that it will not miss quality resources that I already have. If I just have to go back and do a manual search later to make sure important resources were not missed, then the whole purpose of the Smart Search is lost. I want Smart Search to better and more easily utilize the resources I already have that have historically required a lot of manual work to find and use. This test result did not accomplish this well.

2) The list within the first page contained non-relevant answers to my question and some not even about the Job in any way. Again, this is an issue because of many high quality resources in my library that should have been included in the results, weren't. Hopefully this will improve with time.

3) Some wacky and irrelevant results such as this one on the search term did:

Entries like this just make the list of results to go through longer and take unnecessary time. Hopefully these will be areas of improvement in the future. So I do see how this could become very useful with some of these issues being addressed, but for now I am still probably looking at using more manual searches than smart ones.

Dave Hooton

@danwdoo

1) Many good resources were missed and were not presented in the first several pages of results (if at all).

How did you establish that they clearly deal with this specific topic?
In essence, prioritized commentaries is not a consideration as results may come from other resources. My Search provided the most relevant results from lower rated, barely used, commentaries via headings that contained "when job lived".

2) The list within the first page contained non-relevant answers to my question and some not even about the Job in any way.

That is unfortunate as you can see that the emphasis can be on other words in the query. But the Synopsis was relevant to Job. So I often run an ALL Search for a broader selection of books and trust the Synopsis.

danwdoo

https://community.logos.com/discussion/comment/1346243#Comment_1346243

Hi Dave.
I determined they dealt with the topic by verifying them manually. Pretty much all commentaries on Job include information regarding dating both the life of Job and the writing of the book even if their conclusion was that the when is unknown (which is still helpful in finding what a general informed consensus is on a topic). The commentaries that were not referenced have sections that clearly dealt with this topic (often extensively at that). One good example was my prioritized NICOT "The Book of Job". It has a very helpful section on dating Job, but it was not referenced at all in the results, the second half of which were not even related to Job. I ran several test searches with both 'all' and 'books' and the results were similar.

Dave Hooton

If you narrow the query to NICOT OT, what is the synopsis? Is it different to your original query? Was NICOT Job present?

danwdoo

If I understand what you are asking, I did a smart search on: NICOT OT.

The first page of results did not include any references to NICOT Job using an ALL smart search or Books smart search. Did I understand correctly what you were asking?

David Taylor, Jr.

I do wonder, and I don't know if this has been addressed elsewhere, does the Smart Search of your own library consider prioritization at all?