AI search is a joke – Exhibit 2
I’ve already posted once about AI search not being up to snuff. Here is exhibit 2. I know I have book on the I AM statements. It is Warren Wiersbe’s book, “Jesus in the Present Tense: The I AM Statements of Christ”. The phrase “I AM” is in the title of the book. I asked Logos AI using Books search the following: “I AM” statements. Remember, the Books search is limited to my books. The search results don’t even list Warren Wiersbe’s book in the top 50. Don’t tell me that one takeaway is that this AI search is a joke at times!
C’mon MJ and Dave Hooton and others who know how to do a precise search. Do something similar to what I’ve done. Do a Logos AI search that you know should turn up some books that your own in the library. Does your AI search pull up those books? If not, please let Logos know about it. Hopefully, they’ll do something to improve this product.
You know what is bad when search doesn’t yield the most meaningful books in your library? People think that they’ve gotten the best out of their library thanks to AI search when all they got is _______ (fill in your own word that shows your annoyance)
I believe in a Win-Win-Win God.
Comments
-
True. It would be nice if we were able to utilize a bit more of the typical chatbot functionality in telling Logos AI how to search through our library to find results. For example, asking, "I know I have a book dealing with the "I AM" statements of Christ. Can you search through the titles of the books in my library to help me find it?" Instead, the how seems to be more or less predetermined regardless of the question. So if the question doesn't fit in well with how Logos AI is programmed to answer, you may well receive unsatisfactory results.
0 -
Not the answer you want, and I do understand your justified frustration, but to present the flip side: I know too much about AI and about Logos/Verbum to be surprised. I consistently get results that are sensible for what not-AI-but-popular-pseudo-AI built with if-then logic and no evaluative judgement of quality (okay, I slightly overstate) and Logos/Verbum general prioritization independent of the user and without fore-knowledge of the answer decides are the most relevant terms/results. I am slowly getting better at guessing the right hint(s) to head both halves in better directions.
It did, however, fail to note my favorite Orthodox resource on the topic which I attribute to it not being carried by Logos. 🙄
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
You appear to be coming at this with a preconceived answer and wanting that answer without considering how AI is coming at it. If this is true, you are going to be disappointed quite often because it will never meet your expectations. I would agree that I wish the AI would search based on our prioritization of resources, but that being said it still provides an answer based on its data. If you know you have a resource that you want considered, you can include that in your question.
0 -
Chill my friend @Christopher Randall. Just go back and re-read my post carefully before replying.
It is exactly because of answers like this that I am not on any social media whatsoever in my life. I don't want to waste my time pointing out obvious problems with replies. Unfortunately, since I am not not smart enough, I have to be on this one as I need to ask questions on this Forum to learn to use the software.
Now, I have not prioritized any resources. I don't know where you got this idea.
I also didn't go searching for when AI is screwing up. I wanted to see what books are there in my library that can help me with I AM statement. And I happened to know one book. For other searches, I may not have any idea what book is there in my library that will have the answers I am looking for.
I have no idea how AI search works. All I know is what any search should do at the minimum. When a book has "I AM statement" in the title and my query has "I AM statement", I would hope it should return this book at the top of the search. Instead, it doesn't even list it in the top 50.
I am no Logos hater. In fact, I love Logos just as much as others in this Forum, but it is perfectly fine to point out where it is bad. In fact, AI search is pretty bad.
I believe in a Win-Win-Win God.
0 -
When a book has "I AM statement" in the title and my query has "I AM statement", I would hope it should return this book at the top of the search.
I would agree with you. But it's possible the AI is searching through the content of the books moreso than the titles.
In fact, AI search is pretty bad.
There are indeed questions that the AI search fails to answer well. There are other questions that it succeeds in answering well (or at least in a way that is helpful to me). I look forward to Logos AI being improved over time.
0 -
In a movie I heard Benicio del Toro say this line: "Don't ask me how the clock works, ask me the time." Honestly, that's exactly how I feel when I can't clearly express what I'm looking for in Logos, and worse, when the AI responds with something irrelevant or completely ignores certain resources - even when they are explicitly mentioned in a query.
I've started to rely more on ChatGPT for some searches, which I admit has been more efficient in some cases. To "help" Logos a bit, I've also been filtering my searches by collections. While it improves the results slightly, it feels like I'm compensating for the system's shortcomings.
Meanwhile, we humbly come here to listen to a patient and kind team… and occasionally, to endure them!
1 -
I’ve noticed responses vary from mobile to desktop searches also. And yes, some results are not really using books we own.
0 -
In a movie I heard Benicio del Toro say this line:
"Don't ask me how the clock works, ask me the time."Whereas my cattleman father in his 60's became interested in having a desktop computer (DOS and floppies day) ordered the components for a computer so he could build his own and therefore, understand what to expect. He went on to become the county computer nerd, retiring from going into the schools to resolve the student problems in his late 80's.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
Perhaps another way to look at it: Precise Search seeks what you asked for, while Smart Search explains what you asked for. (FWIW, I prefer Precise searches; Smart search results are too bland and shallow for my taste.)
0 -
I'm curious about the summary that it provided. If it was accurate and informative, then perhaps the AI felt the contents of the book were redundant. I don't believe the AI is claiming to give you a comprehensive list of the resources in your library that have application to your search question. Or maybe the AI just doesn't like Warren Wiersbe.
0 -
@Bob Venem. Nice nuance to explain difference between Precise and AI search.
In my situation, I couldn't have asked for Precise search because I was looking for all resources in my library that had something to say about I AM statements. It just so happened that I knew of one book in my library whose entire (100%) content was related to I AM statements. And AI search didn't even pick it up. That is the annoying part. Imagine, if I didn't know about Warren Wiersbe's I AM book in my library. Since AI didn't direct me to this book, I would have been wasting my time on other books. I thought they hyped up AI search as one that will help users maximize the content in their library. It is just hype sometimes.
I believe in a Win-Win-Win God.
0 -
Here you go my dear: @Mark Allison
The concept of "I AM" statements in religious and philosophical contexts carries significant weight. In biblical texts, such statements are often associated with divine declarations, as exemplified by Jesus' assertion "Before Abraham was, I am". (1) This type of statement is seen as a claim to divinity or absolute truth. In Christian theology, "I AM" statements are also used to affirm a believer's identity in Christ, with some sources providing lists of such affirmations for Christians to embrace. (2) However, the use and interpretation of "I AM" statements can be contentious. Some argue that claiming absolute knowledge or truth through such statements is problematic, as it implies that those who disagree are wrong. (3) In broader religious history, particularly in the context of ancient Israel, similar absolute statements about God's nature and uniqueness appear, such as "I am the first and I am the last; besides me there is no God" found in Isaiah. (4) These declarations often served to assert the supremacy of one deity over others, especially during times of cultural and religious conflict. (4)
1Fred G. Zaspel, The Theology of B. B. Warfield: A Systematic Summary (Wheaton, IL: Crossway, 2010), 263.
2Stan Toler, Rethink Your Life: A Unique Diet to Renew Your Mind (Indianapolis, IN: Wesleyan Publishing House, 2008), 78.
3Cornelius Van Til, The Protestant Doctrine of Scripture (The Presbyterian and Reformed Publishing Company: Phillipsburg, NJ, 1967), 82.
4Carsten Colpe et al., “God,” in The Encyclopedia of Christianity (Grand Rapids, MI; Leiden, Netherlands: Wm. B. Eerdmans; Brill, 1999–2003) 427.
I believe in a Win-Win-Win God.
1 -
I'd think (judging from the experiences so far), that the AI is a 'discovery' type feature, while precise is find-it-or-give-up. Meaning, AI's not-find isn't to be relied on. 'However', (hasn't been explained yet otherwise), Mark (Logos) said the actual search is traditional; the AI is picking and choosing the 'best' results.
Not fully related, though similar, but I notice on Logos.com, if you don't put 'title:xxx', the results returned may start out with an exact title match ('may'), but then devolve into who-knows. The 'older' Logos.com search was more 'precise'. I also notice, even with a title search, Logos.com dumps 2025 libraries in, that don't have the title. It's tiresome.
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
1 -
Dear MJ,
Thank you for for sharing the inspiring story about your father. I deeply admire that level of curiosity and determination.
I'm practical... but let's not get carried away!😉 I just wanted to point out that tools like Logos can sometimes be more complicated than necessary. While I absolutely value understanding how things work, I also believe that tools should strive to be intuitive so that we can spend more time learning and less time overcoming technical hurdles.
0 -
Sorry, I was not trying to come across negatively. Personally, I think you need to chill. You are the one stating AI is not taking into consideration what you see as meaningful and that it is a "joke." I'm just trying to state that we cannot expect it to give us the results from resources we want. Like what Aaron said, "it's possible the AI is searching through the content of the books moreso than the titles."
Or it is like what MJ is implying with the statement: "I consistently get results that are sensible for what not-AI-but-popular-pseudo-AI built with if-then logic and no evaluative judgement of quality (okay, I slightly overstate) and Logos/Verbum general prioritization independent of the user and without fore-knowledge of the answer decides are the most relevant terms/results."
All I am saying is, you may know that you have a resource that has the information you want. AI is not going to know what you are looking for without you being more specific in your query. It is going to answer based on its own prioritization of resources or its own algorithm or whatever its processes are that determine which resource it considers bests, whatever that may be. That "prioritization" or whatever we want to call it is not going to be what we necessarily want at any given moment.
When the answer comes back not the way we expect it, we have to remember it is just a tool that is still being developed. But we also have to realize sometimes we need to ask the question differently.
0 -
When the answer comes back not the way we expect it, we have to remember it is just a tool that is still being developed. But we also have to realize sometimes we need to ask the question differently.
@Christopher Randall asking the question differently is definitely the way to go (especially short-term). Even as AI is improved, we will have to learn and adapt to maximize its usefulness. That said, it would be great long-term if the way questions were asked would more effectively affect the AI's prioritization choices.
1 -
I get the feeling the answer to AI on Logos is 'ok, whatever':
- It's in development for whatever.
- It didn't like the question' for whatever.
- It does something else we don't know about; whatever.
Sounds like worth subscribing to.
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
whatever.
5 -
I just have been curious.
Typing the bold in the search field:
“I Am” statements
yields a more reasonable answer than this text:
“I AM” statements
0 -
I use the AI features in Logos multiple times a day and it's brilliantly useful for me. When I'm forced to use the competitor's software, it feels a bit crippled now.
2 -
Brilliantly is in the eye of the (AI) beholder! Maybe they designed it for you!!
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
1 -
True. Just like not everybody gets the joke.
0 -
Sorry @Christopher Randall , I am not wasting my time responding to you. It will take too much of my time and it will go nowhere, so no one gains and I lose, so I'll pass.
@Aaron Hamilton. I understand it is a a tool, but if you want to call a tool a hammer, it better be able to hammer in a nail. Otherwise, call it something else. Don't call it AI search, when it fails some basic criteria for what it should return.
Dear techie friends! Is there some objective criteria that can be employed to judge the quality of AI search output? That will settle some dust.
I believe in a Win-Win-Win God.
0 -
How do you bold the search phrase?
Are you saying I should try "I Am" instead of "I AM"? Sorry, could you please clarify?
I believe in a Win-Win-Win God.
0 -
Good for you. Your experience reminded me of a situation when our friends from Washington DC visited us, they found home prices in our neighborhood to be cheap compared to Washington DC. But that is because they are making a relative comparison. So I understand that, as a Accordance user, you are happy. A functioning Logos software should have made you happy. But I believe in an objective absolute criteria, not some arbitrary relative criteria to judge whether a product is good or not.
I believe in a Win-Win-Win God.
1 -
Please understand that the comments regarding Logos AI being a tool were not my own.
0 -
Yes, I'm finding AI search less than helpful. I was hoping for some clarification here.
1 -
I think we may have here a case of not brilliant. Scholars may demur.
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
1 -
I might frame it slightly differently. FaithLife is pitching the new AI-based features as a great leap forward that's going to transform users' Bible study. But at least some users are finding that they aren't as intuitive and easy to use as the sales materials may have led them to believe. That just reaffirms my belief that they've prematurely doubled down on the genrative AI technology.
1 -
Knowing you are non-subscription (until you tell me otherwise), I assure you that Logos went for a very useful, very conservative implementation that ,were it not for all the misleading hype about LLM/Chatbots at this time, would have Logosians very excited. The problem is, and it is a problem that I take seriously, that people's expectations are way off base in a variety of directions. It is similar to comparing Google to Logos and finding Logos wanting because it is less capable that Google for some questions - while ignoring all the questions Logos handles easily that Google can't handle. I think of it as similar to learning when to use an encyclopedia, dictionary, atlas, or almanac in the dead tree era. But managing expectations is a skill I admire rather than have.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
5
