5 free AI engines vs. Logos AI
Hi everyone,
I bought subscription at launch – am giving Logos the benefit of doubt that they will deliver something of value over the next 2 years. But today was the first day trying out AI search.
I asked the question “What does criticism mean in biblical studies” to 7 different apps: Gemini, Copilot, ChatGPT, Perplexity, Claude (all 5 are free) and Logos “Books” search (have over 12K books) and Logos “All” search. Do this experiment yourself.
What is my takeaway: The 5 free AI output is so much better than the 2 Logos AI output. Logos AI output is a joke––sorry, I don’t have a better way to phrase this. It is beyond disappointing.
Now, each of 5 free AI outputs has some good and unique things that are missing from the others. So I made up my own summary of “Criticism in Biblical studies” by stringing together the best key phrases and sentences from the 5 free outputs.
My suggestion to Logos (based on admittedly 1 data point): Logos should take the users query for “All” search, feed the query into the 5 free AI machines, and then summarize the output after checking for accuracy using its reliable resources so that it can give citations in its summary like it does now.
What are the advantages of my suggestion?
( 1) Logos will not waste its resources trying to improve its weakness. Trying to duplicate ChatGPT/Gemini/Copilot/Perplexity/Claude… is a waste of resources as Logos can never match up to the big boys
(2) Logos will focus on its strengths – providing accurate information with citations, which customers crave.
(3) Customers will pay because summarizing the various AI outputs is what I would do manually as in the above example. And the summary of summary is so much better than any one output.
Any thoughts from Logos and tech-smart people?
I believe in a Win-Win-Win God.
Comments
-
I wanted to say something but I'm not from Logos, nor am I tech-smart, so I'll keep quiet.
2 -
"then summarize the output after checking for accuracy using its reliable resources so that it can give citations in its summary like it does now."
What you're asking for here is actual AI, which does not exist yet. Logos responses are less detailed because they are based on your library. That's the downside. The upside is that basing results in specific resources means that the answers aren't just made up, which your other examples frequently do.
Using Logos as a pastor, seminary professor, and Tyndale author
1 -
Each AI that you mentioned does different things, and they also have different sources (though similar). For example, Perplexity is primarily an AI search engine; Claude, has or used to have a larger intake of information, but is unique in how it presents the information (I prefer this one for help in writing and thinking); ChatGPT…well it's ChatGPT…anyway…my point is, they are different.
Logos appears to be going a different route, instead of the internet or an archived version of the internet, its primary source is its own resources (if I understand them right), whether that be what you own (books search) or everything they own (all search). They also limit the AI output to giving just a brief synopsis or summary of the resource(s) instead of a more "robust" answer that you will see in the above-mentioned Large Language Model (LLM) AIs. This limitation ensures a lower likelihood of hallucination results (though I must say AI has improved significantly in this area). Also, there is an ethical concern about the use of AI in certain aspects. I believe Logos is doing it right by taking this slow.
What you are asking for would be nice to have within Logos itself. Until things improve, I use Logos and another AI side-by-side. I like asking multiple questions about a subject to help me understand it better (kind of like thinking out loud with a partner) - Logos AI doesn't help with that.0 -
Comparing chat bots against LLM's to enhanced searches against Logos resources is an unfair comparison - pomelos against kumquats. My personal gauge is - the pre-L10 all search I used occasionally to find items outside my normal range of books, the pre-AI-L10 all search I avoided at all cost as the order of presentation made it useless on a library of my size, the post-AI all search I actually use because it usually succeeds in going beyond my precise wording to find the most relevant results for what I was actually asking.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
1 -
Logos is not using their own AI from what I can gather, they are borrowing/paying for an existing one and modifying it into their platform, this is why you have a number of credits each month.
Personally, I think this is the greatest thing Logos has ever done, as my biggest complaint was how outdated the search function was. Now there is very little need to use syntax searches, and you can search like you would in a normal google search. It was to the point where I was googling verses to find them, since I couldn't remember exact references, or my memory was multiple translations mashed together so Logos search could not find it. That is fixed now with AI search with the fuzzy option.
4 -
I use Smart Search vs AI chatbots for two totally different purposes, which seems to be what each was designed for. When I need to run a search on my library, Smart Search, as well as Search Summarizations, generally finds what I need (only an occasional weakness, I haven't found it to be a "joke"). When I need to "converse" with an AI chatbot to locate something that would be difficult to find in a conventional (or even "smart") search, and it doesn't matter if it's coming from books I own, I use an AI chatbot, generally BibleMate (I keep it pinned on the toolbar). I use it for questions such as what Bible passage mentions (topic X) that I couldn't easily word in a Bible search.
Dr. Nathan Parker
2 -
The answer is very simple.
To Summarize😉Logos does a smart search on the logos library (All) or your library (Books) while those others search the internet or other bots and give you results which may be fact, fiction or opinion or a combination.
The Logos search is exactly that - a Logos search! It's now a lot smarter than it used to be before, and a great way to find answers in the Logos libraries.1 -
REPLY TO JUSTIN GATLIN
[1 Cor 10:31: "then summarize the output after checking for accuracy using its reliable resources so that it can give citations in its summary like it does now."
Justin Gatlin: What you're asking for here is actual AI, which does not exist yet. ]
I am not smart enough to know what is “actual AI”. I’ve asked the tech person in my school to summarize many years of my student evaluations. He has said that AI can easily summarize if I get PDFs. So I don’t see why Logos can’t do this. If a business school tech person can do it, surely Logos can do it.
[Justin Gatlin: Logos responses are less detailed because they are based on your library. That's the downside. The upside is that basing results in specific resources means that the answers aren't just made up, which your other examples frequently do.]
Yes, what you say is true, BUT only when we select a “Books” search, which is based on each one’s personal library. Here is the output based on my library of 12k+ books...
In biblical studies, criticism refers to the application of modern literary and historical-critical methods to study the Bible, aiming to understand its composition, sources, authorship, and literary development. This approach encompasses several interrelated disciplines, including higher criticism, literary criticism, form criticism, and tradition criticism. Biblical criticism seeks to appreciate the text through a fuller understanding of its literary history and message, rather than to lead readers to faith or action. The field has evolved to include various methodologies such as source criticism, which attempts to identify underlying documents or sources, and redaction criticism, which examines how authors shape inherited traditions according to their theological or narrative interests. While some may perceive biblical criticism as deprecating the Word of God, its intent is to develop a discriminatory understanding of the text's historical and cultural context. However, the application of critical methods to biblical studies has been a subject of debate, particularly regarding its compatibility with traditional views of biblical authorship and infallibility.
Logos has a “All” search, which is based on the entire Logos library. Here is the output…
In biblical studies, criticism refers to various analytical approaches used to study and interpret biblical texts. These approaches include historical criticism, which examines the historical context of the texts, and literary criticism, which focuses on the formal features of the text and the role of the reader. Some scholars have applied postmodern theories to biblical studies, though there is debate about what constitutes "post-modernism" in this context. Reader-oriented criticism and feminist criticism are also mentioned as approaches within biblical studies. Additionally, the field has seen the application of computer technology to assist in textual analysis and hypothesis formulation. It's important to note that while historical criticism has been dominant, there is ongoing discussion about the need to consider not just historical reliability but also the meaning and relevance of biblical texts for contemporary readers.
Comparing the 2 results, clearly there is nothing special in the “ALL” search compared to the “BOOKS” search. The above 2 pale in comparison to what comes out of the 5 free AI. Here is the Word file with all 5 outputs.
See for yourself.
[Justin Gatlin: The upside is that basing results in specific resources means that the answers aren't just made up, which your other examples frequently do.]
This is exactly, why I’m asking Logos to take the output and rewrite the summary using citations.
I believe in a Win-Win-Win God.
0 -
[1 Cor 10:31: "then summarize the output after checking for accuracy using its reliable resources so that it can give citations in its summary like it does now."
Justin Gatlin: What you're asking for here is actual AI, which does not exist yet. ]
I am not smart enough to know what is “actual AI”. I’ve asked the tech person in my school to summarize many years of my student evaluations. He has said that AI can easily summarize if I get PDFs. So I don’t see why Logos can’t do this. If a business school tech person can do it, surely Logos can do it.
[Justin Gatlin: Logos responses are less detailed because they are based on your library. That's the downside.]
Yes, what you say is true, BUT only when we select a “Books” search, which is based on each one’s personal library. Here is the output based on my library of 12k+ books...
In biblical studies, criticism refers to the application of modern literary and historical-critical methods to study the Bible, aiming to understand its composition, sources, authorship, and literary development. This approach encompasses several interrelated disciplines, including higher criticism, literary criticism, form criticism, and tradition criticism. Biblical criticism seeks to appreciate the text through a fuller understanding of its literary history and message, rather than to lead readers to faith or action. The field has evolved to include various methodologies such as source criticism, which attempts to identify underlying documents or sources, and redaction criticism, which examines how authors shape inherited traditions according to their theological or narrative interests. While some may perceive biblical criticism as deprecating the Word of God, its intent is to develop a discriminatory understanding of the text's historical and cultural context. However, the application of critical methods to biblical studies has been a subject of debate, particularly regarding its compatibility with traditional views of biblical authorship and infallibility.
Logos has a “All” search, which is based on the entire Logos library. Here is the output…
In biblical studies, criticism refers to various analytical approaches used to study and interpret biblical texts. These approaches include historical criticism, which examines the historical context of the texts, and literary criticism, which focuses on the formal features of the text and the role of the reader. Some scholars have applied postmodern theories to biblical studies, though there is debate about what constitutes "post-modernism" in this context. Reader-oriented criticism and feminist criticism are also mentioned as approaches within biblical studies. Additionally, the field has seen the application of computer technology to assist in textual analysis and hypothesis formulation. It's important to note that while historical criticism has been dominant, there is ongoing discussion about the need to consider not just historical reliability but also the meaning and relevance of biblical texts for contemporary readers.
Comparing the 2 results, clearly there is nothing special in the “ALL” search compared to the “BOOKS” search. The above 2 pale in comparison to what comes out of the 5 free AI. Here is the Word file with all 5 outputs. See for yourself.
[Justin Gatlin: The upside is that basing results in specific resources means that the answers aren't just made up, which your other examples frequently do.]
This is exactly, why I’m asking Logos to take the output and rewrite the summary using citations.
I believe in a Win-Win-Win God.
0 -
Hi MJ!
could you please tell which of the 6 discussed in this article are chatbots vs LLM? i want to be sure we do a correct comparison.
thanks
I believe in a Win-Win-Win God.
0 -
could you please tell which of the 6 discussed in this article are chatbots vs LLM?
I meant the chatbot run using LLM's as their data source. chatbots are the app/user interface; LLM's are the data. Logos provides a Logos search enhanced especially in the sequencing of potential answers from most to least likely pertinent. The synopsis is simply a synopsis of the top few potential answers. It does not seek "the answer" but rather to select the answers (plural) found in your library. A chatbot tries to find an answer (singular) to the question NOT sources for the answer as in Logos. Depending upon the chatbot it may document the sources of its information.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
1 -
REPLY TO JUSTIN GATLIN
[1 Cor 10:31: "then summarize the output after checking for accuracy using its reliable resources so that it can give citations in its summary like it does now."
Justin Gatlin: What you're asking for here is actual AI, which does not exist yet. ]
I am not smart enough to know what is “actual AI”. I’ve asked the tech person in my school to summarize many years of my student evaluations. He has said that AI can easily summarize if I get PDFs. So I don’t see why Logos can’t do this. If a business school tech person can do it, surely Logos can do it.
[Justin Gatlin: Logos responses are less detailed because they are based on your library. That's the downside. The upside is that basing results in specific resources means that the answers aren't just made up, which your other examples frequently do.]
Yes, what you say is true, BUT only when we select a “Books” search, which is based on each one’s personal library. Here is the output based on my library of 12k+ books...
In biblical studies, criticism refers to the application of modern literary and historical-critical methods to study the Bible, aiming to understand its composition, sources, authorship, and literary development. This approach encompasses several interrelated disciplines, including higher criticism, literary criticism, form criticism, and tradition criticism. Biblical criticism seeks to appreciate the text through a fuller understanding of its literary history and message, rather than to lead readers to faith or action. The field has evolved to include various methodologies such as source criticism, which attempts to identify underlying documents or sources, and redaction criticism, which examines how authors shape inherited traditions according to their theological or narrative interests. While some may perceive biblical criticism as deprecating the Word of God, its intent is to develop a discriminatory understanding of the text's historical and cultural context. However, the application of critical methods to biblical studies has been a subject of debate, particularly regarding its compatibility with traditional views of biblical authorship and infallibility.
Logos has a “All” search, which is based on the entire Logos library. Here is the output…
In biblical studies, criticism refers to various analytical approaches used to study and interpret biblical texts. These approaches include historical criticism, which examines the historical context of the texts, and literary criticism, which focuses on the formal features of the text and the role of the reader. Some scholars have applied postmodern theories to biblical studies, though there is debate about what constitutes "post-modernism" in this context. Reader-oriented criticism and feminist criticism are also mentioned as approaches within biblical studies. Additionally, the field has seen the application of computer technology to assist in textual analysis and hypothesis formulation. It's important to note that while historical criticism has been dominant, there is ongoing discussion about the need to consider not just historical reliability but also the meaning and relevance of biblical texts for contemporary readers.
Comparing the 2 results, clearly there is nothing special in the “ALL” search compared to the “BOOKS” search. The above 2 pale in comparison to what comes out of the 5 free AI. Here is the Word file with all 5 outputs.
See for yourself.
[Justin Gatlin: The upside is that basing results in specific resources means that the answers aren't just made up, which your other examples frequently do.]
This is exactly, why I’m asking Logos to take the output and rewrite the summary using citations.
I believe in a Win-Win-Win God.
0 -
I really don't see how Logos AI will ever be able to compete with the most popular chatbots run by multibillion dollar companies. I agree that there will always be room for improvement, but I do not think that providing answers that compete with these other chatbots should be the goal. For me, that's not the point of Logos AI, which exists primarily to help me navigate my library so I can search out answers myself.
One must use the tools at one's disposal in such a way as to take advantage of each tool's strengths. Logos AI absolutely helps me in ways that other chatbots never will be able to. But directly competing with those chatbots on answering general questions is neither financially feasible nor advantageous to its purpose. Logos AI pulls from hundreds to thousands of sources. The largest chatbots easily comb through millions of sources.
Logos AI compliments the use of other chatbots. But it does not replace it. As AI continues to advance I don't think that one single AI product will ever have all the answers.
4 -
Apples to oranges.
Maybe someday Logos can have that level of functionality, but that is not what Logos AI is currently meant to do.
0 -
@Christopher Randall. Thank you. I questioned the 5 AI engines what the differences are among the 5 engines and it gave me answers that were themselves different! Apparently, they themselves don’t know what exactly the others are doing.
@Christopher Randall. You said “ They also limit the AI output to giving just a brief synopsis or summary of the resource(s) instead of a more "robust" answer that you will see in the above-mentioned Large Language Model (LLM) AIs.”
I am going to show you that the information Logos generates is worse than what I get from guides. For example, iIf I enter criticism in “Factbook”, I see “Biblical criticism” as one of the topics that is auto populated. If I select “Biblical criticism”, the Lexham Bible Dictionary is the first link. The article in this dictionary is much better than the AI synopsis that we are given. Why are we paying for something that is giving inferior output?
The joke is that Logos doesn’t even give my Lexham Bible Dictionary as a resource in the summary. What explains that?
I believe in a Win-Win-Win God.
0 -
You should try the 5 free AI egines for free. You don't need Logos to do something that is done free elsewhere.
I believe in a Win-Win-Win God.
0 -
Since I am not a techie person, I didn't really understand your answer initially. So I also asked the 5 AI engines "what is the difference between chatbot and LLM" and the answers helped me truly understand your answer.
Given my understanding, it seems that Logos AI search is NEITHER a chatbot that gives A (singular) answer NOR a precise search. In fact, it seems to give us the WORST of both worlds. One way to come to this conclusion is to realize that Guides, Factbook of Logos 10 are all specialized searches that yields answers based on tagging and hence, precise. These old tools provided multiple resources that matched the criteria - kind of like Google search, but limited by the correctness and efficiency of tagging.
the new AI search is neither a chatbot nor a precise search and hence we have the worst of both worlds and people are happy with it? See my previous reply and how this Logos AI search didn't even pick up Lexham Bible Dictionary. Please, this is a joke.
What I am asking for is the best of both worlds, not the worst of both worlds.
I believe in a Win-Win-Win God.
0 -
I did a smart search on my books for biblical criticism and the 3rd book in the summary was indeed the Lexham Bible Dictionary, so I'm not sure what you did differently?
0 -
If we had 5 AI engines working on our individual libraries, wouldn't that be awesome? We don't quite have that, but Logos is offering the next best thing. In my experience, it isn't perfect, but it is very good. I rarely do fancy searches anymore because AI in Logos does much of that work for me—better than I can do it in most cases.
I don't need Logos to do AI searches of the Chat kind. I have free sources for that. I want the benefits of AI integrated with my specific library. Logos does that increasingly well.2 -
You have to ask Logos why they give different answers to different people asking the same question. Here is the question I posed: "What does criticism mean in biblical studies"
I believe in a Win-Win-Win God.
0 -
I totally agree with you. you are speaking my language. we have to use the tool that is most appropriate. I do not expect Logos to be competing with Gemini, Copilot, ChatGPT, Perplexity, Claude…i don't want them to waste resources competing with them either. so 100% with you.
But as i pointed out, Logos is giving me neither an answer like a chatbot, nor pulling out precise resources from my library (or its own library). When my question "What does criticism mean in biblical studies" doesn't pull up Lexham Bible Dictionary article whereas old Factbook tool (a precise search) pulls up LBD tells me that what we are served is AI in name but not AI in quality.
Do you know what Logos AI is trained on? its own resources?
I believe in a Win-Win-Win God.
0 -
I'm slowly sinking into 'who knows what Logos does' … didn't take long (absent a subscription, and I bet that's most Logosians). But I thought Logos was using a regular search, with 'AI' summarizing the 'best' answers. Or not.
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
Do you know what Logos AI is trained on?
@1Cor10 31 I have no idea. I do agree wholeheartedly that Logos AI can be improved upon when it comes to helping me study and navigate my library. Perhaps where we differ is that I find it already to be a significant asset as is.
0 -
@1Cor10 31 I have experimented a bit with your query—the result I got with Logos was very similar to yours.
I have spent many decades in the academic community and I recognize something going on with the AI style of response. The Logos response is one directed to someone already familiar with the field and just needs a quick memory jog and some references to the primary literature. The five AI engines you sampled are geared to a much more general audience and give more specific explanations or examples of the technical terms involved. It is much like undergraduate vs. graduate vs. post graduate literature.
Perhaps Logos could have a setting for the style of response its AI gives. Right now it is set to respond to an individual with familiarity with the topic or is one who has a high level of experience in complex topics and is comfortable burrowing down the research tree with modified queries until the information matches a comfortable level of sophistication.
This reminds me of the debate going on in my field of work in the early 1970’s when there was a mismatch between the topics being offered and the level of experience of the students. It has remained a constant reminder of our need to match the expectations of the client since I first read this essay in the early 1970’s.
From ChatGPT: "J. Dudley Herron wrote his famous essay “Elevate Them Guns a Little Lower” in 1971. The essay is widely recognized in the field of chemistry education and focuses on the challenges of teaching and learning science, emphasizing the need to adapt teaching methods to the cognitive abilities of students. The title humorously highlights the misalignment that can occur when educators aim too high, failing to meet students at their level of understanding."
As always, finding that perfect "elevation" is really more of an art than a science. ;)
—SollyJoseph F. Sollenberger, Jr.
0 -
@Aaron Hamilton [I have no idea.] @DMB [I thought Logos was using a regular search, with 'AI' summarizing the 'best' answers. Or not.]
calling tech stalwarts in the forum like @MJ. Smith and @Dave Hooton and others to chime in. what exactly is Logos doing?
here is another alternative: Why can't it extract the keywords from the questions we pose, then search its database for tags that match the keywords and then give a simple link to all the resources that match the criteria.
I believe in a Win-Win-Win God.
0 -
I understand what you are trying to state, but I think you are misinformed on the purpose (which other people are pointing out - maybe take a breather). The Facebook you mention is taking you to a direct source for you to read. The AI Synopsis, is just summarizing based on a set of predetermined rules using the top 5 (not always 5 sources are used). It is not meant to give you an indepth research that the authors of a dictionary is going to give you, instead a brief snyposis of the top 5 sources. Were you under the impression it did more than that? A LLM chatbot (ChatGPT, Claude…) might be able to do that, but Logos is not designed to do that, it is just summarizing to get you to a starting point on what to research. For some people, it's answers are enough based on their question. I have an idea of Logos limits by playing with it so I base my questions on those limits, while continuing to play with it.
Another way of looking at this is, Logos AI points you in a direction to do your own research while sometimes giving you enough of an answer at the getgo; whereas, LLM Chatbots are able to "do" the work for you, but even then it is better to validiate by going to a direct source.
It comes down to, what are you wanting. You sound like you're wanting what a chatbot can give you, but Logos AI isn't designed for that at the moment. If that doesn't suffice, then it is not useful to you. I wouldn't call it a joke though. I don't know the data algorithms it uses to determine what it considers top resources, but I have found it to be helpful. Would I say some resources it points to may not be what I would choose, sure, but it still is going in the right direction. It is finding resources that contain information based on my query.1 -
Mark Barnes described the workings of the smart search as “In Smart Search, we perform a very loose search without AI. That finds 50 articles that roughly match your query. We then use AI to identify a snippet from each article that most closely matches your query. We don't rewrite any text; we only identify the most relevant existing text in the book. Once we have these snippets, we use AI to re-order them to put the most relevant ones at the top.”1
1New feature: Smart Search - Logos Forums accessed 17 September 2024
What actually concerns me the most about the many threads on AI is the degree to which some people are willing to trust sources that they don't understand. It encourages me to push Logos to be extremely conservative in adding AI features and makes me flat out opposed to the integration of chatbots. Those who wish to use the chatbots, as I do for certain tasks, have access to chatbots of their choice elsewhere.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
6 -
This is a really interesting conversation. I've been somewhat obsessed lately with AI and while I've learned alot, I have barely scratched the surface. The big AI models (openai, gemini, claude, llama,…) don't do searches for answers. Based on the models, it guesses the next best words. And they are so good at it, it's almost scary. While no other companies will be able to come close to competing with the big boys, the latest thing is something called agentic AI. It allows smaller models to be augmented with other data/info and used as though as its part of the bigger model. Many companies have API's that allow you to utilize their info in custom AI applications. This would be a perfect use case for a Logos API that would allow use of Logos resources along with the "intelligence" of the larger generic model. Logos could charge for the API access and make money giving outside access while keeping the Smart Search functionality for the internal use it was meant to be, a search engine augmented by ai.
0 -
If you really want to understand AI, I highly recommend
Where I get off the AI train is simple - I believe that we read scripture in order to listen to God and allow ourselves to be shaped by God. I don't read scripture to collect fool's gold of information about scripture.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
3 -
I am less than impressed with the AI Search built into Logos. Yes, it is helpful for identifying resources in my Library, and it will suggest volumes I do not own.
But Logos could make this a far more robust and useful tool. I would like to feed the Logos AI a prompt that would actually help with my research. For example, a simple query like "How has the development of Trinitarian theology influenced the development of pastoral theology over the course of church history?" or "Trace the contributions of Gregory of Nazianzus to pastoral theology and explain why his thinking should be important to pastors today."
Limit the search to the Logos catalog. Now I've got something useful.
1
