Sermon Assistant Review

I have used the new Sermon Assistant for a number of sermons, looking only for Illustrations. Here is my take on it:

On the positive side, this tool seems capable of finding illustrations well outside my library resources. It also can stimulate thought with what it finds.
However, its responses are frequently too generic outside the Bible. It refers to people, stories, or events, without saying who the people, stories, or events are. Saying that someone did something isn't precise enough to put in a sermon. (I have been limiting my choices to serious biblical or historical illustrations for adults.)
Its responses seem quite limited. I suspect this is due to the range of material that is being consulted by the software.
It takes awhile to determine the right phrase or question to ask to get results I am interested in. It would help to be able to further refine a query (as in ChatGPT) rather than try to reword the question or phrase. For example, "Give specific references" when it fails to do so.
Its personal illustrations are not useful since they are not, in fact, personal. The idea might be to stimulate our thought, but I think we'd be just as well off without this option.

So far it has been of very limited help. There is certainly potential here, but it isn't there yet.

Suggestion: If it is not doing so (and I don't see signs that it is) I'd suggest applying the tool to our library resources. The current illustrations tool is quite limited. AI might be able to expand what we can find, especially when there is no topical heading for what we want to locate.

Find more posts tagged with

Comments

MJ. Smith

Nice review Mark. I would also fault it (as I do Logos) for not providing the traditional illustrations:

Desert fathers and mothers
Exempla
authors such as Anthony de Mello, Idries Shah (used by Fr. Keating), Yoel Hoffman (koans)

Stephan Thelen

https://community.logos.com/discussion/comment/1287281#Comment_1287281

I fully agree with your assessment. I also regularly create my sermons with logos and always use (and look for) matching illustrations. The need is there, because the corresponding tool in the Bible passage assistant is not very helpful: there are far too many results. On the other hand, the illustrations listed there from my own library have much more depth. In my opinion, the AI only provides flat illustrations that are thought-provoking at best, but can hardly be integrated into your own sermon as they are - especially since I still lack the connection to the Deepl translation.

Mark Barnes (Logos)

It would be great to have examples of the prompts you're using to find illustrations so we use them as test cases to try and improve.

Mark Smith

https://community.logos.com/discussion/comment/1287587#Comment_1287587

It would be great to have examples of the prompts you're using to find illustrations so we use them as test cases to try and improve.

Now that I've forgotten them.

When I have time I'll try to recreate them.

Mark Smith

https://community.logos.com/discussion/comment/1287604#Comment_1287604

It would be great to have examples of the prompts you're using to find illustrations so we use them as test cases to try and improve.

Here are some:

overcoming our past

being presented with a great opportunity

making a rash vow

making a hasty vow

people defeated by personal weakness

strong start weak ending

Mark Barnes (Logos)

https://community.logos.com/discussion/comment/1287629#Comment_1287629

Thanks, Mark. That's very helpful. We've been prototyping some improvements to this tool, and I think that we will be able to deliver some improvements.

You may also find that more detailed prompts will make a big difference. For example, switching from overcoming our past to overcoming our past failures improved the output considerably for me (at least on my prototype). Here are the five it gave me:

Imagine a gardener who plants beautiful flowers in his garden but forgets to water them regularly. As a result, the flowers wither and die. But instead of giving up, the gardener learns from his mistake. He starts watering the new flowers consistently, ensuring their growth and beauty. In the same way, we can overcome our past failures by recognizing where we went wrong and making positive changes. Just like the gardener didn't let one mistake define his entire garden, we shouldn't let our past failures define our future. Learn, grow, and bloom!
Picture a young archer learning to shoot arrows at a target. At first, many of his shots miss the mark, falling far from the bullseye. But instead of giving up in frustration, he practices diligently, adjusting his aim with each attempt. Soon, his arrows begin hitting closer to the center, until finally, he scores a perfect bullseye. Similarly, in life, our past failures may cause us to miss the mark, but with perseverance and practice, we can improve and eventually hit the target. Let your past failures be stepping stones to future success!
In the Bible, we read about Peter, who denied knowing Jesus three times during His trial. Despite this failure, Peter went on to become a strong leader in the early church. It was through his repentance, growth, and faith in God's forgiveness that Peter overcame his past mistakes. Just like Peter, we too can rise above our failures by seeking forgiveness, learning from our errors, and trusting in God's grace to transform us. Our past does not define us; it refines us for a better future.
In history, we find the story of Thomas Edison, who failed over a thousand times when trying to invent the light bulb. Each failure taught him what didn't work, bringing him one step closer to success. When asked about his failures, Edison famously said, 'I have not failed. I've just found 10,000 ways that won't work.' Similarly, our past failures can be valuable lessons that guide us towards eventual triumph. Embrace your failures as opportunities for growth and discovery, just like Edison did in his pursuit of light.
Consider a child learning to ride a bike for the first time. Initially, they may wobble, lose balance, and fall down. It's a normal part of the learning process. But with each fall, the child gets back up, tries again, and eventually masters the art of riding a bike. Our past failures are like those falls - they may knock us down temporarily, but they should never stop us from getting back up and trying again. Remember, even the most skilled bikers were beginners once; they didn't give up, and neither should you!

Mark Smith

https://community.logos.com/discussion/comment/1287636#Comment_1287636

For example, switching from overcoming our past to overcoming our past failures

Except I wanted examples of people who had difficult pasts that they overcame, not their own failures, therefore the "overcoming our past".

When I select Historical illustrations is where the results get fuzzy and less helpful, yet that is exactly the area I'd like to see good results. Of the five results above, one is a decent historical illustration.

One other comment, and that is about learning what sorts of phrase or questions produce usable results. If one has to work too hard at this, it becomes demotivating to use the tool. I realize there is only so much that can be done, but it isn't always obvious how to phrase a query to get good results. This is also why I suggested follow-up questions or statements. Instead of starting over, let me try to refine or better describe the sorts of results I prefer. In the example you gave, I would have wanted to add "Give me more historical examples with the names of those involved".

Mark Smith

https://community.logos.com/discussion/comment/1287636#Comment_1287636

Mark, one other follow-up.

Where is this tool searching for information? In a few cases where I've tried it in both, the historical responses look very similar to those from ChatGPT, though there are differences. Are both tools searching the same 'database'? Are you simply porting ChatGPT into Logos and therefore the 'database' is identical and the results should also align very well?

Also, will there be a way (some day) to apply this tool to our library of resources?

Mark Barnes (Logos)

https://community.logos.com/discussion/comment/1287647#Comment_1287647

In a few cases where I've tried it in both, the historical responses look very similar to those from ChatGPT, though there are differences. Are both tools searching the same 'database'? Are you simply porting ChatGPT into Logos and therefore the 'database' is identical and the results should also align very well?

Neither ChatGPT nor our AI has a database as such.

The different AI features that we're currently testing (or building for the future) work in different ways. Most of them rely on content from your library or the wider Logos ecosystem (or, in the case of Discussion Questions, on your own content). But we're experimenting a bit with different approaches, and Illustrations is currently the only AI feature in beta that does NOT use that content – and maybe that partly explains your dissatisfaction with it compared with other features, which is good for us to know.

We agree that most of the value we can add with AI comes when we combine AI with our enormous ecosystem of books and datasets, but perhaps sometimes we can also add value by integrating relatively simple functionality right into the place where you write your sermons and providing a simple UX to do it. That's what we're currently doing with the illustrations feature.

All these features are subject to frequent review, and some of them will change over time as the technology matures or we can come up with better approaches. If our current approach with illustrations proves not to be good enough, even after we've refined it, then perhaps we'll need to look into better utilizing our existing data.

MJ. Smith

https://community.logos.com/discussion/comment/1287827#Comment_1287827

Neither ChatGPT nor our AI has a database as such.

[1706.03762] Attention Is All You Need (arxiv.org) provides the information on the Generative Pre-trained Transformer (GPT) data model.

Mark Smith

https://community.logos.com/discussion/comment/1287827#Comment_1287827

Thanks for the helpful reply, Mark, and for letting us know what you hope to be able to deliver wrt to illustrations.