Feedback on Books Smart Search
I am almost positive others have mentioned this @Mark Barnes (Logos) , but my biggest disappointment with Smart Books search is the choices it makes in the top books returned in searches. So many times, the books it chooses couldn't be further from even my top half of resources! It seems to choose from (imho) obscure denominations and authors - none of my more notable resources are used. Of course, it comes nowhere close to looking at my priority list - and often I am asking questions that should have answers in commentaries, not that it's the only solution.
I am not against seeing obscure resources show up, if the question is truly an obscure one that many books will not be helpful in response.
It would be great if more could be done to make Smart Search better at knowing our preferences in resources, to at least start there. Can it learn how to peruse our priority list first, or ratings (we could then "train" it on our own if we don't like the choices), or some other clever way to train it?
Comments
-
I think this applies to the Search Synopsis also… it seems to only synopsize the first 3-4 books returned from the smart search. I wish that could be expanded to at least 10 or so.
0 -
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
If we did this, how should we determine what your "notable" books are? Which do you (or anyone else) think are the biggest signals that a book is valuable to you?
- How frequently it's opened?
- How much of it you've read?
- What star rating it has?
- Whether you've prioritized it?
- What community rating it has?
- How many people own it?
- Something else?
0 -
3 and 4 seem to give the user the most control. 1 and 2 would cause me to be concerned about what resources I am opening up. I think the #1 solution would be to entirely redesign prioritized resources (long overdue) and integrate that into smart search prioritization.
0 -
While I understand the OP comments, I think users should consider using smart search on collections of books that allow for a more "targeted" smart search. I like getting resources I don't expect when I search all or all my books. However, if I had to choose from Mark's list I also prefer 3 or 4, but most people won't rate their monographs or prioritize them. Monographs is where smart search really helps.
3 -
@Mark Barnes (Logos) I agree with @John Fidel on this.
- Something else?-Show AI results from a collection in the synopsis and relevant listings first, then AI search results from the whole library by relevance below that.
- How frequently it's opened? -not practical if a new user
- How much of it you've read?-not practical if a new user
- What star rating it has?-not practical if a new user
- Whether you've prioritized it?- new users most likely have not done this yet
- What community rating it has?- same and difficult if the user hasn't become familiar with the resource, especially monographs
- How many people own it?-not practical
If I understand your question Mark, a new users may want to see what AI finds in their whole library at first until they become more familiar with their books. They may be more confused if the search were to be automatically sorted by any of your listed bullet points, because they may not have understood or had experience with any of those things yet. The list assumes a knowledge of, all of their books, ratings, prioritization, etc.
Emphasis on video training on creating and choosing a collection to do AI search would be more beneficial in my opinion.
Too soon old. Too late smart.
0 -
Emphasis on video training on creating and choosing a collection to do AI search would be more beneficial in my opinion.
Does the video at the top of https://support.logos.com/hc/en-us/articles/30426521209869-How-to-use-AI-Tools-for-Research-and-Writing help with this?
It doesn't talk about creating collections - but that is covered in https://support.logos.com/hc/en-us/articles/360016299092-Create-and-Use-Resource-Collections
1 -
- How frequently it's opened? - no, that reflects particular projects not usefulness
- How much of it you've read? - no, I read a lot of theological trash
- What star rating it has? - no, I don't set stars
- Whether you've prioritized it? - this would be the best among this list
- What community rating it has? - more likely the inverse of the community rating - I dislike the community ratings and highlights
- How many people own it? - more likely the inverse of ownership
- Something else? Allow users to specify resources to use … oh wait, you already do that if we switch to a book search
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
What about having several different options?
The first option is to search as it does now.
A second option will search all but will prioritize books you've starred or prioritized (possibly even fine tune this to allow you to select only books you've ranked 4 or 5 stars, or even select only books you've ranked 1 star, for example).
And I'm sure there's more variations.
What may also be helpful is to allow us to make a collection of books that would ALWAYS be excluded from searches (unless you specifically chose to include them)?
0 -
I would go with searching Prioritized books first. Most people have done that; maybe new users haven't, but just like in pre-AI search days, seeing books that are not their faves come up, they will learn to prioritize. Ratings has only been used for collections as far as I know, so searching based on ratings introduces a whole new dimension that may not work well if a user doesn't rate things correctly and comprehensively.
I would not be a fan at all of using collections - I have gotten away from collections as there were some real overhead issues with using them too much in the past - not sure if that still holds or not. But I wouldn't want to go to all the trouble to create collections just for a 'smart' search. It is nice to know I can do that now, if I want. I just would like to see the real power of the AI search - with just a little nudge from me, "check these out first".
If you think about it, isn't that consistent with the "learn" part of an AI engine, that it gets clues as to what choices a user makes and factors that in - maybe not entirely, but it is factored in? What better clue thann the priority lists we have all set up?
For example, to show I am looking to just make the AI smarter, not do its job for it (which imho is what building collections to have it search would be), I would be fine if it made the 1st result some obscure book because it happened to be BETTER than my prioritized books for a given search! In that case, I would know that if I see a non-prioritized book 1st in the search results, that must have some great points that caused the AI engine to push it to the top!
I am only looking for a little boost at the top of the smart search from my own faves. And we do have to be careful over-engineering this, in other words trying to "out-AI" the AI engine. Seems like most if not all like using prioritization, how about just one small step and see how it works? Try it in beta and see if it passes muster?
0 -
Thank you for the links Graham. These are good training videos.
The first linked video was close to what I was referring to, but it did not explain or use the word "collection" when referencing the drop down to select alternatives in "your books". It uses the the phrase " a subset of books" and moves so quickly a viewer couldn't see that the subset chosen was from a collection.
If I were a new user, which I'm not, that would have been helpful to mention and if they didn't know how to create a collection the second link would show how, possibly adding that link to the end of the first video would be helpful in directing them right to it.
I guess what I was saying previously is when a new user doesn't know about, or how to use collections, it needs to be made obvious in some way to them in search, or AI search training videos. Those minor details can make a big difference to a new user's understanding and success in using the tool.
Too soon old. Too late smart.
0 -
You can do a smart search on books based on their star rating already. For example, rating:5 will get all of your 5 star books, or rating:>=4 will get you all of your books with a rating of at least 4 stars.
Using Logos as a pastor, seminary professor, and Tyndale author
0 -
They may be more confused if the search were to be automatically sorted by any of your listed bullet points, because they may not have understood or had experience with any of those things yet.
To be clear, I'm NOT suggesting that we sort the list by any of the things I was asking about. The sorting works reasonably well right now, and certainly better than sorting by rating or any of those other criteria.
If we did make changes here, they would be subtle. Our current algorithm gives every result a score. Maybe we boost the score by 5% if the user has rated the book as 5-stars, but reduce the score by 5% if the user has rated it 1-star. Or a 3% boost if the user has prioritized the book. With an approach like that, you should see books they like higher up the list, but it wouldn't take all the "smartness" out of the existing ranking.
There's a risk, though, that such changes would make the ranking objectively worse because maybe a book you like answers your question less well than a book you don't like.
0 -
I would like to plead the case again that the Search Synopsis should look at more than the top 3-4 books/results, but expand to the first 5-10 books/results.
1 -
Mark, when you quote like this, I do not know who you are quoting. Please show whom you are quoting.
1 -
I would probably be just as effective to prioritize standard reference works and academic publishers with the same kind of light extra weight and to slightly underweight ebooks vs research editions. If one of the IVP black dictionaries and a devotional ebook both seem to address the same question, most users probably want the more authoritative source.
Using Logos as a pastor, seminary professor, and Tyndale author
0 -
If we did make changes here, they would be subtle. Our current algorithm gives every result a score. Maybe we boost the score by 5% if the user has rated the book as 5-stars, but reduce the score by 5% if the user has rated it 1-star. Or a 3% boost if the user has prioritized the book. With an approach like that, you should see books they like higher up the list, but it wouldn't take all the "smartness" out of the existing ranking.
@Mark Barnes (Logos) Okay, I understand better now. I still think ratings are unpredictable but, I could see prioritization and collections being used. The only concern I have is are new users or anyone else that is not a part of this discussion going to understand (maybe they won't care),how this will work or how it will affect their results, therefore, how does it encourage them to prioritize or make collections in order to fine tune the results, since all of this happens in the background.
There's a risk, though, that such changes would make the ranking objectively worse because maybe a book you like answers your question less well than a book you don't like.
I agree. That is why I was concerned about AI giving preference to books I already like to begin with. One of the features I like about AI is that it searches ALL of my 24000 book library in order to introduce me to something I possibly haven't explored previously. I might learn something new and it may get me out of a rut… or not.😊 The reason I suggested collections is because many of my collections are not filled with my favorites especially but, are created based on subjects, types or mytags.
Too soon old. Too late smart.
0
