Shepherds Notes - Quality???

Page 1 of 2 (28 items) 1 2 Next >
This post has 27 Replies | 2 Followers

Posts 408
Roy | Forum Activity | Posted: Tue, Feb 23 2016 7:58 AM

I have been watching the Shepherds Notes sale trying to decide if I wanted to buy into the books.

I decided to try one or two first and picked up Genesis. I am somewhat disappointed. Not with the text itself, but with the lack of attention to the scripture linking that was put into place.

I an fully aware that Vyrso books do not have the same level of "added value" that regular LOGOS books do. That does not however mean that that fact can be used as an excuse for shoddy work. The "added value" that is put in place needs to be correctly done or it is of no value.

What I am referring to are the scripture links that go to wrong verses.  If it were just a few that could be overlooked but there are just too many of them. How do they add in the links? Far too often there is a verse reference to one bible book done correctly, and then the very next reference (going by just chapter and verse without referencing the book name) is linked to the wrong book,  the book from the previous reference, but the text clearly meaning something different.

The attached clip is just one example. The early references work properly, but then you get to the ones where the book name is not present. It is clear from the text that they reference Genesis, but they both link to ...Romans.

I understand not to expect the same "Level" of linking, but if you are going to bother to put links in, at least make sure they are right! Is this the level of overall "Value" I can expect from the rest of the books?

Posts 29256
Forum MVP
JT (alabama24) | Forum Activity | Replied: Tue, Feb 23 2016 8:12 AM

Roy:
How do they add in the links

These are done automatically. It works great when the reference is explicit. In this case, it's not. The reason it links to Romans is because that was the last clear reference. 

Roy:
 Is this the level of overall "Value" I can expect from the rest of the books?

Yep. You will need to decide if the "value" of the resource is worth it for you. It is clearly of more value to me to have this resource in Vyrso rather than in kindle. In my mind the choice is Vyrso or nothing. 

macOS, iOS & iPadOS | Logs |  Install

Posts 408
Roy | Forum Activity | Replied: Tue, Feb 23 2016 8:25 AM

This is what I have been noticing. Every time the "Wrong" link is non-explicit.

I am not sure if I wouldn't prefer to see no links at all instead of seeing the bad ones. It just screams "Poor Quality" to me. Is it wrong to have a certain level of expectation? To expect them to at least get "Right" the work they do try to do?

I figured that there is some sort of macro doing the tagging. I wonder if it wouldn't be a better idea not tag any reference that isn't explicit.

Posts 2771
Jan Krohn | Forum Activity | Replied: Tue, Feb 23 2016 8:56 AM

Perhaps the automatic reference finder logic is flawed.

If I have a book about Genesis, then a non-explicit link is surely Romans or something else Hmm

(It might not hurt to code a slightly modified auto-tagger for resources about one specific Bible book.)

Past IT Consultant. Past Mission Worker. Entrepreneur. Seminary Student (VIU).
Christian Debate Forum --- Auferstanden! Blog

Posts 1080
William Gabriel | Forum Activity | Replied: Tue, Feb 23 2016 9:34 AM

Jan Krohn:

Perhaps the automatic reference finder logic is flawed.

If I have a book about Genesis, then a non-explicit link is surely Romans or something else Hmm

(It might not hurt to code a slightly modified auto-tagger for resources about one specific Bible book.)

Particularly when you come to a Heading. It may have made sense to keep it Romans if it was in the same paragraph, probably a tossup if it's in the same section (if I were programming it, I'd look for verse continuity with previous references), but as soon as you move to a new section, it will mostly likely go to the default book (Genesis, in this case).

Posts 31943
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Feb 23 2016 12:18 PM

Unfortunately, here I would have to say that the expectations are flawed. When dealing with natural language processing such as adding tagging via an algorithm, one has to anticipate a certain level of error which is always higher than we would like.

If, for example we estimate that using that last Bible booked referenced is correct just 85% of the time would you rather have 1000 references unlinked or 850 linked correctly and 150 errors? Generally people prefer the 850 to be linked correctly. The only solution short of a brilliant mind devising a better algorithm is to provide the user the opportunity to correct the errors and to allow a certain level of identical corrections to update the resource. In other words, crowd source a manual editing process.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 408
Roy | Forum Activity | Replied: Tue, Feb 23 2016 1:23 PM

Humm...

First I would opt for the user to have the ability to edit their own copy of the resource. Whither the changes we make would then be passed back up to LOGOS/Vyros would be up to LOGOS in how they implemented the feature.

As far as putting up with the bad links to get the good?

If that were the only way to get anything, then yes, I guess so. However, I think that, as mentioned earlier, the algorithm used to check for "Valid" scripture references is flawed. If you are going to have someone go back and verify that the links have been tagged correctly, then use the current system. If there is not going to be any kind of error checking, then the tagging system should be modified (or a different system used) to include only those detected references that are explicit. Ones that include Book Chapter and Verse.

Why? Again (in my opinion) if the tagging is wrong, then it is worthless. I remember seeing recently in the Vyrso book "Seeking Allah, Finding Jesus" where the author mentioned a verse from the Koran by reference. Guess what! The book was tagged to go to a Bible verse. That kind of error is just crazy.

M.J. When you said...

"The only solution short of a brilliant mind devising a better algorithm is to provide the user the opportunity to correct the errors and to allow a certain level of identical corrections to update the resource. In other words, crowd source a manual editing process."

It would not be 'that hard' to develop the system to check first for explicit references, then if you wanted it to, check for possible implicit references. If the tagging system finds one, have it ask "Is this in Romans?" or whatever Book it is guessing at. If yes, a single click or key tap allows it to continue. If not then the user can supply the correct answer (from a pick list or some such).

It goes back to the old adage, "If a job is worth doing, it is worth doing correctly".

Posts 990
David Carter | Forum Activity | Replied: Tue, Feb 23 2016 1:34 PM

Roy:
I an fully aware that Vyrso books do not have the same level of "added value" that regular LOGOS books do. That does not however mean that that fact can be used as an excuse for shoddy work. The "added value" that is put in place needs to be correctly done or it is of no value.

Like these examples from Psalms you mean?

Posts 29256
Forum MVP
JT (alabama24) | Forum Activity | Replied: Tue, Feb 23 2016 1:35 PM

I am never going to argue against a "job well done," however... We have been told that Vyrso is kind of like a hobby, and has very small profit margins. I would rather see this resource in Vyrso with the wrong hyperlinks, than to not have it at all... Especially when you are picking it up for a buck. Wink

[ Think about that! A buck. How much do you think goes to the publisher/author? 70%? Then remember that visa/MasterCard takes their cut. FL may end up selling that resource to you with only a quarter for them, which goes to pay for web hosting, app development, staff, etc.]

By the way, if you are interested in "the right people" seeing this, you either need to post in the Vyrso forum or send an email. 

macOS, iOS & iPadOS | Logs |  Install

Posts 990
David Carter | Forum Activity | Replied: Tue, Feb 23 2016 1:40 PM

alabama24:
I would rather see this resource in Vyrso with the wrong hyperlinks, than to not have it at all... Especially when you are picking it up for a buck.

If a job's worth doing, it's worth doing well - as my dear old dad used to say Yes

Personally, I would rather give my buck to Amazon and have it without hyperlinks than have this mess.

Posts 31943
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Feb 23 2016 2:06 PM

Roy:

It would not be 'that hard' to develop the system to check first for explicit references, then if you wanted it to, check for possible implicit references. If the tagging system finds one, have it ask "Is this in Romans?" or whatever Book it is guessing at. If yes, a single click or key tap allows it to continue. If not then the user can supply the correct answer (from a pick list or some such).

It goes back to the old adage, "If a job is worth doing, it is worth doing correctly".

To the best of my knowledge, the algorithm used by Logos is state of the art. If you know of a better algorithm, I'm sure they'd be happy to upgrade. If you understand the algorithm there are no "crazy" errors, there are simply cases that the algorithm cannot assign correctly because it lacks the information to identify them correctly. What is immediately obvious to the human eye is often very difficult to define to the computer. If you want human interaction when the tagger knows it may have a problem, you need to pay Logos prices not Vyrso prices.

Roy:

As far as putting up with the bad links to get the good?

If that were the only way to get anything, then yes, I guess so.

It is. That is precisely why we have what we have.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 31943
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Feb 23 2016 2:24 PM

David Carter:
If a job's worth doing, it's worth doing well - as my dear old dad used to say Yes

Do you know of any algorithm that has a higher accuracy rate? If not, my friend, it is being done well. Look - it's not that I'm a fan defending Logos, its that I have taken enough classes in Natural Language Processing and designing algorithms for unstructured data and data mining to know what reasonable expectations are. Yes, we would all like to improve the accuracy rate but it helps only if you have a concrete way to improve it.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 408
Roy | Forum Activity | Replied: Tue, Feb 23 2016 2:47 PM

MJ. Smith:
To the best of my knowledge, the algorithm used by Logos is state of the art. If you know of a better algorithm, I'm sure they'd be happy to upgrade. If you understand the algorithm there are no "crazy" errors, there are simply cases that the algorithm cannot assign correctly because it lacks the information to identify them correctly. What is immediately obvious to the human eye is often very difficult to define to the computer. If you want human interaction when the tagger knows it may have a problem, you need to pay Logos prices not Vyrso prices.

Nah... I do not have a "Better Idea of how to do it". I don't know exactly how they have that task coded out, but I can guess. I have done "some" coding in the past to parse out strings of text (processing EDI transactions) in order to get at the possible buried data. I understand "How" it is done, and also have a rough idea as to how I would try to code it in this instance. (But I admit, my code would most likely be a real kludge.)

I also understand the really low price and admit it may be about time for a ..."Good grief Roy, it's only a buck! Quit complaining!!"...

As for me, and this is just me, I think (maybe) I would however prefer to see the book in Vyrso with no tagging (and at that price point I would actually  expect that-no tagging) than to see all of the errors. When I see that the time was taken to add a tag, I guess I expect that tag to work correctly. That is why I said a parser that found only explicit references "may" be a better idea. While I realized that it was the minimal level/quantity of the tagging being done by Vyrso that made for the lower price, I guess I did not expect the human factor to be that low.

I can live with the errors considering the price point, I just have to lower my expectations. Ok... I'll quit complaining now Zip it!

Posts 990
David Carter | Forum Activity | Replied: Tue, Feb 23 2016 3:07 PM

MJ. Smith:
Do you know of any algorithm that has a higher accuracy rate?

No, but I do have (somewhere) a 15 year old (or so) Perl script that I used to run on one of my old web sites to convert Bible  references to hyperlinks that performed exactly as the Vyrso one does - it was great until the "base" book changed!

MJ. Smith:
Yes, we would all like to improve the accuracy rate but it helps only if you have a concrete way to improve it.

I used to change all the wrong ones manually - it was a very slow process Sad.

There is only so much you can do without human intervention, but I guess this would be an added cost  too great for Vyrso to bear unfortunately.

Posts 1080
William Gabriel | Forum Activity | Replied: Tue, Feb 23 2016 3:08 PM

MJ. Smith:

The only solution short of a brilliant mind devising a better algorithm is to provide the user the opportunity to correct the errors and to allow a certain level of identical corrections to update the resource. In other words, crowd source a manual editing process.

I think an automated crowdsourcing typo solution would be excellent as it would fix many problems quickly and it would add value to the product for future sales, too. I think it would be compelling for Logos and Vyrso. (By the way, if they ever do this, I think that it should be based on a reputation score assigned to each user based on quality)

Another option would be to offer a public API for the algorithm and solicit public feedback/trials, like what Netflix did with their recommendation algorithm. I would have fun with that.

Posts 1080
William Gabriel | Forum Activity | Replied: Tue, Feb 23 2016 3:09 PM

David Carter:

Personally, I would rather give my buck to Amazon and have it without hyperlinks than have this mess.

Amazon will never change the source unless the publisher does. Logos/Vyrso/Faithlife has a track record of improving products over time, albeit slowly. But you at least have that chance with a Logos system book.

Posts 990
David Carter | Forum Activity | Replied: Tue, Feb 23 2016 3:14 PM

William Gabriel:
But you at least have that chance with a Logos system book.

It would be quite nice if my Vyrso Shepherd's Notes books were one day to be made into full Logos editions Big SmileBig SmileBig Smile

Posts 31943
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Feb 23 2016 3:42 PM

Roy:
I guess I did not expect the human factor to be that low.

If I understand the structure of Vyrso, the human factor is as near zero as possible. Bob has indicated they get mere pennies on many Vyrso resources. I suspect it is a feed driven by the publishers.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 2589
Lee | Forum Activity | Replied: Tue, Feb 23 2016 3:47 PM

A long time suggestion from me and others: typo hunting by credible users, for bounty (credit). Win-win.

Posts 29256
Forum MVP
JT (alabama24) | Forum Activity | Replied: Tue, Feb 23 2016 5:39 PM

Lee:

A long time suggestion from me and others: typo hunting by credible users, for bounty (credit). Win-win.

Yes

macOS, iOS & iPadOS | Logs |  Install

Page 1 of 2 (28 items) 1 2 Next > | RSS