Quality control in Logos 6 - a new perspective

Recent discussions have highlighted yet how promising Logos 6 is in terms of biblical studies. I think "promise" is a very appropriate word because it has that double-edge: a promise holds hope yet is not always fulfilled.

For instance, searches of cultural concepts or other tagged entities shows a lot of gaps and inconsistencies in tagging. This is a quality control problem.

I think that quality control is the leading issue with the ongoing development of Logos. I don't only think so, I know so. I know that because I participated in a development project for several months and had an insider view of this problem. I became increasingly uncomfortable with my participation in the project because I could see how it was being pushed steadily forward for earlier than later release at the cost of cutting many corners and releasing a product that will be flawed from the start. Those who led the project knew that but they are pressing on. I am sad when I think about the marketing that is being done in anticipation of release, having the awareness that there will be some very disappointed buyers out there.

And yet, it does not need to be so! This post, if anything, is yet another plea to reverse what has become a consistent blotch on a product that is otherwise exciting and holds so much promise.

As it is, I could use some of the tagging "to save hours" (this is a key marketing promise). This can be done by automating what would otherwise need to be done manually. Combing large sections for data over and over is extremely time-consuming. BUT unfortunately, at this point, I cannot rely on results. So, I still have to do the manual work to ascertain what I find. And this is not just compensating for the occasional human error. This is more systemic.

Jesus told the parable of two sons confronted by the request to do right. One said "I will" but did not do it. The other said "no" but then ended up doing it. Logos, what kind of "son" will you prove to be?

This being said, if I and others simply don't understand that it ain't so, perhaps some explaining would help. Is Logos so broke it must release stuff asap or die and thus considers it better overall to release incomplete or error-filled, bug-ridden products? I do not mean to slander those who may be hard-working and trying as hard as they realistically can. But I am perplexed (and in the case of the experience I allude to above, dismayed) and above all, desire a good outcome.

Find more posts tagged with

Comments

Rosie Perera

Luke 14:28-30 also comes to mind.

Brian Davidson

I also wish Logos would make quality control a higher priority.

Rene Atchley

Clearly having a monopoly in a field does not assure an equally dominant product from that field.

Donnie Hale

Clearly having a monopoly in a field does not assure an equally dominant product from that field.

I don't think there's a monopoly. I've sat all week in a classroom where Bibleworks was used the whole week. And one of the podcasts I regularly watch uses Accordance for nearly all of its biblical language work. What those both have in common is that they are better (mostly: faster) than Logos at those features.

Donnie

Rene Atchley

I find it incredibly frustrating to post on this site. Every generalization seems to be opposed by someone somehow by injecting that "wow" that statement isn't true because it isn't true for me. The point of the thread and post is a critique of the quality control issue for a product from a company that sells a very expensive bible library product. So yes they may not be a monopoly but that doesn't detract from the ogopolistic nature of the bible software industry on PC/Mac that often (but not all the time under all situations in all time periods) quality controlled problems (but not for everyone all the time under all situations in all time periods of history known to humanity).

Todd Phillips

As it is, I could use some of the tagging "to save hours" (this is a key marketing promise). This can be done by automating what would otherwise need to be done manually. Combing large sections for data over and over is extremely time-consuming. BUT unfortunately, at this point, I cannot rely on results. So, I still have to do the manual work to ascertain what I find. And this is not just compensating for the occasional human error. This is more systemic.

I would find it difficult to rely on results from Faithlife-created datasets and tagging for the same reason. With much of Faithlife's future developmental effort going into creating theses new datasets, I hope they aren't shooting themselves in the foot by creating a reputation of delivering unreliable and missing data.

Is Logos so broke it must release stuff asap or die and thus considers it better overall to release incomplete or error-filled, bug-ridden products?

I do understand the need to expand and broaden when you see that you may be about to saturate your market. They may not be broke, but they do need to keep finding ways to generate income. (But it probably needs to be done without alienating your customers.)

DMB

Having not worked at Logos, one can only surmise. But it sure looks like 'deja vu'. The boss and team put together the next dreams. Everybody gets to work, with varying degrees of success. The boss notices a competitor is releasing the next major version. Asks the teams 'can we do it??'. Of course the teams can (though they really can't). And didn't.

The sad part is the misplaced dreams. I was really hoping the Logos databases, the Proclaim platform, the Faithlife community product, the mobiles, and oh yes, the dating game ... the ecosystem (as Logos calls it) ... had great promise. But each piece is not well designed and always incomplete.

Yesterday and this morning I felt really sorry for the Reverend that just wanted to delete his faithlife group, after giving up. No one seemed to know how. No manuals. I would have like to have helped. The Logos guy answered they'd try to do better ... but no answer for the Reverend.

The Logos6 release I hope won't be repeated in Logos7; too much product that was not complete at sell-time. Pressure now to produce it. Requests for quality? I haven't even bothered to see what I bought (since I can't find it!).

Lee

If the situation described by the OP is true, then it is sad indeed.

The quality of the tagging, linguistic analysis etc. can be evaluated objectively by folks trained in those areas. If there are major errors, there's nowhere to hide.

Francis

I hope they aren't shooting themselves in the foot by creating a reputation of delivering unreliable and missing data.

Is not this already happening? There is also a reputation of greed floating around.

without alienating your customers

Here is a sad fact of my experience. I have used Logos since version 2. Over the years, I have "sold" Logos right, left and center. But in the last while, this has changed. I use Logos because I must, not because I want to. But I am no longer an enthusiastic advocate of Logos. There is a language version coming out on which I did translation. Considering what I know of it, I have not and will not recommend it to anyone. In fact, this was part of what I alluded to earlier: I was at first proud of the idea of helping it be available to more people (and bringing better Bible study to them) but now I would be ashamed to be associated with that version.

I am not saying this to bash Logos needlessly. I feel that this is a bad turn but it can be reversed. I sure hope so.

But I do wonder, in the present estate and in light of all that has been going on, what user polls would indicate if the simple question was asked where only yes or no was allowed as answer "Do you [still] trust Logos?" I hope Logos is listening and more importantly, hearing.

Matthew C Jones

what user polls would indicate if the simple question was asked where only yes or no was allowed as answer "Do you [still] trust Logos?"

YES!

Francis

The question was not what individual answers might be but what the overall picture would look like. If the individual answer is yes for you, I'm glad for you. We may not have the same needs, experiences or simply responses.

Michael McLane

I do understand the need to expand and broaden when you see that you may be about to saturate your market.

I have gotten the impression that they are a company that often reacts and operates out of fear. I am probably wrong.

Looking at my context, there is absolutely no saturation. Out of six pastors in my local group, I am the only one who uses Logos. Another chose Accordance the other do not use anything. And of the community I serve, no one uses bible software. There is plenty of room for growth in my context.

However, as I have said before, I cannot recommend Logos or any FaithLife products because of the quality issues. I would love to be able to "sell" Logos to the people I serve. But, I can see it coming back to embarrass me and having to spend too much time supporting them. (Not to mention that it is not user friendly and has no reference manual. I have been endeavoring to understand the new Milestone search feature. Thank goodness for the community here for helping. But surely there has to be a way to make that into a menu item that is more accessible to someone without mastering search syntax strings. Just my experience.)

Do I trust Logos? As a reader, yes. But, I have gotten into the habit of verifying its result otherwise, or doing them the hard way because it is just as easy. So, maybe the answer is, no. I buy a lot of resources, but I have been contemplating their value since the release of 6.

Like Francis, I want Logos to succeed and grow.

EDIT: On further reflection, I would like to make an amendment/confession. I think I was quick to answer due to many of my resent frustrations with the software, particularly since the 6 release. I think it was unfair of me to insinuate that I do not trust Logos (or to outright say it). I do not know enough about its search perimeters, abilities, etc. on the datasets to even begin to form an opinion, which is what the OP was referring to, I think. My concern for QC centers more around the reliability of the program itself to run, to do its job timely, the UI etc., which I hope came out a bit.

Thank you John Fidel (below) for your perspective and my apologies to the community. I am trying to be more positive towards FaithLife in the forums, yet maintain the truth of what I see that they may improve. I particularly want to be sensitive to those that do the hard work at FaithLife in producing the software and resources. I know they want to do a great job and it is a complex, never-ending job. My apologies to you as well if I have seemed overly harsh. As a pastor, most days we hear more criticisms than anything. I forget that others deal with that as well. Thanks also to Sean for his words.

So, in terms of the quality of the "programmed" application and services, I think there is much room for improvement (hence my difficulty to recommend it. Though maybe I will give it a shot). I know it will get better. Though, I think I would have rather waited for a better product.

I do love Logos. I am in it many hours a day. Even after all these years it is mostly for reading. I think one of the frustrating things is the steep learning curve to really get at the power.

Michael

Dave Hooton

I think that quality control is the leading issue with the ongoing development of Logos.

That has been a concern of mine for some years and particularly with software for Logos 5.1+, Logos 6.0 Beta and 6.0a, 6.0b betas. Complex datasets have always been a concern starting (for me) with OpenText on Logos 3, then Clause Search, BSL, People/Place/Thing tagging, and Referent tagging in Logos 5 followed by the new datasets in Logos 6. OpenText was so buggy it probably drove Logos down the path of doing its own datasets and being able to manage both development and maintenance (parallel to managing their own bibles). LCV (topics) in Logos 4 was the signal for Faithlife wanting to manage data for a better experience in searching. Now we see LCO (Cultural Concepts), Literary Typing, Semantic Roles and Case Frames which are quite complex in the way they are applied to resource text.

The question of trust arises in the design and intended application of datasets. Whilst quality control (QC) is important in their management, most errors are due to human error/interpretation in applying the (bible-based) data. Do I trust Clause Search -- No! Do I use it - Yes. Would I recommend it - No.

The QC of the software and the direction of development showed a marked down turn in latter part of Logos 5. The Logos 6.0 beta showed that Faithlife could not keep up with the bug reports, or wanted developers to interpret what was important to them. The lack of response in the forums and the silly errors in the 6.0a/6.0b betas is indicative.

MJ. Smith

LCV (topics) in Logos 4 was the signal for Faithlife wanting to manage data for a better experience in searching. Now we see LCO (Cultural Concepts), Literary Typing, Semantic Roles and Case Frames which are quite complex in the way they are applied to resource text.

The question of trust arises in the design and intended application of datasets. Whilst quality control (QC) is important in their management, most errors are due to human error/interpretation in applying the (bible-based) data. Do I trust Clause Search -- No! Do I use it - Yes. Would I recommend it - No.

The QC of the software and the direction of development showed a marked down turn in latter part of Logos 5. The Logos 6.0 beta showed that Faithlife could not keep up with the bug reports, or wanted developers to interpret what was important to them. The lack of response in the forums and the silly errors in the 6.0a/6.0b betas is indicative.

There's another piece that needs to be considered in your history --> the move towards giving as set data the results of what had been simply user techniques prior to computers. Many of these datasets, even morphology, hide the ambiguity that was why a scholar used the technique. Add to this the Faithlife attempting to be all things to all people - when we don't agree on a base text, don't agree on the "literal" meaning, don't use the same terminology -- I'm not sure there's a gold standard to measure the Logos data against. Which means Faithlife must be very explicit about what their coding means and how ambiguous cases are resolved.

Edit: I see Fr. Devin Rosa made my point better than I while I was writing this.

Francis

This hardly addresses all of the concerns that have been mentioned, but perhaps we need something like the Bible Sense Lexicon to help us find how something may have been tagged (ideally, it would it not make a lot of sense if BSL and tags were consistent or shared the very same base?). I did a quick check to see whether it could be used this way (using the feast search in John as a test case). I searched "feast" and got "feast (meal)" proposed to me (with Passover as sub-category). I right-clicked on it to see if I could search it and I got this:

Apparently, there is only 1 feast-meal (not to mention, Passover) mentioned in John!

Dave Hooton

There's another piece that needs to be considered in your history --> the move towards giving as set data the results of what had been simply user techniques prior to computers.

Yes, I had to give a potted history in order to meet an important (family) deadline! The debacle of OpenText in Logos 3 illustrated how a potentially good design (by OpenText.org) can be ruined by a lack of attention to detail (the morphology was improved in Logos 4 but I can't fully trust the method). The ambiguities of Case Frames e.g. 'love' as an Agent, leads to a lack of trust, but I don't believe that the method could help me to "better understand the verb in its context"! It's far too mechanical.

The ultimate frustration is that Faithlife have been doing resource metadata for over a decade and it still isn't consistent and right! I don't look forward to this as an example for correcting the newer datasets!

MJ. Smith

The ambiguities of Case Frames e.g. 'love' as an Agent, leads to a lack of trust, but I don't believe that the method could help me to "better understand the verb in its context"! It's far too mechanical.

Actually, I've been using case frames for close to a decade and find it really helps me in understanding translation. But in the sense that it fits nicely into computational linguistics your complaint is not unfounded.

Matthew C Jones

Francis, Do you like Logos (the software program?)

DMB

Super, you're sounding like Kanbe (that's a complement). It's the NHK weekly drama that just finished up. Kanbe was the strategist that essentially put Hideyoshi in power. So, you have a 'Kanbe question'. The query-ee bows deeply and answers 'absolutely', with Kanbe quietly smiling. They both know the real truth.

MJ. Smith

Francis obviously likes it well enough to care about its weaknesses. I don't think he wants any of the weak point to eventual break.

Matthew C Jones

Francis obviously likes it well enough to care about its weaknesses. I don't think he wants any of the weak point to eventual break.

He sure seems unhappy....Kinda like a bad marriage.

Matthew C Jones

Super, you're sounding like Kanbe (that's a complement).

Thanks.

BKMitchell

Francis, Do you like Logos (the software program?)

I am hoping that Francis will answer you, soon.However, I believe the answer is yes he has liked Logos(and still does), but since he started working on a Phd he has begun to see Logos in a different light. Not, only have his scholarly needs grown and changed, but those in his field apparently aren't using Logos. And, I think Francis would like Logos to be taken more seriously in the academic world he now lives in. For me his post look a lot more like tough love than hate.

JAL

For me his post look a lot more like tough love than hate.

What BKMitchell said - and of course what Denise, MJ, and Dave have said.

Francis

SuperTramp, I don't know why you persist in trying to question individuals at the personal level. This happened already among several users earlier in this thread and it should be clear by now that it is not constructive.

What I am interested in is seeing progress in the discussion but even progress there will be useless, unless it translates in tangible progress with regard to the issues.

I appreciate whatever I read of Sean Boisen's responses, the fact that he responds, the tone of his responses and the information he provides. Thank you, Sean. Overall though, I have yet to see a response from Logos that is commensurate to the issues this thread raises. "Good discussion. We appreciate the input. We will be working on some stuff" is not sufficient to communicate that the message has really gone through and reassure that there will be change. I reiterate what I said before, my concern is not with "lo, here is bug A" response: "thanks, we'll fix bug A in RC 5". My concern is with systemic issues and the overall situation. Personally, I would like to hear indications of change. Sean made an allusion to business decision earlier, this may actually be more the heart of the matter...

Finally, I am not sure that all users realize that Bible Software has become a professional tool now (not only Logos). It's not just for personal enjoyment, Bible study and preparing Sunday school classes (to charicature a bit). When you use something for work, you need it not to waste your time going round in circles. You need to do what it claims reliably and to have documentation that matches its claims. It's not a want, it's a must. Some issues sometimes is normal; many issues all the time is not.