Quality control in Logos 6 - a new perspective

I have cut and pasted selections from the thread to show a certain pattern. Of course, this is selective, otherwise it would just reproduce the whole deal. Statements of satisfaction are not included -- not to deny them -- but because this is not the point I am trying to make. The point I am trying to make is simply this: look at the many users here (incl. a good number of long-time, high-proficiency users) who are saying 1) we noticed a sustained downward trend as far as quality (not innovation!) is concerned 2) that is a real cause of concern and 3) this is systemic. Many more concerns of this kind are found in many threads and have been for quite a while. 

Is Logos so broke it must release stuff asap or die and thus considers it better overall to release incomplete or error-filled, bug-ridden products?

Luke 14:28-30 also comes to mind.

I also wish Logos would make quality control a higher priority.

I would find it difficult to rely on results from Faithlife-created datasets and tagging for the same reason.  With much of Faithlife's future developmental effort going into creating theses new datasets, I hope they aren't shooting themselves in the foot by creating a reputation of delivering unreliable and missing data.

The sad part is the misplaced dreams.  I was really hoping the Logos databases, the Proclaim platform, the Faithlife community product, the mobiles, and oh yes, the dating game ... the ecosystem (as Logos calls it) ... had great promise.  But each piece is not well designed and always incomplete.

That has been a concern of mine for some years and particularly with software for Logos 5.1+, Logos 6.0 Beta and 6.0a, 6.0b betas. Complex datasets have always been a concern starting (for me) with OpenText on Logos 3, then Clause Search, BSL, People/Place/Thing tagging, and Referent tagging in Logos 5 followed by the new datasets in Logos 6. OpenText was so buggy it probably drove Logos down the path of doing its own datasets and being able to manage both development and maintenance (parallel to managing their own bibles). LCV (topics) in Logos 4 was the signal for Faithlife wanting to manage data for a better experience in searching. Now we see LCO (Cultural Concepts), Literary Typing, Semantic Roles and Case Frames which are quite complex in the way they are applied to resource text.

The question of trust arises in the design and intended application of datasets. Whilst quality control (QC) is important in their management, most errors are due to human error/interpretation in applying the (bible-based) data. Do I trust Clause Search -- No! Do I use it - Yes. Would I recommend it - No.

The QC of the software and the direction of development showed a marked down turn in latter part of Logos 5. The Logos 6.0 beta showed that Faithlife could not keep up with the bug reports, or wanted developers to interpret what was important to them. The lack of response in the forums and the silly errors in the 6.0a/6.0b betas is indicative.

Some users will keep quiet after a while; some users will make a personal spending decision. Expecting regular users to take up the slack is just too much.

Francis, I agree that the QA is inadequate. The number of resources that have been released then sent back for basic coding is not acceptable. Examples" the BCP without the Psalm links (Latin names rather than Psalm number), Faith of the Fathers without the link to the index that provides the primary value, the number of liturgy books not tagged to appear in the liturgy section ... those are just ones I complained about. I also agree that the Faithlife fall back on an overall error rate and their lack of time to manually check each resource is a poor excuse. At the start of each resource's development a check list of required flags for processing, indexing noting special features etc. etc. ought to be made and ought to follow the resource and manage the workflow. But I do think we have to have realistic expectations and recognize our own position in the process.

The ultimate frustration is that Faithlife have been doing resource metadata for over a decade and it still isn't consistent and right! I don't look forward to this as an example for correcting the newer datasets!

Also there are two levels at which these issue can be addressed. On a case by case bases - and yes logging errors is a good approach to this. But it takes effort away from what we as customers should really be doing - using and enjoying the product - and being empowered better to be about service.

The other level is drawing attention to a "general" need for Faithlife to sharpen and refocus their activities if they wish the retain and promote an enthusiastic user base.

What's "funny" in a VERY sad way is: The folks who are not complaining. When L6 was intro'd I upgraded to a very large package. I test drove it for a month and could not justify keeping it. Simply too buggy. I knew if I complained , I would just be chastised by the fanboys. So I quietly returned it. I suspect there are many others who did the same. Faithlife will see the trend in their sales/return numbers. I just hope it's not too late.

I just want Logos 6 to do what it says on the tin. I just want my Logos program to work as it says it will. that's not seeking perfection – just adequate functionality.

I think it would be in everyone's interests to have a stable, reliable engine that just works. Then all those existing resources, datasets and functions would work without falling over, freezing, etc.

I need Logos to work every time. As it is I spend a considerable amount of time every week when I should be working on the Bible text for Bible Study and Sermon Preparation, just trying to get things to work!

Critical bugs are easy because anybody can detect them and the 'crash' has to be fixed. But QA has to ensure there is a program to detect less severe bugs and have them fixed before the software is released. This is usually called the Alpha process, but the 'Beta' process is commonly used in its place these days and I don't think it is acceptable for users to pick up silly errors or find that features are unusable when they are not mentioned in the Release Notes  i.e. when they are unrelated (as far as the user knows) to the bugs being fixed or features being implemented.

Testing is the art of finding the bugs that are inherent in software and it is accepted that there will be latent/hidden bugs even after the most rigorous of testing programs. But I find it unacceptable that known bugs are ignored, new bugs are not acknowledged and that developer preferences override user/customer requests!

So you can bag Windows for its 'security' issues but MS doesn't compromise essential usability to the extent that I (and others) perceive in Logos, and I won't mention what Apple does to its customers... (also remember that I'm not talking about platform preferences).

I could say that Faithlife have lost the perspective of their customers' needs since Logos 5 and you only have to read the forums on indexing issues to get some insight on that matter. 

Is all this being critical or negative? That can be the perception, but if it is based on facts then it should be regarded more objectively. At least that has been my aim and that of the OP. I trust Faithlife to listen and I trust Faithlife to be using their product, but I can overcome most of the technical issues (crashes!) that come my way whilst others simply cannot.

And usability also comes down to documentation; particularly for the new Logos 6 tagging and search features.

they are willing to deliver incomplete and unreliable product

That said people who buy 6 should not be used virtually as beta testers. To want to jump into a new feature like the atlas and find that is not complete feels amateurish (LOGOS/VERBUM 6 were/are in ways incomplete, it was made known that this was the case, but I ask WHY release it before being done, you must know there will be complaints). A springtime launch would not have been the end of the earth. I do feel the "Quality Control" was less than stellar for 6.... I would hope in the future Faithlife would wait till the product is more stable and datasets/features more complete.

Then when Bob asked right after the release of Logos 6 what we wanted in Logos 7, he said: "Wait -- I already know the answer! "Fix annoying-thing in Logos 6 first!" :-)"And I know that he thought that that was funny, but I didn't. And don't get me wrong. There is nothing wrong with "blue skying" and planning for the future; in fact it is needed. BUT there is a time and place for that. And what is needed first is to get the quality of the Logos software back where it used to be.  You can still be innovative, but at a high standard of quality.

I am heavily invested in Logos and so there is basically "no turning back" for me and I realize this. But I don't recommend Logos to others now. It is too complicated for the new person and there is not good documentation for the average user. Yes, I know that there are plenty of places to track down the information to learn about the software, such as the forum, but the average or new user will not do that.

I AM concerned with what has been happening the last few years.

gradually through the years I have seen the quality of what was being released become less of what it used to be. It is not that they were being "less innovative." It is just that the quality was not up to the same standard.

Strongly agree with Charlene. Very buggy.

Please don't release new software every 2 years!

Please!

Motto: "Perform as advertised."

I can't imagine why anyone would defend Logos 6.  I got it for free and I would most definitely have requested a refund had I paid for it.  It has been a train wreck from the beginning.  It seems that every update decreases performance.  

Logos 4 was slow and buggy (Mac).  Logos 5 came out and we received more features, greater stability, and an increase in performance.  That is very difficult to do but Logos did it.  Here is my question to Bob and Faithlife.  What strategy, mindset, philosophy, prioritization changed that resulted in Logos 6.  It seems like Logos sacrificed performance and stability on the altar of increased features and error prone datasets.  If this is the new philosophy and future direction for Faithlife I would like to know it.

The presumption behind this comment seems to be that something systematic has changed: I don't think that's the case. In the 8 years I've been with Faithlife, we have always focused on creating new features for our users. We've always had more ideas that we've had resources to implement, so we've had to make hard decisions about how to prioritize them. New resources, features and datasets are how we keep our business going and continue to serve those who have invested in our product.

That mindset presumes the data we've already shipped is useful. If you feel that our datasets are "bug-ridden", "error-prone", or "not usable", I'd honestly like to know which ones and why, so we can investigate. I ask for examples because that helps us know what to investigate: generalities aren't actionable. If we can find straightforward errors of analysis,

When it comes to performance and stability problems, I'd echo Sean's comment:

I ask for examples because that helps us know what to investigate: generalities aren't actionable.

I think that what many users would like to hear (and actually see) is something like this: "Yes, we acknowledge that unfortunately many users have had a frustrating experience. We are starting a process to get to the bottom of how we have gotten there and what we need to do differently so that this cycle does not continue". 

Not: "if you want to have a better experience, the onus is on you to do the required quality control work in our place, being guinea pigs and spending your time reporting bugs, sending logs, contacting tech support, and figuring things out on the forum, wiki or on your own. We think that it is perfectly normal that so many of our users spend hours doing this week in and week out in order to be able to have normal use of our software".

In other words, Faithlife, you need to acknowledge what has been happening, do your own homework and stop turning tables on us.