Quality control in Logos 6 - a new perspective
Comments
-
Francis gets home. Boy, oh, boy, oh, boy! MJ seemed really excited about that one, must be a really cool and useful feature! So, let's find out how to use it. Open the help file, type "milestone". Alright, let's see, several sections: milestones, personal books syntax, how-to, visual filters and personal books, milestones search here it is!!!
Let's see: definition, good. Search syntax, helpful. Wow! These examples are fascinating! I could really use this! I really love that link to the datatype sections. Oh man! So many datatypes. Good thing that there is index to tell me what to use depending on which edition of Josephus I search, otherwise I would never remember. Look at that! I can even use the context-menu and reference box to help me. Cool! Let's do it. It works!!! Yay! Email to MJ: Hi MJ, I tried the new feature you told me about, it's awesome!
---------------------------------------------
But the problem clearly is documentation here, which should be an integral part of releasing a product.
YES! This. [Y]
0 -
(Thursday morning @ the Faithlife café)
Francis: Hi, MJ, how are you doing?
MJ: Great! Just grabbing a quick coffee. Say, have you heard of the new cool feature in Logos? Now we can do milestones searches!
. . .
Advanced does not have to be arcane. But the problem clearly is documentation here, which should be an integral part of releasing a product.
It's not that I cannot figure out how to use it. It's that I don't have the time to do sleuth work of the kind described above nor should I or anyone have to if documentation was done right. This is in essence the problem: users should be given clear instructions and reference aids to use the tools, not have to figure things out by themselves and with such convoluted processes.
Love your presentation ... and agree with your conclusion re: documentation of use. I think our differences are in expectations of Faithlife's responsibility in teaching us to utilize the data their software supplies. But it is definitely their responsibility to provide documentation on how to get the data out and what the data is. Your case of milestones is a bit of egg on their face in that the search doesn't even have an example ... their usual method of showing us the syntax.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
It's not that the feature is so advanced that people can't use it. I am a postgraduate researcher in biblical studies and you tell me that not using that feature is fine! That's not the right answer or approach. Advanced does not have to be arcane. But the problem clearly is documentation here, which should be an integral part of releasing a product.
It's not that I cannot figure out how to use it. It's that I don't have the time to do sleuth work of the kind described above nor should I or anyone have to if documentation was done right. This is in essence the problem: users should be given clear instructions and reference aids to use the tools, not have to figure things out by themselves and with such convoluted processes.
I get the feeling that this is not what Logos wants to hear.
0 -
Hopefully it also answers what you wrote Bradley. It's not that the feature is so advanced that people can't use it. I am a postgraduate researcher in biblical studies and you tell me that not using that feature is fine! That's not the right answer or approach. Advanced does not have to be arcane. But the problem clearly is documentation here, which should be an integral part of releasing a product.
It's not that I cannot figure out how to use it. It's that I don't have the time to do sleuth work of the kind described above nor should I or anyone have to if documentation was done right. This is in essence the problem: users should be given clear instructions and reference aids to use the tools, not have to figure things out by themselves and with such convoluted processes.
The solution to the problem is not as complicated as you make it sound (at least in this case).
Spot on, Francis.
0 -
[Y] Good job, Francis!
Charlene
0 -
-
Thank you, Francis, for an articulate presentation of what has been my experience.
0 -
3. A specific problem with the Bible Sense Lexicon. I reported this problem to data@logos.com on the 1st of July, 2013. I have never received a response, and the problem persists:
[quote]
In Genesis 49:24, 2 Samuel 23:3 and Isaiah 30:29, the word “stone” or “rock” in the phrase “Rock of Israel” or “Stone of Israel” is currently classified under the sense “stone”. It should be classified as “God ⇔ rock”.
My first report could be excused because you were busy releasing Logos and Verbum 6, but given that you have had 18 months to fix this report, and it is easy to fix, it might indicate a serious problem with the way these emails are handled. The sense “God ⇔ rock” does already exist, so I am pretty sure it's not just that you aren't yet doing this type of classification.
Fr Devin:
These changes have now been made in the system and should be in a future update.
0 -
The forums are the best place to report bugs.
OK - it's best to report bugs in the forums.
What is best for other problems like the following example?
Fr Devin Roza:3. A specific problem with the Bible Sense Lexicon. I reported this problem to data@logos.com on the 1st of July, 2013. I have never received a response, and the problem persists:
In Genesis 49:24, 2 Samuel 23:3 and Isaiah 30:29, the word “stone” or “rock” in the phrase “Rock of Israel” or “Stone of Israel” is currently classified under the sense “stone”. It should be classified as “God ⇔ rock”.
...
These changes have now been made in the system and should be in a future update.
Thanks Fr. Devin for your concrete, actionable post.
[Y]Thanks everybody![&]
"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963
0 -
Fr. Devin, let me add my thanks to those of others for your clear and helpful suggestions.
2. The lack of response from data@logos.com. The second is related, and that is the type of response that is given from data@logos.com. That is, none. I think that an email should be sent thanking people for sending in corrections. I'm still not even sure if you ever received this email.
I'll see if we can add an auto-response to this account so at least you'll have confirmation that your email was received (and an expression of our gratitude for taking the time, which is sincere).
We do read all these emails. We triage them to add them into our bug tracking system, prioritize them, and assign them to the right personnel for resolution. We're currently discussing some proposals for how to add something like release notes for datasets (which actually get updated independently from the application itself), with the intent to document major upgrades, changes, bug fixes, and known issues there, just like we do with the app.
Along with Todd's affirmation about posting bugs to the forums, you're also welcome to post data errors (and suggestions, questions, and general criticisms) to the forums. That has several advantages:
- not everything that looks like an error actually is: on the forums, another knowledgable user may be able to correct or redirect
- the forums have a persistent record of issues raised and addressed that others can find
4. Morph tagging on the Biblical Dead Sea Scrolls. As I have said before, I am quite happy with the morph tagging of the Greek and Hebrew Bible, and have never had trouble with the Septuagint tagging either. I have not been as happy with the tagging of the Biblical Dead Sea Scrolls, although I can't say it's bad either, just not quite up to par to what I would hope for. From October 2012 to January 2013 I worked quite a bit with these texts at the Hebrew University, and remember running across a surprising number of errors (maybe one or two per chapter? I'm not sure). At the time, I sent in a bunch of corrections through the "Report a Typo" mechanism. If you have those Typo reports still registered somewhere or another, I would recommend trying to pull them up and look through them to see if they are accurate, and then based on what you find there get a feel for how few or how many errors there may be by extrapolation and how worth it or not a revision would be.
I'll make sure we take a look at this.
5. Recommendations for Documentation. I am very happy to see the commitment to document tools that are of use for academics. I hope that by the time Logos 7 comes out, all the tools of most interest to academics have been documented. I think that it is important that there be a place to go to that is specifically for academics. I know that Logos has a huge and broad market, but as this thread has made clear, there are certain requirements of academics that are specific.
Here are the tools that I think should be highest priority...
As I said in a previous post, I don't think documenting these tools needs to be something out of this world, or even take that long. The basic question is what does an academic need to know to use these tools, who did them, including even interns, criteria used, and what their limitations are, what is missing in the tagging, etc.
BTW, in my opinion Rick Brannon is an example of someone who seems to understand and do these things already. He also has a keen sense of the importance of releasing complete and accurate data. Thanks, Rick! I'm sure there are many others as well, I have just particularly noticed that about him, so, thanks!
Thanks for these suggestions: we've already started planning for this documentation, and we'll take your priorities into account. (and I agree with your assessments of Rick [:)])
7. Communication regarding incomplete datasets - In an earlier post, Sean earlier mentioned the Factbook Septuagint Deuterocanonicals dataset. For me, that dataset is actually an example of decent communication. It was always advertised as being only the Deuterocanonicals. Perfect. What is problematic is when datasets are published and it isn't clear what is missing. Think the Journals tool, Outlines Tool, Bible Book Factbook Entries, etc.
Don't get me wrong - I fully agree you need to advertise, be positive, etc. I don't think these limitations need to be bragged about. But at the same time the information needs to be available.
For tools like the Journal tool, the Outlines Tool, the Bible Book Factbook Entries, etc. that are currently being worked on, ideally I think there would be either a resource in the library that listed what works were included, or maybe a webpage with that info, or maybe both. When the datasets are updated to add more works, the list of works included is also updated.
Maybe as well on the Guides and Factbook sections there could be some sort of standard Info bubble, or help bubble that is available when documentation is available, that would bring you to the academic documentation / list of works included up to now, etc. Some tools would always have this info button (such as the Ancient Literature tool), others only when it applies (such as the Factbook entry for a Bible book).
Thanks everybody!
We plan to improve this in our dataset documentation and release notes, and we're actively discussing how to implement this.
0 -
I hope that this "dramatized" description gets something across.
Thanks for making this point.
0 -
Thanks for making this point.
Thanks, Bradley. It's good to read.
I think that this thread has been productive in articulating issues and engaging the conversation one level up. It's been great to see how many people have been involved including the several contributors/responders from Faithlife. I think we can do even better yet!
The reason I have resisted the "actionable" report route because of what my goal and plea has been all along: not fixes here and there, but change. The former would not be sufficient progress as compared to what the situation was already, except to the extent that some issues have been sufficiently pressed that they have led to specific initiatives to address them. This is not a negligible gain!
Change, on the other hand, starts with the recognition that the problem does not merely lie with users mistakes (without denying them) or with the inherent and unavoidable complexities of the advanced features (without denying them either) or inability to predict how users would use or respond to new features (without denying the challenges therewith). It starts with recognizing that, to use a few specific examples that have been mentioned in the thread (1) a journal section delivered empty; (2) a milestone search delivered without documentation; (3) a user document search delivered practically unusable (to cite just a few) cannot be explained away as the vicissitudes of realistic production of innovative Bible software. This was sloppiness (or cutting corners if you prefer), nothing less and this, regardless of how many individual issues are reported and subsequently fixed, is where is the greatest longing and plea for change is. As grateful as I am (and I am) for all the responses and reports of steps taken from your team, I can't help but feel that there has been some resistance to admitting this.
But the point is not to "win", nor is it apologies, nor is it neat-picking. The point is making sure that Logos 6 improves, allowing its great features not to be tarnished by these blotches, to improve significantly user experience, confidence, and ability to take advantage of the full potential of the software and to pave the way for much better expectations, clearer skies and smoother sailing when Logos 7 comes out. To put it another way. It's great to report problems with Logos 6 and get them fixed, but we don't want to have to do this all over again and to the same extent with Logos 7.
0 -
Francis, I applaud your direction. But chatting with mid-level managers, and customer contact folks in most companies will go nowhere; what are they going to do? They can only fix what is presented (although I'm a bit amazed about the DSS reply). Actually most quality initiatives come from the inside ... people know very well what works and what doesn't. And they almost always come from changes in leadership. Not being critical; but 'change' is rare.
Now in defense of Logos' inability to see customer issues, I use my iPod and iPad. The wishlist almost never goes and gets the app I want to buy (so I have to re-search for it). The search field for apps is always a single app that I bought maybe 6 months ago. It never changes. And iOS8 is 'still' crashing wildly (I'd say maybe every 3 days). I've no doubt if I were on an Apple forum with an Apple person, I would wonder how they can miss the obvious? Where's the quality? They'd say ... we'll need specific details. Apple?
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
See http://scripturestudysoftware.com/2015/01/07/verbum-visit/ for an example of Verbum/Logos gathering input from users ... okay most of us don't have a week's worth of useful information but still ...
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
But chatting with mid-level managers, and customer contact folks in most companies will go nowhere; what are they going to do? They can only fix what is presented (although I'm a bit amazed about the DSS reply).
On the one hand, I do not take for granted at all where we will be at when the dust settles. On the other hand, this realism is mixed -- for my part -- with greater optimism than your reply reflects, Denise. I have considered the fact that our friends who reply here may be powerless to effect further-reaching changes to policy beyond the "actionable" items they keep asking for. But I know Bob can read and that he does on these forums.
My previous post could have been entitled There and Back Again: A Journey Toward Better Quality Control. I feel that having made so much progress in the right direction, it would be a shame to stop short of reaching the destination.
0 -
I just came across this thread and I wanted to tell you Francis that I admire your persistence and the time and effort you put into this. You don't give up and it is admirable! It doesn't matter if your efforts are successful. We are forever in your debt. Thank you!
0 -
I posted this on another two threads regarding the slowness of L6 and my initial experiences, but I think it fits best here with some additions. At first, L6 seemed to run okay on my MB Air. But after using it more and leaving it open as I normally would, performance seems to have dropped notably. I can confirm that just idling with one Bible pane and one commentary open, L6 occupies 780 MB of Ram. That is nearly 25%. Compared to roughly 100 MB of another Bible study application.
1. Scrolling in just an English Bible that is NOT linked to any other pane, skips and hangs. Jumping to a new location, say from Genesis 1 to Revelation 1, begins and then freezes upon arrival for 5 seconds (I timed it). This happens in a layout with other panes open, a few of which ARE linked.
2. Scrolling hangs for 4-5 seconds, and then skips ahead imprecisely when you stop.
3. Closing a tab or adding one, creates a 4-5 second delay, distorts the tabs of the panes visually and then lags to repaint them.
4. I am encountering other problems in Bible Word Study and Factbook with scrolling, and erratic dropdown menus appearing in the entry box of FactBook and Inline search when my cursor is nowhere near it. Some things appear to be bugs which I will begin to note and report.
5. Hebrew resources (EDIT: Bibles and devotionals in Hebrew) are just about unusable at this point, but I know a fix is on the way for that. I hope it comes soon.
6. Library scrolls very well and has no hesitation. In the "Details" view, scrolling causes the book icons to do a slight jiggle. Aesthetically annoying but functional and not a show-stopper.
7. Searches run quite well in English, Greek and even Hebrew bibles. The problem with Hebrew texts does not seem to affect the searches nor scrolling in the search result pane. Only scrolling in a Hebrew bible pane. Inline searches in a Hebrew bible pane run fine, but scrolling within that pane afterwards is painful.
I've had no crashes yet, which is positive. But I was truly hopeful that the upgrade would not be this trouble ridden. I am not intending to be sarcastic here but ask sincerely, how does a release that is this important to a company, go from beta to full release with so many performance troubles?
I ask that question with genuine ignorance as to how so many problems can be missed during testing (especially the lag, Hebrew text problems and others that have been listed by others and myself).
You see, unfortunately, this reminds me of the very problem riddled days of L4 on Mac, which was a complete disaster when it first came out for me. Crash upon crash (a little lag is tolerable but not lost work). And I helped beta test and report countless problems before hand.
Logos customers (in particular, Mac users) are either very forgiving of things like this or so heavily invested that this type of consumer experience can happen and yet we come back for more. Probably hoping that, this time...it will be different. That was my hope.
I appreciate Logos and until now have used L5 regularly, but unfortunately with the current performance of L6 I'm not sure how much use I'll be able to get out of it on my Mac until things are improved. I have a Windows 8 machine and may need to default to that for Logos use.
Each time I get burnt with Logos for Mac, my trust in the reliability and integrity of a new releases/upgrades diminishes. And rightfully so. After all, as a Mac user, I have yet to have a release that performed up to its promise right from the start. Yes, L5 got a lot of things right and problems eventually get worked out. But when a product is sold, the product should be ready.
I am thankful that Logos is not my only option for digital Bible study (a competitor's program is my go to workhorse that never fails on Mac) otherwise I'd be having a very hard time doing the job for which Bible Software is made and for which we customers invest our hard-earned money.
Mark Nigro
Pastor/Teacher/Student/Writer
www.calvarychapelbiblecollege.com
0 -
But when a product is sold, the product should be ready.
While your at it, send a copy of this post to the car industry and Microsoft and Dell computer and and and...
0 -
I ask that question with genuine ignorance as to how so many problems can be missed during testing (especially the lag, Hebrew text problems and others that have been listed by others and myself).
Speaking for myself, I can say that the problems you have experienced survived testing because I never experienced them—early 2013 27" iMac—nor did I see any such problems reported by other Mac users during Beta testing.
Having said that, I do believe that the Beta cycle was rushed because the release date was determined prior to the commencement of testing.
UPDATE:
5. Hebrew resources (EDIT: Bibles and devotionals in Hebrew) are just about unusable at this point, but I know a fix is on the way for that. I hope it comes soon
After reading your post, I tried scrolling in an unlinked Hebrew Bible—which I did not do during Beta testing. Scrolling was slightly jerky, and the text did sometimes jump ahead when scrolling ceased. It was not as bad as you reported, but my machine is more powerful than an MBA, and I do have a Fusion drive. Hopefully, 6.1 will fix this.
0 -
I am very happy to see the commitment to document tools that are of use for academics. I hope that by the time Logos 7 comes out, all the tools of most interest to academics have been documented.
we've already started planning for this documentation
I hope that this "dramatized" description gets something across.
Thanks for making this point.
Thank you Sean and Bradley for indicating that documentation is in the works. Many (myself included) will be looking forward to it.
I would have to take exception at the above quoted statement from you, Devin, in that I don't think that it should even be contemplated that documentation for features that have been sold as part of the Get Logos 6 should take anywhere near as long to be made available as the time it might take for Logos 7 to come out. It is a big part of what was delivered "incomplete" in Logos 6 and the tacit agreement that is made to customers to deliver what is advertised needs to be made good ASAP.
To illustrate the problem, I now have to contemplate whether to purchase other training products in order to pally for this. Should I, should I not? I don't know because I don't know how long we may have to wait to see this documentation. I am sure I am not the only one is this position.
So, this question would be for you, Sean and Bradley: what kind of timeline will we be looking at in order to get this?
Likewise, I know you indicated Bradley (in a different thread) that you folks were looking into making tag searches more usable "soon". In the last few weeks, I have been tagging numerous entries. Since I cannot search them meaningfully, I have been tagging them with a T- prefix in order to be able to find them with CTL-F. Needless to say, this is not a tidy solution. Again, since timing of when some of these remedies may take place is vague, it is hard to know what one should do. On the one hand, I may be doing something useless (adding the prefix to older entries) if the fix is for soon. On the other hand, I don't want this to be my permanent tagging system, so at some point, I will have to start removing these prefixes. All the while, I am really not sure how extensively I will be able to use my tags when I need them (which may be sooner than your fix if your fix takes too long to come out). You see the problem?
Bottom-line is that for those of us who have to make do with crutches until the promised remedies are implemented, it would good to know that we are not going to have to wait too long and to have a sense of what we can expect.
0 -
Bob answered some of the questions discussed in this thread here:
0 -
Performance improvements for this resource will be included in the next beta
Performance improvements were made in 6.1 for displaying and scrolling clause visualizations.
0 -
Just reporting personal experience; not complaining. Logos Bible Software 6.1.0.0016
- Visably better than 6.0. Even WIVU comes up pretty decently fast. But still not as smooth as L5. L5 seems to have been a classic in the 32 world.
- CNTTS remains clunky as in 6.0x. It scrolls along fine, until you hit the buffer/apparatus block. Then it's 5 secs to go get another block (similar to lightning > thunder).
All in all, I don't think significant; I'm happy.
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
Actually I really fail to see a single problem with the OP. It is basic in software development that the phases are completed in order before the next phase begins.
Years ago I was a DoD program manager for an Alpha test and our third party developer made a serious mistake in the file table build where there should be two options..mandatory or optional keystrokes M, O specifically. When we were given the software to test in preparation of a wide beta release, with our keystroke by keystroke testing, it was found that if any other key was hit than an M or O , the software it its then current form would interpret this as optional. Of course this would have been an error that if released would cause needless deaths so we immediately issued an SIR and the project was halted until that coding error was corrected.
I had received criticism for making my team do the keystroke at a time testing, but in doing so many otherwise missed program functionality errors were resolved before we released the beta builds. Even in the beta testing in the field we insisted on the same testing..i.e., keystroke by keystroke and encouraged our commands to continue submitting relevant SCR and SIRs yet because of our alpha testing few were submitted before this hardware/software system went live.
I realize that Logos is not a life/death software yet I am expecting software from a Christian company would not compromise but instead follow the Biblical admonition of Col 3:17 and unfortunately with all the reported problems this is not happening.
0
