Quality control in Logos 6 - a new perspective

Page 7 of 9 (175 items) « First ... < Previous 5 6 7 8 9 Next >
This post has 174 Replies | 12 Followers

Posts 8088
LogosEmployee

Denise:

Ok, Bradley. To avoid another thread titled 'Logos6 is so [fill in the blank]', here's a specific example (Sean can take a break; this is L6 performance).

wivucgraphs
2009-03-24T22:03:00Z
wivucgraphs.lbxclv

This resource has been a great bell-weather of Logos performance every since L4 was released.  I just tried it over on Libronix, it pages quickly, redraws quickly, opens quickly, and also stands on its head when requested.  On Logos4, it could barely move.  This went on thru the various L4 updates.  Then L5 came along, and it seemed to have woken up.  I even linked to my other OT resources, on my OT window.  It didn't slow anybody down.  Now L6 comes along, and the situation is very similar to L4 ... very slow to open, doesn't respond to pageups/downs, and so forth.  I had to de-link it, since OT Bible study had really pretty much stopped.

Thanks for the specific problem report. Performance improvements for this resource will be included in the next beta.

(FWIW, the code for displaying this resource hasn't changed since Logos 4. I can only hypothesise that you saw a speed-up in Logos 5 due to the move to .NET 4.5, and saw a slowdown in Logos 6 due to the switch to 64-bit. We've identified a performance bottleneck in the .NET Framework that's causing this to be slow and have developed a workaround in the next beta.)

Posts 763
LogosEmployee
Sean Boisen | Forum Activity | Replied: Tue, Jan 13 2015 4:54 PM

Francis:


... I think that what many users would like to hear (and actually see) is something like this: "Yes, we acknowledge that unfortunately many users have had a frustrating experience. We are starting a process to get to the bottom of how we have gotten there and what we need to do differently so that this cycle does not continue". ...



I acknowledge the frustration expressed in this thread: I've heard the message clearly, and I'm sorry for the frustrating experience. I'm taking steps to see where we can improve, and we'll do our best to ensure our data is as good as we can make it.

Posts 763
LogosEmployee
Sean Boisen | Forum Activity | Replied: Tue, Jan 13 2015 5:06 PM

Denise:

Seal my lips! Libby just works!  I was over there just minutes ago checking out Reuben's family tree (which Libby reports 6 sons; not 4 or 5 ... I think Eliab's hebrew might be confusing).

Denise, I'm not quite sure if you're reporting a problem here or not. If you are, here's my guess:

We have three family tree diagrams for Reuben:

  • The most general one shows Eliab below Pallu, based on Num 26:8, where ESV lists him as one of "the sons of Pallu". Dt 11:6 describes him as "son of Reuben".
  • This diagram and this one represent the text of 1 Chr 5:1-3 and 1-10, which lists four sons of Reuben.

Our editorial policy for this kind of data is to represent the text(s), rather than attempt to harmonize it, when we think the facts aren't clear. Honest people can disagree about these judgments, but we tend to be more conservative in such cases. There are Bible dictionaries for the deeper discussions, and we don't have enough space in a brief label like "<relative> of Pallu" to debate the issue.

Please try again if I've misunderstood.

Posts 623
JAL | Forum Activity | Replied: Tue, Jan 13 2015 5:37 PM

Francis, I would like once again to thank you for starting and moderating this thread.

I have a stake in the success of the Faithlife enterprise. This is the sum of my orders since 2008. I'd been buying Logos products for ten years already.

I hope the observations made in this thread result in a vast impvement of QA / QC at Faithlife.

If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled. Much of what I currently need from Faithlife works well.

I hope for change. I really do! I also don't believe that Faithlife is indifferent to our barking. Though for know it seems improvement will only come if users report specific problems and submit logs.

"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963

Posts 10339
Denise | Forum Activity | Replied: Tue, Jan 13 2015 6:22 PM

Sean ... I appreciate your help but you cogently illustrated 'the problem' (from one participant in this thread).  Earlier today, a gentileman wanted 'geneologies' in Logos6. TJ, another great gentleman, helped out with the new Factbook, and suggested 'Reuben'.  I decided I could learn from TJ, so I did what TJ suggested.  That's when it presented 5 people diagrams ... 1 for Reuben with a father and 4 sons, plus a great-grandfather, and several step-brothers randomly selected.

As a naive user, I THOUGHT that was as good has you had.  (Reuben's father and 4 sons).  A couple days back, again due to datasets, I assumed you had nothing on 1 Peter and concluded the Factbook only had a few of the Bible books completed.

Here's the problem.

- When you (not really you) curated the people diagrams and also the maps (a major earlier complaint), your algorythm or dataset has no ability to show the user the most likely to be needed (in this case 'Reuben'). Curated = amateurish upon delivery.

- When I DID find the 3 Reuben diagrams, 2 were labeled as to their source (good); one wasn't source-labeled, and totally disagreed with the other two.  That's when I checked Libby only to find another Reuben son.  Now, 3 mysteries.

- I went back to L6 to see what was causing the difference.  No hints.  I went back to Libby, who had kindly listed the Bible references for each of the participants ... thereby explaining the differences.  Logos6 was a waste of time (in this situation).

This is a very small example but it demonstrates that when you're trying to study, loosely designed, possibly incomplete work, wastes immense amounts of time.  Now, I never did find actual geneologies (the original question) even though (1) they're heavily emphasized in the Bible (2) they're critical to jewish mindsets and (3) knowing the geneology explains the major argument (eg Qumran, etc).


Posts 10339
Denise | Forum Activity | Replied: Tue, Jan 13 2015 6:29 PM

Bradley ... thank you for the follow-up.  I will say the Logos5 improvement was (for me) an amazing bit of wizardry in coding.  I had maybe 30-40 resources linked, and pagedowns had almost instant results even with the WIVU 'pig' jogging along.  Hoping you'll find the solution for Logos6.


Posts 27047
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Jan 13 2015 6:41 PM

JAL:

If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled.

What puzzles me is the lack of specifics. I agree that there are a number of QA issues and that Faithlife needs to make QA an integral part of company culture. But I also see a large number of full-time ministry users, academics, scholars who use Logos/Verbum as integral tool in their work. I have seen horror stories related to specific hardware/software configurations - they have my sympathy. But I have also seen a large number of complaints that are based on misunderstandings on the part of the user e.g. "incompatible" Factbook information that simply reflects different passages of Scripture; morphological "errors" compared to other products without recognizing differences of base text and linguistic theory, "missing data" that is there if one looks in the right place.

If you follow the forums, you'll see that many of the complaint threads belong in the "how do I" column or in faulty user memory of how previous versions work. All of which brings me back full circle to the need to distinguish between documentation & training issues, true system flaws (bugs and design) and user expectations of their dream software.

Note this is written after a nearly 3 day period of having neither my beta or my stable versions running which put an important project behind schedule very near the turnover to another person i.e. very bad time. 

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 27047
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Jan 13 2015 7:19 PM

Denise:

Here's the problem.

- When you (not really you) curated the people diagrams and also the maps (a major earlier complaint), your algorythm or dataset has no ability to show the user the most likely to be needed (in this case 'Reuben'). Curated = amateurish upon delivery.

My take on Reuben ... when I go to the Factbook and enter Reuben, I don't divulge anything that implies whether I am more interested in his ancestors or his descendants or a chart for a given Bible passage. What I get is:

A mouse over shows me who key individual of the chart and what passage, if any, the chart applies to.

The Search media link informs me that the results shown are incomplete. If I don't see what I want I should run a search.

Ah, I see I have many additional charts to chose between ... I chose two that are not tied to specific passages ... one with Reuben's ancestor Abraham as the focal point, one with Reuben himself as the focal point. But wait, as I do a mouse-over I find others that have Reuben as the focal point that are specific to a particular Bible passage.

Hey look at the symbols... two refer to the tribe of Reuben ... a collection of people referred to as "Reuben". I am a bit surprised that there was not a distinction between Reuben himself and Reuben's descendants ... I may have found a bug. 

Sorry Denise, I can't agree that it is hard to find what you want for genealogies - the focal point and the size of the chart are pretty good give aways. As for tied to a passage vs. not tied to a passage, if you want "not tied to a specific passage" added to the titles, it would do no harm but I am perfectly capable of recognizing there is a passage specified/there is not a passage specified.

I do not know how Logos/Verbum determined which items to show in the Factbook preview ... I will not defend it but will say that the most likely wanted hits is a difficult target shown by how often Google is nowhere in the ballpark and Bing hasn't even found the park.

So you are dissatisfied. Precisely how would you like the behavior changed to meet your needs? What information do you need to select the appropriate genealogies is not available? How would you like it displayed?

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 623
JAL | Forum Activity | Replied: Tue, Jan 13 2015 7:24 PM

MJ. Smith:

JAL:

If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled.

What puzzles me is the lack of specifics.

MJ. Smith:
Note this is written after a nearly 3 day period of having neither my beta or my stable versions running which put an important project behind schedule very near the turnover to another person i.e. very bad time.

?

MJ. Smith:
If you follow the forums

I've been addicted to following the forums from day one and have been guided and consoled by the effort of many for which I am grateful.

JAL:
Much of what I currently need from Faithlife works well.

What I need seems to work as I understand it was designed to, and often very well.

"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963

Posts 27047
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Jan 13 2015 7:39 PM

JAL:

MJ. Smith:

JAL:

If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled.

What puzzles me is the lack of specifics.

MJ. Smith:
Note this is written after a nearly 3 day period of having neither my beta or my stable versions running which put an important project behind schedule very near the turnover to another person i.e. very bad time.

?

You want me to embarrass myself by admitting how long it took me to figure out that there was nothing I could do to get Bradley more data when indexing eventually went to lots of memory but no processing? And that I had a big hint in the logs telling me they found a active index thread to kill despite Task Manager showing no task? Funny how restarting the machine rather than just the app cleared the problem ... the indexing completed AND let the main program take over. Still don't know how the problem from running the beta sync'ed to the production but all is well and I know running a beta leaves me out of a call to support so if I want to trust a project to a beta version, I have to take responsibility when it didn't exactly work out. And if I dedicated time to resolving it rather than multi-multi-tasking I hope I'd find it faster.Embarrassed

However, the point I was trying to make was that people express distrust as a whole rather than distrust of a particular feature or dataset where Faithlife can see if the data is actually error prone, or the algorithm is actually flawed or the user doesn't understand how to use the feature (also a Faithlife concern for an "intuitive" UI)

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 10339
Denise | Forum Activity | Replied: Tue, Jan 13 2015 7:44 PM

MJ ... no offense, but Sean caught the point at the very start.  The OT has a divergence in geneologies for Reuben, and always has.  Logos6 simply presents several pictures and doesn't provide info on why, which  Bible passages, etc.  Libby did (also written by Logos).  You, as a study-er, should have caught that (as Sean did). Logos6 didn't help you.  And you wasted your time writing your reply.

And no, you didn't show any geneologies, as the OP this day requested.  Who in their right mind is going to paste 10-15 of them together?

And give me a break on appealing to Google.  The Factbook query on maps is attrocious and even nuttier on Reuban (Asher???).


Posts 27047
Forum MVP
MJ. Smith | Forum Activity | Replied: Tue, Jan 13 2015 7:49 PM

Denise - I was showing that the divergence was to be expected not a "mystery" without any reference to Libby. If you are looking for the verbal explanation why limit the search to media? You have fun being a quick-witted naysayer; I have fun being the logical alternative finder. Luckily, the forum benefits from the contrast.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 623
JAL | Forum Activity | Replied: Tue, Jan 13 2015 8:11 PM

MJ. Smith:
However, the point I was trying to make was that people express distrust as a whole rather than distrust of a particular feature or dataset where Faithlife can see if the data is actually error prone, or the algorithm is actually flawed or the user doesn't understand how to use the feature (also a Faithlife concern for an "intuitive" UI)

The whole is the sum of it's parts, is it not?

MJ. Smith:
I know running a beta leaves me out of a call to support so if I want to trust a project to a beta version, I have to take responsibility 

I've only braved the beta testing process once with Verbum 5.3. That was cut short but I received an early upgrade to Verbum 6.

MJ. Smith:
Denise - ...You have fun being a quick-witted naysayer; I have fun being the logical alternative finder. Luckily, the forum benefits from the contrast.

This is why when my time is limited I go directly to the most recent posts by each of you - really.

"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963

Posts 8967
RIP
Matthew C Jones | Forum Activity | Replied: Tue, Jan 13 2015 8:12 PM

MJ. Smith:
You have fun being a quick-witted naysayer; I have fun being the logical alternative finder. Luckily, the forum benefits from the contrast.

I like contrast.  That is one reason I post.  And I like Logos.  That is one reason I don't curse the project.

JAL:
I have a stake in the success of the Faithlife enterprise. This is the sum of my orders since 2008.

          This is the sum of my orders since April 18th, 2008.              contrast.

                            

Logos 7 Collectors Edition

Posts 623
JAL | Forum Activity | Replied: Tue, Jan 13 2015 8:31 PM

Super.Tramp:
This is the sum of my orders since April 18th, 2008.              contrast.

Yes - many have a much larger stake than mine. Not merely in financial terms but in meaningful time spent contributing to the benefit of other users.

Thank you Super.Tramp.

"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963

Posts 150
LogosEmployee
Todd White (Faithlife) | Forum Activity | Replied: Tue, Jan 13 2015 10:09 PM

Sean Boisen:

Francis:


... I think that what many users would like to hear (and actually see) is something like this: "Yes, we acknowledge that unfortunately many users have had a frustrating experience. We are starting a process to get to the bottom of how we have gotten there and what we need to do differently so that this cycle does not continue". ...



I acknowledge the frustration expressed in this thread: I've heard the message clearly, and I'm sorry for the frustrating experience. I'm taking steps to see where we can improve, and we'll do our best to ensure our data is as good as we can make it.

Like Sean I would like to acknowledge the frustration expressed in this thread.

Some have asked for specifics to what we are doing to address these issues. The following is a bit of insider information most of which was in motion before this thread started.

Starting after the Logos 6 launch the Desktop team set a few plans in motion to actively improve the stability and performance of the application. During 6.0a and 6.0b the majority of the team has been focused on bug fixing and stability.

In December we built a test harness to give us active feedback on performance. The goal of this tool is with every internal build to have a set of performance tests automatically run and give us reports similar to http://www.arewefastyet.com/. Having tools like this will help us to catch regressions in performance for key parts of the application. Last week with the test harness somewhat usable we spun up a team to focus on improving performance. However as Bradley has stated this can be hard without specifics. We know some of the key pain points, but sometimes we are surprised with the ways that people leverage the tools we built. Without knowing those surprising ways it is difficult to fix your pet issue. Like using a shoe as a hammer it isn't that it is wrong, but we may need to improve the shoe design if we know people are regularly using it that way.

My goal for the team is to see improved stability and performance in upcoming releases. Even with new features we hope to maintain a reasonable quality bar and respond quickly to issues we miss. To this end we will be experimenting with shortening our development cycle to every six weeks. This shorter cycle will have fewer changes in each release. This will give our beta testers, internal testers, and developers greater focus for each beta cycle. The shortly cycle will also enable us to get fixes and performance improvements into peoples hands sooner.

I appreciate that you all continue to hold us to a high standard for the product we produce. While the developers are not always actively posting on the forums many of them read a lot of this content. We are often talking about and sharing content from here and challenge each other through the challenges you post to us.

Thank you,

Todd

P.S. In addition, as Bob mentioned over at https://community.logos.com/forums/p/98999/684866.aspx#684866, we are even beefing up the mobile team to improve the quality and feature set of those applications.

Posts 623
JAL | Forum Activity | Replied: Tue, Jan 13 2015 10:37 PM

Todd White (Faithlife):
I appreciate that you all continue to hold us to a high standard for the product we produce.

Thank you all.Dog

"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963

Posts 468
BKMitchell | Forum Activity | Replied: Wed, Jan 14 2015 2:24 AM

Super.Tramp:
子犬を抱きしめ

Translation:

Hug a puppy (koinu o dakishime)

そんなの関係ねぇ と思いますStick out tongue

 

חַפְּשׂוּ בַּתּוֹרָה הֵיטֵב וְאַל תִּסְתַּמְּכוּ עַל דְּבָרַי

Posts 3691
Francis | Forum Activity | Replied: Wed, Jan 14 2015 6:36 AM

Thank you Sean and Todd. I appreciate both what acknowledgement there is and the sharing of some of the initiatives. Ultimately, I will be more interested in seeing how it translates in the end-user experience. I recognize also that you have ventured into new areas that have snowballed beyond your expectations. 

A few suggestions:

1. I refer again to the example of searching for the "religious feasts" as a cultural concept in John only to find that Passover and the Feast of Dedication were not included. I think that there will always be room to debate this kind of categorization. But ways must be found to bridge between how the database builders may think and the end-user. For instance, include suggestions of alternate dataset entries before the results (as in the Topic Guide). In this case, "Passover, Feast of Dedication..." or "Festivals (Bible Sense Lexicon)". In addition (and perhaps it would be simpler to implement) design a lexicon for each database. Make sure that every single entry is in it and that it tells the user where to find it. The BSL does a bit of that and Factbook too, though it does not work with Bible references. The cultural concepts section in PG is too specific to help locate the blindspots (see observations about the centrality of the search window below).

2. In the case of tags, I guess it's more of a question than a suggestion: I simply do not understand how tagging of user-documents could be introduced as a feature if it is not properly searchable (and for that matter, the same is true of all user-document searches). How did you intend this to be used? 

3. There have been discussions of release cycles. I am not personally bent on any specific duration. A great job can be done in 2 years and a lousy job in 5 years. I hear also what Sean said -- that new features and resources are the bread and butter of the company -- but, I wonder, how are the other Bible software companies faring that do not release as much as often? How are they surviving? I don't know the answer to this question but I am wondering, and especially whether it must be so with Faithlife. Regardless, from the standpoint of user satisfaction, I would want to answer one who asked whether all innovations should be suspended until everything is fixed? As much as the latter could be desired, I think it a more sensible approach not to set these in opposition but in relationship. Namely, keep on developing but ease up with it and increase proportionately fixing what is already here and upping quality control of new features before they come out. Say it were 50-50 right now, I would say go 30-70 instead. I do think that in the present situation, fixing has become an emergency (this includes not only bugs and missing data but also documentation). In the long-run, it's not all about having more toys to play with. I trust that the company and we the customers can be a bit more discerning there.

4. Someone stated that Logos works well most of the time. This is the type of situation in which Logos works well: I need to read on socio-rhetorical studies, I search my library (or use lookup, factbook, etc) and find enough hits to satisfy my need. However, if I were looking for a specific article on socio-rhetorical studies, I may find that it is not coming up and that it is not tagged properly. There are many of my tasks that are of the first kind (fairly simple or broad) and yes, for these, Logos works rather well and is very useful. But in the second category, where specificity and accuracy are important, that's where problems tend to arise (in my experience very frequently). I would suggest that the core of Logos will always be its searching capability. It's nice to go and open the Atlas or BSL and look up something there, but the meat is found in the search window. When I spoke of trust before, this is one of the key areas in my thinking: when I search something specific, how often can I trust that looking for x will find all x? So whenever you design a dataset, regardless of if there is a link under tools for it, a neat concept or a shiny interface: you should expect and design it knowing that users will build searches based on it and will judge its value accordingly.

5. Never release a new feature without solid documentation otherwise you can be guaranteed that users will be frustrated, will misuse it and that it will project badly on both the product and the company.

Of course, I am CERTAIN that not everybody will agree. 

Posts 692
Kevin A Lewis | Forum Activity | Replied: Wed, Jan 14 2015 6:49 AM

Francis:

Of course, I am CERTAIN that not everybody will agree. 

However I do! - and I trust Faithlife will come to see the wisdom of some of this - at least blending in an appreciation of these issue with their other concerns that we (as users/customer) may be unaware of.

Blessings all.

And kudos to you Francis for sticking with this - against a lot of pushback - interestingly little of which was from the company.

Shalom.

Page 7 of 9 (175 items) « First ... < Previous 5 6 7 8 9 Next > | RSS