Quality control in Logos 6 - a new perspective
Comments
-
I acknowledge the frustration expressed in this thread: I've heard the message clearly, and I'm sorry for the frustrating experience. I'm taking steps to see where we can improve, and we'll do our best to ensure our data is as good as we can make it.
... I think that what many users would like to hear (and actually see) is something like this: "Yes, we acknowledge that unfortunately many users have had a frustrating experience. We are starting a process to get to the bottom of how we have gotten there and what we need to do differently so that this cycle does not continue". ...
0 -
Seal my lips! Libby just works! I was over there just minutes ago checking out Reuben's family tree (which Libby reports 6 sons; not 4 or 5 ... I think Eliab's hebrew might be confusing).
Denise, I'm not quite sure if you're reporting a problem here or not. If you are, here's my guess:
We have three family tree diagrams for Reuben:
- The most general one shows Eliab below Pallu, based on Num 26:8, where ESV lists him as one of "the sons of Pallu". Dt 11:6 describes him as "son of Reuben".
- This diagram and this one represent the text of 1 Chr 5:1-3 and 1-10, which lists four sons of Reuben.
Our editorial policy for this kind of data is to represent the text(s), rather than attempt to harmonize it, when we think the facts aren't clear. Honest people can disagree about these judgments, but we tend to be more conservative in such cases. There are Bible dictionaries for the deeper discussions, and we don't have enough space in a brief label like "<relative> of Pallu" to debate the issue.
Please try again if I've misunderstood.
0 -
Francis, I would like once again to thank you for starting and moderating this thread.
I have a stake in the success of the Faithlife enterprise. This is the sum of my orders since 2008. I'd been buying Logos products for ten years already.
I hope the observations made in this thread result in a vast impvement of QA / QC at Faithlife.
If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled. Much of what I currently need from Faithlife works well.
I hope for change. I really do! I also don't believe that Faithlife is indifferent to our barking. Though for know it seems improvement will only come if users report specific problems and submit logs.
"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963
0 -
Sean ... I appreciate your help but you cogently illustrated 'the problem' (from one participant in this thread). Earlier today, a gentileman wanted 'geneologies' in Logos6. TJ, another great gentleman, helped out with the new Factbook, and suggested 'Reuben'. I decided I could learn from TJ, so I did what TJ suggested. That's when it presented 5 people diagrams ... 1 for Reuben with a father and 4 sons, plus a great-grandfather, and several step-brothers randomly selected.
As a naive user, I THOUGHT that was as good has you had. (Reuben's father and 4 sons). A couple days back, again due to datasets, I assumed you had nothing on 1 Peter and concluded the Factbook only had a few of the Bible books completed.
Here's the problem.
- When you (not really you) curated the people diagrams and also the maps (a major earlier complaint), your algorythm or dataset has no ability to show the user the most likely to be needed (in this case 'Reuben'). Curated = amateurish upon delivery.
- When I DID find the 3 Reuben diagrams, 2 were labeled as to their source (good); one wasn't source-labeled, and totally disagreed with the other two. That's when I checked Libby only to find another Reuben son. Now, 3 mysteries.
- I went back to L6 to see what was causing the difference. No hints. I went back to Libby, who had kindly listed the Bible references for each of the participants ... thereby explaining the differences. Logos6 was a waste of time (in this situation).
This is a very small example but it demonstrates that when you're trying to study, loosely designed, possibly incomplete work, wastes immense amounts of time. Now, I never did find actual geneologies (the original question) even though (1) they're heavily emphasized in the Bible (2) they're critical to jewish mindsets and (3) knowing the geneology explains the major argument (eg Qumran, etc).
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
Bradley ... thank you for the follow-up. I will say the Logos5 improvement was (for me) an amazing bit of wizardry in coding. I had maybe 30-40 resources linked, and pagedowns had almost instant results even with the WIVU 'pig' jogging along. Hoping you'll find the solution for Logos6.
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled.
What puzzles me is the lack of specifics. I agree that there are a number of QA issues and that Faithlife needs to make QA an integral part of company culture. But I also see a large number of full-time ministry users, academics, scholars who use Logos/Verbum as integral tool in their work. I have seen horror stories related to specific hardware/software configurations - they have my sympathy. But I have also seen a large number of complaints that are based on misunderstandings on the part of the user e.g. "incompatible" Factbook information that simply reflects different passages of Scripture; morphological "errors" compared to other products without recognizing differences of base text and linguistic theory, "missing data" that is there if one looks in the right place.
If you follow the forums, you'll see that many of the complaint threads belong in the "how do I" column or in faulty user memory of how previous versions work. All of which brings me back full circle to the need to distinguish between documentation & training issues, true system flaws (bugs and design) and user expectations of their dream software.
Note this is written after a nearly 3 day period of having neither my beta or my stable versions running which put an important project behind schedule very near the turnover to another person i.e. very bad time.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
Here's the problem.
- When you (not really you) curated the people diagrams and also the maps (a major earlier complaint), your algorythm or dataset has no ability to show the user the most likely to be needed (in this case 'Reuben'). Curated = amateurish upon delivery.
My take on Reuben ... when I go to the Factbook and enter Reuben, I don't divulge anything that implies whether I am more interested in his ancestors or his descendants or a chart for a given Bible passage. What I get is:
A mouse over shows me who key individual of the chart and what passage, if any, the chart applies to.
The Search media link informs me that the results shown are incomplete. If I don't see what I want I should run a search.
Ah, I see I have many additional charts to chose between ... I chose two that are not tied to specific passages ... one with Reuben's ancestor Abraham as the focal point, one with Reuben himself as the focal point. But wait, as I do a mouse-over I find others that have Reuben as the focal point that are specific to a particular Bible passage.
Hey look at the symbols... two refer to the tribe of Reuben ... a collection of people referred to as "Reuben". I am a bit surprised that there was not a distinction between Reuben himself and Reuben's descendants ... I may have found a bug.
Sorry Denise, I can't agree that it is hard to find what you want for genealogies - the focal point and the size of the chart are pretty good give aways. As for tied to a passage vs. not tied to a passage, if you want "not tied to a specific passage" added to the titles, it would do no harm but I am perfectly capable of recognizing there is a passage specified/there is not a passage specified.
I do not know how Logos/Verbum determined which items to show in the Factbook preview ... I will not defend it but will say that the most likely wanted hits is a difficult target shown by how often Google is nowhere in the ballpark and Bing hasn't even found the park.
So you are dissatisfied. Precisely how would you like the behavior changed to meet your needs? What information do you need to select the appropriate genealogies is not available? How would you like it displayed?
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled.
What puzzles me is the lack of specifics.
Note this is written after a nearly 3 day period of having neither my beta or my stable versions running which put an important project behind schedule very near the turnover to another person i.e. very bad time.
?
If you follow the forums
I've been addicted to following the forums from day one and have been guided and consoled by the effort of many for which I am grateful.
Much of what I currently need from Faithlife works well.
What I need seems to work as I understand it was designed to, and often very well.
"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963
0 -
If I needed to use Faithlife products for full-time ministry, academic, or scholarly pursuits I think I'd be troubled.
What puzzles me is the lack of specifics.
Note this is written after a nearly 3 day period of having neither my beta or my stable versions running which put an important project behind schedule very near the turnover to another person i.e. very bad time.
?
You want me to embarrass myself by admitting how long it took me to figure out that there was nothing I could do to get Bradley more data when indexing eventually went to lots of memory but no processing? And that I had a big hint in the logs telling me they found a active index thread to kill despite Task Manager showing no task? Funny how restarting the machine rather than just the app cleared the problem ... the indexing completed AND let the main program take over. Still don't know how the problem from running the beta sync'ed to the production but all is well and I know running a beta leaves me out of a call to support so if I want to trust a project to a beta version, I have to take responsibility when it didn't exactly work out. And if I dedicated time to resolving it rather than multi-multi-tasking I hope I'd find it faster.[:$]
However, the point I was trying to make was that people express distrust as a whole rather than distrust of a particular feature or dataset where Faithlife can see if the data is actually error prone, or the algorithm is actually flawed or the user doesn't understand how to use the feature (also a Faithlife concern for an "intuitive" UI)
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
MJ ... no offense, but Sean caught the point at the very start. The OT has a divergence in geneologies for Reuben, and always has. Logos6 simply presents several pictures and doesn't provide info on why, which Bible passages, etc. Libby did (also written by Logos). You, as a study-er, should have caught that (as Sean did). Logos6 didn't help you. And you wasted your time writing your reply.
And no, you didn't show any geneologies, as the OP this day requested. Who in their right mind is going to paste 10-15 of them together?
And give me a break on appealing to Google. The Factbook query on maps is attrocious and even nuttier on Reuban (Asher???).
"If myth is ideology in narrative form, then scholarship is myth with footnotes." B. Lincolm 1999.
0 -
Denise - I was showing that the divergence was to be expected not a "mystery" without any reference to Libby. If you are looking for the verbal explanation why limit the search to media? You have fun being a quick-witted naysayer; I have fun being the logical alternative finder. Luckily, the forum benefits from the contrast.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
However, the point I was trying to make was that people express distrust as a whole rather than distrust of a particular feature or dataset where Faithlife can see if the data is actually error prone, or the algorithm is actually flawed or the user doesn't understand how to use the feature (also a Faithlife concern for an "intuitive" UI)
The whole is the sum of it's parts, is it not?
I know running a beta leaves me out of a call to support so if I want to trust a project to a beta version, I have to take responsibility
I've only braved the beta testing process once with Verbum 5.3. That was cut short but I received an early upgrade to Verbum 6.
Denise - ...You have fun being a quick-witted naysayer; I have fun being the logical alternative finder. Luckily, the forum benefits from the contrast.
This is why when my time is limited I go directly to the most recent posts by each of you - really.
"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963
0 -
You have fun being a quick-witted naysayer; I have fun being the logical alternative finder. Luckily, the forum benefits from the contrast.
I like contrast. That is one reason I post. And I like Logos. That is one reason I don't curse the project.
I have a stake in the success of the Faithlife enterprise. This is the sum of my orders since 2008.
This is the sum of my orders since April 18th, 2008.
contrast.Logos 7 Collectors Edition
0 -
This is the sum of my orders since April 18th, 2008.
contrast.Yes - many have a much larger stake than mine. Not merely in financial terms but in meaningful time spent contributing to the benefit of other users.
Thank you Super.****.
"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963
0 -
I acknowledge the frustration expressed in this thread: I've heard the message clearly, and I'm sorry for the frustrating experience. I'm taking steps to see where we can improve, and we'll do our best to ensure our data is as good as we can make it.
... I think that what many users would like to hear (and actually see) is something like this: "Yes, we acknowledge that unfortunately many users have had a frustrating experience. We are starting a process to get to the bottom of how we have gotten there and what we need to do differently so that this cycle does not continue". ...Like Sean I would like to acknowledge the frustration expressed in this thread.
Some have asked for specifics to what we are doing to address these issues. The following is a bit of insider information most of which was in motion before this thread started.
Starting after the Logos 6 launch the Desktop team set a few plans in motion to actively improve the stability and performance of the application. During 6.0a and 6.0b the majority of the team has been focused on bug fixing and stability.
In December we built a test harness to give us active feedback on performance. The goal of this tool is with every internal build to have a set of performance tests automatically run and give us reports similar to http://www.arewefastyet.com/. Having tools like this will help us to catch regressions in performance for key parts of the application. Last week with the test harness somewhat usable we spun up a team to focus on improving performance. However as Bradley has stated this can be hard without specifics. We know some of the key pain points, but sometimes we are surprised with the ways that people leverage the tools we built. Without knowing those surprising ways it is difficult to fix your pet issue. Like using a shoe as a hammer it isn't that it is wrong, but we may need to improve the shoe design if we know people are regularly using it that way.
My goal for the team is to see improved stability and performance in upcoming releases. Even with new features we hope to maintain a reasonable quality bar and respond quickly to issues we miss. To this end we will be experimenting with shortening our development cycle to every six weeks. This shorter cycle will have fewer changes in each release. This will give our beta testers, internal testers, and developers greater focus for each beta cycle. The shortly cycle will also enable us to get fixes and performance improvements into peoples hands sooner.
I appreciate that you all continue to hold us to a high standard for the product we produce. While the developers are not always actively posting on the forums many of them read a lot of this content. We are often talking about and sharing content from here and challenge each other through the challenges you post to us.
Thank you,
Todd
P.S. In addition, as Bob mentioned over at https://community.logos.com/forums/p/98999/684866.aspx#684866, we are even beefing up the mobile team to improve the quality and feature set of those applications.
0 -
I appreciate that you all continue to hold us to a high standard for the product we produce.
Thank you all.[&]
"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963
0 -
子犬を抱きしめ
Translation:
Hug a puppy (koinu o dakishime)
そんなの関係ねぇ と思います[:P]。
חַפְּשׂוּ בַּתּוֹרָה הֵיטֵב וְאַל תִּסְתַּמְּכוּ עַל דְּבָרַי
0 -
Thank you Sean and Todd. I appreciate both what acknowledgement there is and the sharing of some of the initiatives. Ultimately, I will be more interested in seeing how it translates in the end-user experience. I recognize also that you have ventured into new areas that have snowballed beyond your expectations.
A few suggestions:
1. I refer again to the example of searching for the "religious feasts" as a cultural concept in John only to find that Passover and the Feast of Dedication were not included. I think that there will always be room to debate this kind of categorization. But ways must be found to bridge between how the database builders may think and the end-user. For instance, include suggestions of alternate dataset entries before the results (as in the Topic Guide). In this case, "Passover, Feast of Dedication..." or "Festivals (Bible Sense Lexicon)". In addition (and perhaps it would be simpler to implement) design a lexicon for each database. Make sure that every single entry is in it and that it tells the user where to find it. The BSL does a bit of that and Factbook too, though it does not work with Bible references. The cultural concepts section in PG is too specific to help locate the blindspots (see observations about the centrality of the search window below).
2. In the case of tags, I guess it's more of a question than a suggestion: I simply do not understand how tagging of user-documents could be introduced as a feature if it is not properly searchable (and for that matter, the same is true of all user-document searches). How did you intend this to be used?
3. There have been discussions of release cycles. I am not personally bent on any specific duration. A great job can be done in 2 years and a lousy job in 5 years. I hear also what Sean said -- that new features and resources are the bread and butter of the company -- but, I wonder, how are the other Bible software companies faring that do not release as much as often? How are they surviving? I don't know the answer to this question but I am wondering, and especially whether it must be so with Faithlife. Regardless, from the standpoint of user satisfaction, I would want to answer one who asked whether all innovations should be suspended until everything is fixed? As much as the latter could be desired, I think it a more sensible approach not to set these in opposition but in relationship. Namely, keep on developing but ease up with it and increase proportionately fixing what is already here and upping quality control of new features before they come out. Say it were 50-50 right now, I would say go 30-70 instead. I do think that in the present situation, fixing has become an emergency (this includes not only bugs and missing data but also documentation). In the long-run, it's not all about having more toys to play with. I trust that the company and we the customers can be a bit more discerning there.
4. Someone stated that Logos works well most of the time. This is the type of situation in which Logos works well: I need to read on socio-rhetorical studies, I search my library (or use lookup, factbook, etc) and find enough hits to satisfy my need. However, if I were looking for a specific article on socio-rhetorical studies, I may find that it is not coming up and that it is not tagged properly. There are many of my tasks that are of the first kind (fairly simple or broad) and yes, for these, Logos works rather well and is very useful. But in the second category, where specificity and accuracy are important, that's where problems tend to arise (in my experience very frequently). I would suggest that the core of Logos will always be its searching capability. It's nice to go and open the Atlas or BSL and look up something there, but the meat is found in the search window. When I spoke of trust before, this is one of the key areas in my thinking: when I search something specific, how often can I trust that looking for x will find all x? So whenever you design a dataset, regardless of if there is a link under tools for it, a neat concept or a shiny interface: you should expect and design it knowing that users will build searches based on it and will judge its value accordingly.
5. Never release a new feature without solid documentation otherwise you can be guaranteed that users will be frustrated, will misuse it and that it will project badly on both the product and the company.
Of course, I am CERTAIN that not everybody will agree.
0 -
Of course, I am CERTAIN that not everybody will agree.
However I do! - and I trust Faithlife will come to see the wisdom of some of this - at least blending in an appreciation of these issue with their other concerns that we (as users/customer) may be unaware of.
Blessings all.
And kudos to you Francis for sticking with this - against a lot of pushback - interestingly little of which was from the company.
Shalom.
0 -
at least blending in an appreciation of these issue with their other concerns that we (as users/customer) may be unaware of.
[Y]0 -
Thanks to Francis, Sean, Bradley, and Todd.
I'm going to try to offer some concrete suggestions and feedback about specific areas I've seen could be improved. Some of these are quite specific, but maybe even those can help to identify broader problems, or at least areas where it might be worthwhile to do a review.
1. Clause Search data. I think Dave had mentioned earlier in this thread he doesn't trust the Clause Search too much, and I have reason to not always trust it either, although it has certainly worked correctly for me most of the time. Here is an email I sent in on the 9th of October, 2014 to data@logos.com. Are these in fact errors, or am I missing something about how this tool is supposed to work?
[quote]
Hello,
I ran a Clause Search today for subject-lemma:אֱלֹהִים verb-morph:V????P and was surprised by the large number of hits that I got where אֱלֹהִים was not a subject at all. It made me worried about the quality of the markup in the Clause Search database.
Some examples of what I would consider false hits (i.e., where Elohim is not in any way a subject in the phrase, not even in a broad sense of being included in a prepositional phrase which modifies the subject) are:
Judg 20:2
1 Sam 8:8
2 Kings 17:7
2 Chr 11:16; 36:16
And I could continue…
Please respond to this email to acknowledge it. In the past I have written to this address but never received a response. If I don’t receive a response I’ll post to the forums. Thanks.
Fr Devin
No response to the email to the date. Apart from the response, I wonder if these errors (supposing they are errors) are indicative of broader problems regarding the classification of subjects of clauses (or of other types of classification?).
2. The lack of response from data@logos.com. The second is related, and that is the type of response that is given from data@logos.com. That is, none. I think that an email should be sent thanking people for sending in corrections. I'm still not even sure if you ever received this email.
3. A specific problem with the Bible Sense Lexicon. I reported this problem to data@logos.com on the 1st of July, 2013. I have never received a response, and the problem persists:
[quote]
In Genesis 49:24, 2 Samuel 23:3 and Isaiah 30:29, the word “stone” or “rock” in the phrase “Rock of Israel” or “Stone of Israel” is currently classified under the sense “stone”. It should be classified as “God ⇔ rock”.
My first report could be excused because you were busy releasing Logos and Verbum 6, but given that you have had 18 months to fix this report, and it is easy to fix, it might indicate a serious problem with the way these emails are handled. The sense “God ⇔ rock” does already exist, so I am pretty sure it's not just that you aren't yet doing this type of classification.
4. Morph tagging on the Biblical Dead Sea Scrolls. As I have said before, I am quite happy with the morph tagging of the Greek and Hebrew Bible, and have never had trouble with the Septuagint tagging either. I have not been as happy with the tagging of the Biblical Dead Sea Scrolls, although I can't say it's bad either, just not quite up to par to what I would hope for. From October 2012 to January 2013 I worked quite a bit with these texts at the Hebrew University, and remember running across a surprising number of errors (maybe one or two per chapter? I'm not sure). At the time, I sent in a bunch of corrections through the "Report a Typo" mechanism. If you have those Typo reports still registered somewhere or another, I would recommend trying to pull them up and look through them to see if they are accurate, and then based on what you find there get a feel for how few or how many errors there may be by extrapolation and how worth it or not a revision would be.
5. Recommendations for Documentation. I am very happy to see the commitment to document tools that are of use for academics. I hope that by the time Logos 7 comes out, all the tools of most interest to academics have been documented. I think that it is important that there be a place to go to that is specifically for academics. I know that Logos has a huge and broad market, but as this thread has made clear, there are certain requirements of academics that are specific.
Here are the tools that I think should be highest priority... I'm sure I'm missing something, but anyway, here is what I see as being most important, with a few notes about things to not forget to include (not meant to be complete, just stuff you could forget but shouldn't):
- Logos Greek, Hebrew, Aramaic, and Latin Morphology databases. Ideally other morph databases as well, at least providing easy access when not already available to the documentation provided by the authors.
- Clause search database - Remember to document how you classify the subject of imperative verbs, and how to find the one actually doing the ordering using {Speaker <Person>}
- Biblical Sense Lexicon
- Factbook tagging - Include what is tagged and what isn't (e.g. abstract nouns, etc.). Include difference between something like <Event> and {Section <Event>}
- Reverse Interlinear tagging - After reading the docs on this, users should know things like they need to use the SBLGNT and LHB for reliable searches, they should know what criteria are used to markup words as primary or secondary, etc.
- Ancient Versions section of the Textual Variants tool (Rick Brannan already documented this in detail during the Logos 6 beta process on the Beta forum, it would be enough to rework that post).
- Ancient Literature Tool (Rick Brannan already documented this in detail during the Logos 6 beta process on the Beta forum, it would be enough to rework those posts, and hopefully provide more detail about the automated logic that went in to the data mining of the Church Fathers references).
- Journals tool - Document what is included and what not
- LiteraryType dataset - Especially important as this dataset is really quite poor as a dataset, in part as it was not designed to be used as a dataset, and so has different criteria and quality than the other datasets, and so users need to know what they can expect
- Cultural Concepts
- Cross References Tool
As I said in a previous post, I don't think documenting these tools needs to be something out of this world, or even take that long. The basic question is what does an academic need to know to use these tools, who did them, including even interns, criteria used, and what their limitations are, what is missing in the tagging, etc.
BTW, in my opinion Rick Brannon is an example of someone who seems to understand and do these things already. He also has a keen sense of the importance of releasing complete and accurate data. Thanks, Rick! I'm sure there are many others as well, I have just particularly noticed that about him, so, thanks!
6. LiteraryType dataset - I've mentioned this before, but it's worth repeating. This dataset is particularly poor as a dataset. I have been happy with the other datasets in general, but not this one. I really think you should consider redoing it in house. At the very least, it should be a priority for documentation to warn users about its limitations, and explain why this dataset is not like the others....
7. Communication regarding incomplete datasets - In an earlier post, Sean earlier mentioned the Factbook Septuagint Deuterocanonicals dataset. For me, that dataset is actually an example of decent communication. It was always advertised as being only the Deuterocanonicals. Perfect. What is problematic is when datasets are published and it isn't clear what is missing. Think the Journals tool, Outlines Tool, Bible Book Factbook Entries, etc.
Don't get me wrong - I fully agree you need to advertise, be positive, etc. I don't think these limitations need to be bragged about. But at the same time the information needs to be available.
For tools like the Journal tool, the Outlines Tool, the Bible Book Factbook Entries, etc. that are currently being worked on, ideally I think there would be either a resource in the library that listed what works were included, or maybe a webpage with that info, or maybe both. When the datasets are updated to add more works, the list of works included is also updated.
Maybe as well on the Guides and Factbook sections there could be some sort of standard Info bubble, or help bubble that is available when documentation is available, that would bring you to the academic documentation / list of works included up to now, etc. Some tools would always have this info button (such as the Ancient Literature tool), others only when it applies (such as the Factbook entry for a Bible book).
Thanks everybody!
0 -
Excellent post, Fr Devin.
Years ago, I suggested something akin to the Literary Type Dataset, but realized at the time that in my own academic work, I would never trust someone else's categorization of a text. Instead, I wanted a way to develop my own dataset, perhaps in conjunction with a published one.
By saying this, I'm not intending to make a suggestion in this current thread, but just a recognition that beyond QC issues, there will always be judgments that may be hard to document completely. Yet, such documentation should reveal as much as is comfortably possible about the approach, philosophy, and qualifications of the persons making these dataset assessments.
0 -
Just for the record, I fully agree with all of your qualifications Lew, and that has been one of my reasons to be somewhat leery when criticisms of the datasets come without specific examples, especially given how common it is on these forums for people to misunderstand how a feature works and misinterpret it as an error.
In the case of the LiteraryType dataset, here is a post with some examples of the type of problems this dataset has: https://community.logos.com/forums/p/94632/660593.aspx. Note these aren't really errors per se... they are more just a design that isn't optimized for the type of searches people would ideally like to do from a dataset like this.
I would peronally still prefer having it than not having it, but I do think it is one where you have to be careful. And, the fact that Faithlife didn't design the data may have implications (?) as regards their ability to correct it or adapt it based on how it's being used??? I'm not sure.
Either way, as I've said before, if you keep in mind where something comes from and what its limitations are, it can almost always be useful (cf. https://community.logos.com/forums/t/92669.aspx for the source of the LiteraryType dataset, also present in the Info window of this resource, although it should also be in an Introduction to that resource as well).
EDIT: Looks like Sean had already said he was hoping to improve and correct this dataset: https://community.logos.com/forums/t/98894.aspx. Good news. If it is low on the priority list, hopefully documentation of its sources (and limitations) can be moved up high on the priority list, at least.
0 -
Thank you, Devin, for your great posts.
given how common it is on these forums for people to misunderstand how a feature works and misinterpret it as an error
I would certainly plead guilty there a number of times. But I think that lack of proper documentation, being used to encounter real errors frequently (typos, bugs and missing data) and the fact that most of the time, I am just trying to get something done, not engage in an inductive study of features/datasets all contribute quite a bit to this happening. As an analogy, one rarely accuses wrongly of lying someone who is considered truthful, but it is easier to mistakenly think that someone is lying when you know they do it often.
And I and others are perhaps just not as smart as you and some others can be. But even conceding this is telling: what's required of a user to be able to use Logos' features in its present state? I feel for users who say that they feel that Logos has become too complicated to use. I think that it does not have to be this way. For instance, milestones search is a great idea but it is beyond usability for most of us who find it tedious (and often unsuccessful) to try to figure out the proper syntax and datatypes format. If only it could be made more user-friendly (better documentation among other things) it would be more accessible. But when you go to a context menu or a resource box to find out the datatype and then try to type it with the milestone search syntax and it does not work you can 1) give up or 2) not understanding what's wrong, think something's is defective (this would be a kind of wrong error report that you alluded to). For my part, my mind is sufficiently crowded with what pertains to my work and there's just not always room for having to do what seems like quadratic equations in order to use a feature.
I speak from personal experience, the last time I tried it, that's exactly what happened. After posting about it, I finally found the answer for that query. Yet the solution was by no means simple to reproduce (though it was a simple milestones search) and the next time I attempted it, I had it wrong again (could not remember the "clear-as-mud" way of doing this!). Bottom-line: I don't use it. As for my wife, who is also doing post-doctorate studies but is less techie than I, the moment I start trying to explain how we're supposed to do this, I've lost her. I would say that she uses perhaps 10% of the functionality I use and she is considered to be a very smart person. I would say that most academics that I know are also in that boat. Most are not techno-geeks however brilliant they may be otherwise.
My point is this: we may have a lot of false positives on the forums, but I would still insist that a lot (not all) of it originates in problems with the product itself. I repeat what I said from the very beginning: there is great promise in the product, wonderful innovations. It is very useful. But at the same time, it is very frustrating and really needs a better finish.
0 -
Thanks Fr. Devin for your concrete, actionable post. Because of different interests and goals in my use of Verbum/Logos, I have a few additional items and probably a different order of priorities but you are both spot on re: what is needed and how to inform Logos effectively of what is needed.
Lew Worthington, your point regarding never trusting someone else's categorization of the text is also spot on. The ability to put notes on nearly everything has finally given us non-scholars the ability to note our own questions and disagreements with the tagging offers. Now Logos needs to make those notes more visible
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
Francis, you make a good point that is requires serious thought. Much of the "advanced" Logos tools either require knowledge of other Logos features or general knowledge. It is often like throwing someone with no knowledge of algebra into calculus (yes, thanks to a much older brother I do know exactly what that feels like at the age of 10). Do you have some suggestions as to how Logos can gently indicate the requisite knowledge to use the tools while not insulting the "jump in with both feet I'll figure it out" folks? Its not a matter of brightness so much as one of style of learning.
Orthodox Bishop Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."; Orthodox proverb: "We know where the Church is, we do not know where it is not."
0 -
For instance, milestones search is a great idea but it is beyond usability for most of us who find it tedious (and often unsuccessful) to try to figure out the proper syntax and datatypes format.
We added this feature knowing that it was complex to use in its current state, but betting that for the people who did understand it, it would be an incredibly powerful feature that opens up all sort of new search possibilities.
Was this the wrong decision? Should we hold back all features until they're sufficiently usable? Would Logos 6 be better right now if we took out Milestone Search?
Bottom-line: I don't use it.
That's OK with us. It was not our design goal in the original version of Logos 6 to make a milestone search feature that everyone could use with just a couple of clicks. We introduced hundreds of new features (https://community.logos.com/forums/t/92701.aspx) knowing that each one had a different audience, and not expecting every single upgrading customer to master them all.
That said, we recognise that it's not ideal that the software has advanced capabilities that many people find too cumbersome to use. So we use your (and others') feedback to drive the priority of UI enhancements to put the full power of those features in your hands. However, it might take us nine months to implement a "search builder" UI that exposes all the new Logos 6 search features in a user-friendly way. What if we had waited to ship Milestone Search until that was done, and then it turned out that no one really cared about that feature? We could have been using those 9 months to build something more useful. So we don't regret shipping an arcane UI for Milestone Search for "power users"; by doing that, we found out that it really is a feature that most customers want to use, and made the functionality available right now for those who want it.
0 -
Since a lot of Logos developers are active in this thread, I'll post this question here: is the forum the best way for us to report bugs/problems? On the one hand, it's great because other Logos users can interact and perhaps help us solve the problem more quickly. On the other hand, is it effective to post about a real bug here and just hope & pray it catches the eye of the right person? (At this point I'm thinking of the very tepid response repeated reports of longstanding bugs are receiving in the Android forum.) Is there a better way to do it, like an email or bug reporting tool?
0 -
Since a lot of Logos developers are active in this thread, I'll post this question here: is the forum the best way for us to report bugs/problems? On the one hand, it's great because other Logos users can interact and perhaps help us solve the problem more quickly. On the other hand, is it effective to post about a real bug here and just hope & pray it catches the eye of the right person? (At this point I'm thinking of the very tepid response repeated reports of longstanding bugs are receiving in the Android forum.) Is there a better way to do it, like an email or bug reporting tool?
The forums are the best place to report bugs. We have explored other ideas, like the respond section in the Factbook. We will continue to evaluate new options and will certainly let everyone in the forums know if that changes.
0 -
Do you have some suggestions
(Thursday morning @ the Faithlife café)
Francis: Hi, MJ, how are you doing?
MJ: Great! Just grabbing a quick coffee. Say, have you heard of the new cool feature in Logos? Now we can do milestones searches!
Francis: Really? What's that? How does it work?
MJ: I must run so I can't tell you much about it right now. What don't you look it up?
Francis: OK! I will!!!
Francis gets home. Boy, oh, boy, oh, boy! MJ seemed really excited about that one, must be a really cool and useful feature! So, let's find out how to use it. Open the help file, type "milestone". Alright, let's see, several sections: milestones, personal books syntax, how-to, visual filters and personal books.
I know milestones have been used in personal books but that's not new, I think I read somewhere that milestone search was for all resources. So, let's start with the "Milestones" entry. OK, a definition of milestones. Nothing about milestones search. Alright, let's try "personal books syntax" then. Let's see, a couple examples of how to put milestones in pbb, page numbers, greek strong numbers, day of the year or Bible reference. I'll have to remember that it's dayofyear not dayoftheyear! I guess 12.25 must be the 25th of december. Right, under headwords, there is one more example. OK, I wished there was actual explanation and illustration on how to use search in relation to milestones, but I suppose I can try to search for one of the examples. Let's try searching the example mentioned under headwords, [[@Headword:Grace]]. I suppose that should be a basic search and I will do "all text" and "all resources". Okaaaay. nothing! Really? With all my resources? Let's see "Lookup grace". Yes! I knew it!
Well, that was not it. Then what? "How-to"? Sounds good. Rats, it's leads me back to the definition section I saw before. Visual filters? Only tells me some filters apply in resources with milestones: the search continues. Last option! "Personal Books". No! It circles me back to the same entries!
MJJJJJJJJJJJJJ! Where are you? Help meeeee!
OK, I know! It says "Help on searching" in the search window. I'll try that. Aaw! Another dead-end! Wiki? Here it is! "Search Extensions / Milestones". Oh! Oh! The section must be in the process of edition! There is nothing there.
OK, forums! Let's see community.logos.com, I will try specific first in the search box, "milestones search". Man alive! All these hits! None of them on the first page is on a thread dedicated to searching milestones, just passing references to my terms in other threads!
Oh no! My class is in 10 minutes. Gotta go!
-------------------------------------------
I hope that this "dramatized" description gets something across.
Now, let's imagine the alternate scenario, actually quite simple.
Francis gets home. Boy, oh, boy, oh, boy! MJ seemed really excited about that one, must be a really cool and useful feature! So, let's find out how to use it. Open the help file, type "milestone". Alright, let's see, several sections: milestones, personal books syntax, how-to, visual filters and personal books, milestones search here it is!!!
Let's see: definition, good. Search syntax, helpful. Wow! These examples are fascinating! I could really use this! I really love that link to the datatype sections. Oh man! So many datatypes. Good thing that there is index to tell me what to use depending on which edition of Josephus I search, otherwise I would never remember. Look at that! I can even use the context-menu and reference box to help me. Cool! Let's do it. It works!!! Yay! Email to MJ: Hi MJ, I tried the new feature you told me about, it's awesome!
---------------------------------------------
Hopefully it also answers what you wrote Bradley. It's not that the feature is so advanced that people can't use it. I am a postgraduate researcher in biblical studies and you tell me that not using that feature is fine! That's not the right answer or approach. Advanced does not have to be arcane. But the problem clearly is documentation here, which should be an integral part of releasing a product.
It's not that I cannot figure out how to use it. It's that I don't have the time to do sleuth work of the kind described above nor should I or anyone have to if documentation was done right. This is in essence the problem: users should be given clear instructions and reference aids to use the tools, not have to figure things out by themselves and with such convoluted processes.
The solution to the problem is not as complicated as you make it sound (at least in this case).
0
