Quality control in Logos 6 - a new perspective

Page 2 of 9 (175 items) < Previous 1 2 3 4 5 Next > ... Last »
This post has 174 Replies | 12 Followers

Posts 856
Michael McLane | Forum Activity | Replied: Thu, Jan 8 2015 9:59 AM

Todd Phillips:
I do understand the need to expand and broaden when you see that you may be about to saturate your market. 

I have gotten the impression that they are a company that often reacts and operates out of fear. I am probably wrong.

Looking at my context, there is absolutely no saturation. Out of six pastors in my local group, I am the only one who uses Logos. Another chose Accordance the other do not use anything. And of the community I serve, no one uses bible software. There is plenty of room for growth in my context.

However, as I have said before, I cannot recommend Logos or any FaithLife products because of the quality issues. I would love to be able to "sell" Logos to the people I serve. But, I can see it coming back to embarrass me and having to spend too much time supporting them. (Not to mention that it is not user friendly and has no reference manual. I have been endeavoring to understand the new Milestone search feature. Thank goodness for the community here for helping. But surely there has to be a way to make that into a menu item that is more accessible to someone without mastering search syntax strings. Just my experience.)

Do I trust Logos? As a reader, yes. But, I have gotten into the habit of verifying its result otherwise, or doing them the hard way because it is just as easy. So, maybe the answer is, no. I buy a lot of resources, but I have been contemplating their value since the release of 6.

Like Francis, I want Logos to succeed and grow.

EDIT: On further reflection, I would like to make an amendment/confession. I think I was quick to answer due to many of my resent frustrations with the software, particularly since the 6 release. I think it was unfair of me to insinuate that I do not trust Logos (or to outright say it). I do not know enough about its search perimeters, abilities, etc. on the datasets to even begin to form an opinion, which is what the OP was referring to, I think. My concern for QC centers more around the reliability of the program itself to run, to do its job timely, the UI etc., which I hope came out a bit.

Thank you John Fidel (below) for your perspective and my apologies to the community. I am trying to be more positive towards FaithLife in the forums, yet maintain the truth of what I see that they may improve. I particularly want to be sensitive to those that do the hard work at FaithLife in producing the software and resources. I know they want to do a great job and it is a complex, never-ending job. My apologies to you as well if I have seemed overly harsh. As a pastor, most days we hear more criticisms than anything. I forget that others deal with that as well. Thanks also to Sean for his words.

So, in terms of the quality of the "programmed" application and services, I think there is much room for improvement (hence my difficulty to recommend it. Though maybe I will give it a shot). I know it will get better. Though, I think I would have rather waited for a better product.

I do love Logos. I am in it many hours a day. Even after all these years it is mostly for reading. I think one of the frustrating things is the steep learning curve to really get at the power.

Michael

Posts 8967
RIP
Matthew C Jones | Forum Activity | Replied: Thu, Jan 8 2015 10:07 AM

Francis:
My rhetorical question (about the poll) has been turned into a pointless confrontation...

I guess if everyone answered "No" there would be no confrontation. Declaring a poll pointless is one way of retreating from your rhetoric.

Logos 7 Collectors Edition

Posts 2336
Forum MVP
John Fidel | Forum Activity | Replied: Thu, Jan 8 2015 10:11 AM

Content deleted

Posts 813
LogosEmployee
Sean Boisen | Forum Activity | Replied: Thu, Jan 8 2015 11:49 AM

I lead the Content Innovation (CI) team: we're primarily responsible for the data sets and linguistic tagging on the biblical text that ships with Logos. So it matters a great deal to me that we produce the best value we can for our users in these areas, and it gets my attention when people raise questions about quality control. I can only speak to the areas I'm responsible for (which don't include the translation project the OP worked on).

Francis:

For instance, searches of cultural concepts or other tagged entities shows a lot of gaps and inconsistencies in tagging. This is a quality control problem. 

I don't know how to respond to this without more specifics. But let me provide some context on the Cultural Concepts project from CI's side, since ultimately I'm the one who's responsible for it. I'm extremely proud of the work our team did here:

  • We created a hierarchical categorization of more than 1100 cultural concepts that are relevant to the biblical text (the Lexham Cultural Ontology, or LCO).
  • We annotated both the Bible and about 15 other ancient text resources with these concepts: I estimate about 5 million words. Note there's no automated process: our team had to read each text and analyze it from the perspective of the LCO.
  • We linked concepts to key dictionary articles explaining their background.
  • We coordinated the LCO concepts with our Topic Guide and Bible Sense Lexicon data to "connect the dots" with other Logos data sets

Every data set project we undertake has to find the right middle ground between benefit to users (things like search, organization, and ultimately insight) and cost to Logos. The primary benefits we imagined for the Cultural Concepts are:

  • Providing easier access to cultural background information for the biblical text
  • Making it possible to find secondary literature on the same concepts (If you browse the Cultural Concepts section in Factbook for Covenant, you'll find a lot of material you'd never find by searching for the word "covenant")
  • Organizing the cultural concepts themselves (Covenant is a sub-concept under Contracts and browsing the glossary may help with exploring the cultural background of the Bible)

Our users are ultimately the ones who have to judge whether we delivered these benefits or not: from my perspective, this is not only a successful feature in our product, but a novel and significant contribution to the field of biblical studies as a whole.

As for cost, we've invested roughly 20 person months of effort in building this data set and annotating texts (we even hired three interns this past summer to help get all the annotation done for Logos 6). All of the people doing this work had graduate level training and consulted existing academic resources in their work.

I've gone into the details on this one project (we have many, of course) to try and provide some perspective on our process. Is this data set perfect? No, and it can't be: there's no objective standard to compare it to, and nobody's ever done anything quite like this before. Is the annotation "done"? Again, no: we prioritized our annotation for the resources we thought would be useful to the most Logos users, but there are dozens more resources we could annotate. Deciding how many more we ultimately do will be determined by trading off benefit and cost, for this project and others. Is the ontology complete or comprehensive? Not really: while we did our best to cover the most important biblical cultural concepts, any model can always be enriched and extended.

Instead, the criteria we typically use to assess data projects like the LCO include

  1. Utility: does it help users accomplish their goals?
  2. Consistency: have we been consistent throughout in our annotation? Are similar passages annotated the same way?
  3. Validity: would external observers who understood the issues be in substantial agreement with our judgments?
  4. Sufficiency: have we provided enough additional data to make the whole useful?

These are all significant challenges, and (like the original analysis itself) none of these criteria can be automatically checked (though we do check as many things as we reasonably can). But I stand by my claims that we did our best to meet these criteria given the effort we invested, and overall, we've provided a great deal of value with this data set.

If you find errors in this or other CI data sets, please email data@logos.com or post on the forums with the specifics, and we'll do our best to fix it.

PS: David Witthoff, who was the project lead for the LCO, has a series of helpful posts on his personal blog about using the LCO for sermon preparation, starting with Logos 6 and Sermon Method.

Posts 492
Mr. Simple | Forum Activity | Replied: Thu, Jan 8 2015 11:59 AM

Thanks  Sean - Enjoyed your post and learned something in the process :)

Posts 3770
Francis | Forum Activity | Replied: Thu, Jan 8 2015 12:30 PM

Hello Sean,

Thank you for your reply. I appreciate the information it provides and the time you took to write it.

I understand your request for specifics, but I cannot be too specific here without taking too much time to go back in history and trace back all the various issues that have arisen. But it has been an accumulation. I must say, and I understand that this is not your department, that I was particularly disillusioned by the other project I alluded to. However, I do not want to say more about it here because I don't want to start mentioning specific people or give enough information that it would point to specific people. This is not my goal. My goal is to call for improvement, not to burn someone at the stake.

As an instance, a couple days ago there was this thread about looking for prophecies and/or feasts in the NT, when avenues were found that were thought to be able to produce the expected results, hits were missing because they were tagged differently. For instance, Passover in John was not tagged as a religious feast, but as Passover. Someone who would look for mentions of feasts in the NT and used that cultural concept search would miss a number of hits. This may be a matter of choice, though the choices and the underlying logic is often not obvious and this is the kind of thing for which it is difficult to find documentation. 

In another thread -- we're leaving cultural concepts now -- we were talking about searching labels and tags. Again, lots of problems there. 

These are the two most recent examples on my mind, but there are others who even in this thread have echoed my experience and perhaps can remember other specific examples. I would think that beyond that, there are very many expressions of frustration about features or data not being complete. Perhaps I am venturing too far (and I am willing to be corrected) but I think that those I hear from who do work that require accuracy generally know that they cannot fully rely on results. 

Features are also delivered incomplete, for instance label searches in annotations or user document searches, or narrative outlines. Resources come out untagged, etc.

I appreciate your answer then and certainly I do not mean this as an attack on you and your team, but I really don't know why there are so many errors all over the software, such a lack of proper documentation and so much incompleteness. I spend way too much time looking for solutions to fix or circumvent these problems, instead of just getting the work done!

This is not to say that I do not value what does work. When it works and is accurate/dependance or complete, it's great. 

Posts 226
Tim | Forum Activity | Replied: Thu, Jan 8 2015 1:36 PM

First I owe Sean a HUGE thanks. You interviewed me a couple years ago when I applied to work with Faithlife (then Logos) as a Lexical Data Curator. You did not hire me, and it is for that I offer a heartfelt thanks. I was called a couple weeks later as a pastor in a wonderful church! The Lord is good!

Now to the topic at hand. I think that perhaps the question of whether one "trusts" Logos/Faithlife is a bit of a misdirection. The Logos Library system has always been just that: a way of collecting and accessing published material in a digital format. Over the last 15 years I have seen huge changes in how that information is accessed, but that has not changed the basic fact that as a library system Logos is in my opinion, unmatched. 

The extent to which one chooses to accept/believe/trust the content stored and accessed in the Logos library should always be a matter of wisdom and discretion. As I read different authors, some I develop a respect for and have an increased receptivity to what they have to teach, others I remain more skeptical of if I find their methods or conclusion faulty or lacking.

When it comes to the content that Faithlife has included in their library system I think we need to use the same wisdom and discretion when considering the quality of information they have gathered and published that we do with other content producers/authors. However I think it is important to maintain the distinction between the library system we use to access the library and the production of content itself.

Yet here is where the content from Faithlife is different from any one of the thousands of books and journals I have: it is dynamic. Where there are errors, they can edit and update it where those errors are pointed out or as they become more experienced and identify their own shortcomings. With electronic versions of a "normal" print library, we have to wait for years as the authors themselves grow and change and then write the second edition, third edition, etc. and then be published. With Faithlife as a content producer, they have the ability to make that change and push it out (publish it) tomorrow! What a blessing!

 If you are wary or disagree with the content that Faithlife has included in their library system, certainly don't use it. Make use of the resources you are confident in and do trust. However, does disliking Faithlife's content justify abandoning the whole library system? I personally do not think so. 

With regards to the library system (how we access the data) certainly tagging is an important part of how data is found and accessed. It seems though that new features provide additional access to existing data so that we have different ways to dig deeper into what we already had. I haven't found anything yet that keeps me from studying and finding the information in the ways I have grown accustomed to. We can hope that Faithlife will continue to expand on what they have already done so that datasets are more complete, but again this does not cripple study, it simply limits the usefulness and viability of the new features until those features are more fully developed. 

Posts 3770
Francis | Forum Activity | Replied: Thu, Jan 8 2015 2:03 PM

Tim:

If you are wary or disagree with the content that Faithlife has included in their library system, certainly don't use it. Make use of the resources you are confident in and do trust. However, does disliking Faithlife's content justify abandoning the whole library system? I personally do not think so. 

With regards to the library system (how we access the data) certainly tagging is an important part of how data is found and accessed. It seems though that new features provide additional access to existing data so that we have different ways to dig deeper into what we already had. I haven't found anything yet that keeps me from studying and finding the information in the ways I have grown accustomed to. We can hope that Faithlife will continue to expand on what they have already done so that datasets are more complete, but again this does not cripple study, it simply limits the usefulness and viability of the new features until those features are more fully developed.

Thank you for your thoughts Tim. I appreciate your desire to be gracious and provide perspective, but I must say I totally disagree with you here.

That your study needs or habits may be such that you can function the way you describe, I can envision. But Logos is now also a product for academics. We do a lot of detailed research and whatever we claim must be accurate. Keeping that in mind, new releases of Logos have each come with a set of promises. What I mean by promises is that features are marketed as reasons to upgrade and they tell you the beauty of all you are going to be able to do. You buy into it (literally) and little by little discover that features are incomplete. You think it's just bugs at the beginning, but then users who have been here since Logos 4 can tell you that while many problems have been addressed, many have also dragged on a long time. Tagging is a sore point. But in addition to this, you have to keep in mind that long-term users have invested a lot in Logos. There is the cost of resources, of all the upgrades but also the investment represented by trusting it with your study and data (all your annotations, all your markings, etc). Over the years, it's huge. The longer you have stuck with the company and the more this dynamic of incompleteness, promises lagging pattern continues, the greater the feeling of being betrayed with regards to the expectations that were fed by the promises and marketing messages and which you have paid for. 

So yes, trust is an issue because I have trusted Logos during all these years, but I do feel that there has been a worsening of that dynamic (more stuff coming out more quickly yet incomplete). I was actually a bit shocked by the timing of the "what do you want to see in Logos 7?" question. I don't think I would be the only who, if things do not change, would take whatever marketing messages and promises are made when Logos 7 comes out with a degree of circumspection and if features of Logos 6 are not fixed by then, with a strong dose of cynicism. 

Posts 11070
Denise | Forum Activity | Replied: Thu, Jan 8 2015 2:13 PM

Well, Sean, I took you up on your PS offer.  I thought I'd get some free training from the horse's mouth:  David Witthof

I did what he said to do, and nothing happened.   He warned me not to type in 1 Samuel or 1st Samuel, so I didn't.  I carefully copied what he said to type in: First Samuel.  My Factbook refused to budge from Moriah (an earlier choice).

Then I looked at what his example actually delivered: 'Book of First Samuel'

Well, there you go!  The trick is to not type the identification of the book ... Samuel, Chronicles, and so forth.  I felt truly trained.  So for my 'finals', I typed 'Peter' (for 1st Peter).  I got another 'F'. 

Yes, it didn't work and yes, I'm happy you guys worked so hard on WBC.  

"I didn't know God made honky tonk angels."

Posts 813
LogosEmployee
Sean Boisen | Forum Activity | Replied: Thu, Jan 8 2015 2:14 PM

Francis:

As an instance, a couple days ago there was this thread about looking for prophecies and/or feasts in the NT, when avenues were found that were thought to be able to produce the expected results, hits were missing because they were tagged differently. For instance, Passover in John was not tagged as a religious feast, but as Passover. Someone who would look for mentions of feasts in the NT and used that cultural concept search would miss a number of hits. This may be a matter of choice, though the choices and the underlying logic is often not obvious and this is the kind of thing for which it is difficult to find documentation. 

Looks like this is the post you mean? I agree there's a data issue here with the ontology: we've categorized Passover as a kind of Organized ceremonial (which is true), but it's also a kind of Religious feast, and our data doesn't capture that. I'll add a case to correct that.

Examples like these are very helpful because they illustrate ways people use our data that we couldn't easily anticipate. As Mark Barnes noted in that thread (for a different case), errors and definitional problems aren't necessarily the same thing. Any time we make some analysis, we're necessarily joining or splitting things that others might not join or split the same way. But (using the criteria from my previous post in this thread) having Passover as a Religious feast would be at least more useful and more valid.

I agree we need to do more to explain the choices and logic behind some of our datasets: creating those introductions is on my list for this year.

Francis:

Features are also delivered incomplete, for instance label searches in annotations or user document searches, or narrative outlines. Resources come out untagged, etc.

We've made a fundamental business decision: not all data sets need to be complete to be useful. Your example of Narrative Character Maps is apt: we could withhold this data until we have character maps for every narrative book in the Bible. But in the meantime, if you're studying the first part of Acts, you might find this character map useful today (and we're actively working on the rest of Acts). We may decide some books aren't worth the effort for this kind of visualization (by contrast, we deliberately decided that we wanted our Biblical People Diagrams to cover every named individual in the Bible, even one of the 30 Zechariah's that's only mentioned once in the text). Some resources get a dozen or more different kinds of markup in our production process: more markup means more production cost, so others get less. These are business and editorial decisions: they're not quality control issues.

Posts 813
LogosEmployee
Sean Boisen | Forum Activity | Replied: Thu, Jan 8 2015 2:35 PM

Denise:

Well, Sean, I took you up on your PS offer.  I thought I'd get some free training from the horse's mouth:  David Witthof

I did what he said to do, and nothing happened.   He warned me not to type in 1 Samuel or 1st Samuel, so I didn't.  I carefully copied what he said to type in: First Samuel.  My Factbook refused to budge from Moriah (an earlier choice).

Then I looked at what his example actually delivered: 'Book of First Samuel'

Well, there you go!  The trick is to not type the identification of the book ... Samuel, Chronicles, and so forth.  I felt truly trained.  So for my 'finals', I typed 'Peter' (for 1st Peter).  I got another 'F'. 

Yes, it didn't work and yes, I'm happy you guys worked so hard on WBC.  

We have a known problem with looking up Bible Book in Factbook (previously noted here, here. here and elsewhere, no doubt). We've got a much longer list of candidate articles for Factbook than previous guides, so we've still got some work to do in sorting and ranking the alternatives. Clearly Bible books shouldn't be buried down at the end of the list.

The best current workaround I know of is to use the Topic type to start searches for Bible books. So the results for "<Topic peter" includes First Epistle of Peter, though it's still much farther down the list than I'd like.

Posts 29375
Forum MVP
MJ. Smith | Forum Activity | Replied: Thu, Jan 8 2015 2:35 PM

Like the OP I have a bit of experience with working with Faithlife on a specific product -- much smaller and much less important in the overall scheme of things. However, I also experienced a pre-determined timeline resulting in a release with embarrassing errors. Unlike him, I also had a basic timeframe in which corrections would be made and knew that the time frame was short partially by a redesign to get it right ... and the redesign was brilliant even if the data in some cases was flawed.

I think there are three separate issues in the OP's original concern:

  1. First, there is the tagging that primarily assists the average user find things in their library and is unlikely to be used in an academic sense that requires 100% accuracy. Here, waiting for additional tagging and a certain margin of error is permissible. What is missing is an easy way to tell what one needs to check manually because it is not tagged - this drives me crazy.
  2. Second, there is the tagging that is apt to be used in an academic setting - primarily the tagging behind the Bible, clause, morphological and syntax searches. Here the tagging needs to be complete and very accurate - and the definitions of how things are tagged must be explicit and accurate. Here I see many of the complaints in the forums based on an incorrect assumption that linguistics is a precise science so they fail to understand why different resources give different results. Here I am often frustrated that Faithlife has not gone with a specific linguistic theory but invented their own (fine to the point) without documenting precisely what their theory is (that's the rub). The release of glossaries with L6 was a step in the right direction but ...
  3. Third, there is the tagging where Faithlife is truly breaking new ground e.g. the Bible Sense Lexicon. Here I think we need to cut them some slack because it really takes thousands of knowledgeable users using it and providing feedback to "get it right". If, over the next decade we don't see gradual improvement in the usefulness, then we have reason to complain.

We also have to remember that with items like the Factbook, Faithlife runs into a serious problem with differences in vocabulary across different schools of theology. In topics.logos.com they had created the opportunity for users to supply synonyms but in general they got little feedback. We, as users, need to hold Faithlife's "feet to the fire" in terms of the order in which tagging is done.

  1. No matter how much your library is limited to one tradition, we need to demand that a wide swath of denominations be included in the early tagging e.g. Orthodox and Jewish terminology is sadly missing from the LCV. I would not be surprised is other groups such as Anabaptists, Restorationists, Quakers ... also find their "native" terminology missing.
  2. If your work requires completeness or if your personal favorites are not tagged (think outlines, journals, sermons ...) you need to push for the ability to have a collection of "not yet tagged" that require different search arguments or manual searching.
  3. When you find errors in tagging, missing tagging et. al. you have an obligation to report it. Faithlife cannot identify and correct the errors that annoy us if we don't report them.
  4. If you don't understand the purpose or meaning of Logos tagging (recent examples: BSL, propositional outlines) keep asking questions on the forum. Eventually enough people understand to be able to answer the questions or Faithlife discovers they need to write an introduction or blogs (think of the Discourse Analysis material we now have).
  5. If you feel the tagging is "too academic" and you need "pastoral/teaching" tools, be precise in what you want: For me it would be tagging of rhetorical devices, microgenres and named relationships between verses (cross-reference and the myriad of other verse lists we have in Logos/Verbum).

Remember that software development is a two way street - the Faithlife software environment will continue to improve only if the users let Faithlife know what does not work, what can be improved and what is needed. That end is our responsibility. Implementing it and broadening our vision is Faithlife's responsibility. Whining is simply poor stewardship of time.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 8550
LogosEmployee

Sean Boisen:
We have a known problem with looking up Bible Book in Factbook

... and it should be fixed in 6.0b (which is shipping soon).

Posts 226
Tim | Forum Activity | Replied: Thu, Jan 8 2015 4:01 PM

@MJ - Thank you for the detailed clarifying response. That was well thought out and well said. 

Posts 25825
Forum MVP
Dave Hooton | Forum Activity | Replied: Thu, Jan 8 2015 4:20 PM

Francis:
I think that quality control is the leading issue with the ongoing development of Logos.

That has been a concern of mine for some years and particularly with software for Logos 5.1+, Logos 6.0 Beta and 6.0a, 6.0b betas. Complex datasets have always been a concern starting (for me) with OpenText on Logos 3, then Clause Search, BSL, People/Place/Thing tagging, and Referent tagging in Logos 5 followed by the new datasets in Logos 6. OpenText was so buggy it probably drove Logos down the path of doing its own datasets and being able to manage both development and maintenance (parallel to managing their own bibles). LCV (topics) in Logos 4 was the signal for Faithlife wanting to manage data for a better experience in searching. Now we see LCO (Cultural Concepts), Literary Typing, Semantic Roles and Case Frames which are quite complex in the way they are applied to resource text.

The question of trust arises in the design and intended application of datasets. Whilst quality control (QC) is important in their management, most errors are due to human error/interpretation in applying the (bible-based) data. Do I trust Clause Search -- No! Do I use it - Yes. Would I recommend it - No.

The QC of the software and the direction of development showed a marked down turn in latter part of Logos 5. The Logos 6.0 beta showed that Faithlife could not keep up with the bug reports, or wanted developers to interpret what was important to them. The lack of response in the forums and the silly errors in the 6.0a/6.0b betas is indicative.

Dave
===

Windows 10 & Android 8

Posts 2588
Lee | Forum Activity | Replied: Thu, Jan 8 2015 4:27 PM

MJ. Smith:

    3. When you find errors in tagging, missing tagging et. al. you have an obligation to report it. Faithlife cannot identify and correct the errors that annoy us if we don't report them.
    4. If you don't understand the purpose or meaning of Logos tagging (recent examples: BSL, propositional outlines) keep asking questions on the forum. Eventually enough people understand to be able to answer the questions or Faithlife discovers they need to write an introduction or blogs (think of the Discourse Analysis material we now have).

Been there, done that, MJ. Some users will keep quiet after a while; some users will make a personal spending decision. Expecting regular users to take up the slack is just too much.

Posts 29375
Forum MVP
MJ. Smith | Forum Activity | Replied: Thu, Jan 8 2015 4:54 PM

Lee:

Been there, done that, MJ. Some users will keep quiet after a while; some users will make a personal spending decision. Expecting regular users to take up the slack is just too much.

Having spent a few decades teaching the "average departmental secretary/reception/payroll clerk/etc." that the only way for me to know how they used the system (vs. how central offices assumed they worked) was for them to tell me, I know the message needs to be hammered home consistently. But until a min-reader function is available for programmers/analysts ... the end user must recognize their role. The alternative is a few thousand well trained beta-users and a year to two or three of more of time (consider Microsoft). Or as in some software tools, only the top few $$ users have their bugs fixed and improvements considered.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 2588
Lee | Forum Activity | Replied: Thu, Jan 8 2015 8:10 PM

What I meant to say was: to expect regular users to constantly give feedback about obvious QA issues, is a dangerous game for any business to play.

Posts 3770
Francis | Forum Activity | Replied: Thu, Jan 8 2015 11:40 PM

Posts 29375
Forum MVP
MJ. Smith | Forum Activity | Replied: Thu, Jan 8 2015 11:49 PM

Francis, I agree that the QA is inadequate. The number of resources that have been released then sent back for basic coding is not acceptable. Examples" the BCP without the Psalm links (Latin names rather than Psalm number), Faith of the Fathers without the link to the index that provides the primary value, the number of liturgy books not tagged to appear in the liturgy section ... those are just ones I complained about. I also agree that the Faithlife fall back on an overall error rate and their lack of time to manually check each resource is a poor excuse. At the start of each resource's development a check list of required flags for processing, indexing noting special features etc. etc. ought to be made and ought to follow the resource and manage the workflow. But I do think we have to have realistic expectations and recognize our own position in the process.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Page 2 of 9 (175 items) < Previous 1 2 3 4 5 Next > ... Last » | RSS