I appreciate all the feedback about quality of both code and data in recent forum threads. You are right, and we are working on it.
We've been through many major release cycles, and every one involves a very similar pattern:
0. Faithlife teams secretly works on new product with previously un-imagined features and data sets we invented. We quietly test ideas on each other and some customers, and come up with a plan and release date. The release date is chosen long in advance and (now that we're big and complicated) becomes set in stone, because hundreds of people, data, licenses, contracts, vacation schedules, baby births, and big-budget expenses are all coordinated against it.
The team works super-hard, has to cut some favored features and content, and sometimes has to choose to ship something incomplete for future delivery (new Atlas maps) or to cut the feature, because the date is set. The team also has to build many features in an environment with limited user-feedback. Sometimes the data set is being shaped by the act of creating it (cultural concepts) -- some of these things are so new we don't know what they'll be like and how they'll work when we're done.
1. The surprise release on the set date.
2(a). Joy, enthusiasm, excitement over new features (25% of regular users).
2(b). We hear that "I never buy point-zero releases -- they're always buggy" and "it's too soon -- I can't afford an upgrade yet" and others are just not paying attention / are in no hurry. (75% of regular users.)
[update with fixes and improvements ships]
3. Another 25% start upgrading while the first 25% send in 'bug reports'. Many of these are good, useful, and 'real bugs'. Some are clearly our fault (bad coding, rushed release, inattentive editing). Others are good, useful, 'real bugs', and the fault of the latest Microsoft / Apple OS release. (Too often our release coincides with one of theirs. Probably because we all like the same pre-Christmas release dates.) Others are good, useful feedback that isn't really a bug, it's a misunderstanding of scope or intention. This feedback helps us improve descriptions, documentation, or even change a feature or data set.
[update with fixes and improvements ships]
4. Strategic-disagreement ensues. :-) "Why would you release something so buggy / unfinished / poorly-document / unexplained / not-what-I-expected-at-this-moment." [tiny fix ships] "You should have waited a year till it was right." [tiny fix ships] "I don't want an upgrade every two years, I want an upgrade every three / four / ten years." [tiny fix ships]
5. Bigger update (the 'point-one' release) ships. Much happiness from many users. More users upgrade.
6. GOTO 4 (and repeat for years until next release).
<smile>
Releases follow a cycle. After a major release like Logos 6, the cycle turns from 'create new things, think bold ideas, and ship them!' to 'fix bugs, respond to complaints, address pain points, improve performance, re-tag the book that nobody paid attention to until the new bundle included it or a new feature made it more prominent', etc.
So the good news is, we're in the maintenance phase now. We've had multiple meetings and internal email threads, and performance, bug fixes, interface improvements, documentation, and resource maintenance (re-tagging with new data types, label markup, data sets, etc.) are top priorities.
Logos 6 has been out for less than 90 days, and we've had many service releases and improvements, and Logos 6.1 went into beta today. (https://wiki.logos.com/Logos_6.1_Beta_1)
Everything is getting more complicated. Sometimes a bug is a bug. Sometimes a bug is a financial or logistical constraint that forces a difficult choice.
Examples:
- We implemented a new Atlas for Logos 6. It works differently and offers different value than the previous implementation, which we largely left in place. We planned it, wrote the code, and wrote the marketing copy over many months, while the maps (250 planned) were slowly created in parallel. There was a lot of prep work on the system, the process, and the background maps that are beneath every thematic map. We couldn't make any thematic maps until the background was right, and that took a lot longer than anticipated (lots of reasons), and we realized we wouldn't have all the maps done by the release date. Question: Do we ship what we have, knowing we can ship new maps at a rate of several a week (roughly) until they're all delivered, or do we pull the feature and just not have a new Atlas feature in Logos 6? It would actually take more work to reverse the new Atlas, change the marketing, etc., but we probably would sell just as many upgrades / earn as much revenue without this one feature.... We decided to ship it and deliver more maps after release. (That's ongoing.) Is this a bug, or a feature we're delivering over time that you're glad to have when it's ready? Or should we have stopped the whole process and delayed launch by a few months to let Atlas catch up?
- LCV, Cultural Concepts (and other data sets) involve manually reading books in our system and tagging them with a new ontology of our own design. We rank books by number of users and importance as key reference works, and then we come up with a list of X books to tag in the first release, and then a budget of how much time (=money) to allocate to continuing down the priority list in future years, even though there's no (direct) new revenue to updating those old books. We have 45,000 books; LCV is on dozens, and Cultural Concepts is on about two dozen of four-dozen key books we have identified. (Though it could arguably be useful on even more.) Question: Is it a bug that Cultural Concepts aren't applied to Pliny's Letters yet? Should we have held the data set for a future release? We decided to ship with the books that were tagged, and to establish a budget for ongoing tagging that works down the priority list, and we insert books into that list based on user feedback.
- Publishers send pre-made EPUB books to us for Vyrso. We never touch a physical book, there are often no page numbers in these EPUBs, and many sell 0 or 1 units per year, on which we make as little as one dollar. We can't afford to page number these texts. Question: Should we not sell them? ('Not high enough quality!') We decided to offer them because many people want the content, and we want to let them get that one book they need in a Logos-compatible format. But we label it an 'Ebook Edition' to distinguish it from our higher-quality 'Logos Edition'.
We can't make everyone happy. Somebody wants the newest stuff now, whatever shape it's in. Somebody doesn't care about Pliny's Letters, and finds Cultural Concepts tagging useful even if only applied to the Bible. Someone else doesn't want anything until it's rock solid, bug-free, and comprehensive. Someone else doesn't want a single tag applied to the Bible or a database including Bible references unless every book in every Christian canon in all of church history is included on equal footing. :-)
Our planned solution.
1. More communication. (It seems like that's the solution to most problems...)
A. Our Content Production team allocates 15% of their effort to maintenance on existing resources. This year it turns out it was more like 12%, with all the work on Logos 6 material. They'll get it back to 15%, and we'll go as high as 20% if it's necessary to keep you (collectively) happy. (Beyond 20%, for just revising things we already shipped and sold, feels like it might be financially difficult for us.) The team is also planning to post more often, and to be clearer about what they're doing, so all their hard word doesn't just silently download in the night.
B. Our Content Innovation department is working on better documentation of our data sets. We plan to add a Library entry for each data set, even if it's an 'invisible' data set that's exposed through features or other resource panels. This will give us a place to provide an Information Pane on the data set, and we'll explain how we created it, who created it, the process, and discuss strengths/weaknesses, as appropriate. We're looking into ways to share 'what's been tagged, and what's next' priority lists, too.
2. Shorter cycles. (This is the new part.)
We're not going to retreat to three or four year release cycles. We believe it's a false conceit that simply taking more time will fix everything. Some quality issues don't show up until thousands of people test thousands of combinations of hardware, network connection, and other software installs. Some quality issues are differences of opinion, and we need to hear the other opinions to know a change needs to be made.
In a world where the iPhone is updated annually and that feels (to some of us) like not often enough, where Google Chrome silently updates every six weeks, and where Facebook and other web sites change behavior, layout, and functionality every single day, it's just weird for an increasingly-connected product to go three years without any significant changes.
A shorter cycle means new ideas get feedback faster. A shorter cycle means bad ideas don't waste as many resources before being abandoned. And a shorter cycle lets us serve everyone. If you don't like shorter cycles, you can just skip them. Make your own 'long cycle' by opting out of the short-cycle offering for one, two, or three years.
This is a conceptual overview, not a product announcement.
It's too early to get bogged down with configuration and pricing details. But I hope I can help with examples, possibilities, and reassurances:
- We're considering subscription content. We license Proclaim by subscription, and offer Pro Media by subscription: each month is just gets better, as we release new content every month. Logos Bible Software could benefit from the same model, especially with our online content and 'never ending' data sets. We could make 10 more Interactives (like Psalms Explorer, Feasts and Sacrifices, etc.) and sit on them for two years until they're 'monetized' in a Logos 7 release. We could create 20 more sets of Teaching Slides and Teaching Videos (deSilva) and included them in that big release. But in reality these are built one at a time, over many months. Wouldn't some users rather subscribe and get new content every month rather than have that new content 'sit in a drawer' for two years, helping no one, waiting for the big bundle release?
- This isn't a money-grab. Don't confuse shorter-cycles, or subscription content, with a money-grab. We aren't trying to get the 'every few years' upgrade revenue from you every month. A shorter cycle will involve a smaller release and it will be priced appropriately. A subscription product goes on indefinitely and would be priced appropriately. Shorter cycles can just lead to 'smoothing' the funding, not necessarily increasing it. And this can be helpful to both the customer and the producer.
- We know some people have workflows that don't adapt to frequent change. Some users have spent a lot of time figuring out how to do their study / sermon-prep, etc. They know what books they use, in what layouts, and exactly what keys to hit and buttons to click, and they may even use cheat-sheets from MP Seminars or other guides they've studied carefully. They don't want things to move and mess them up. We understand this, and intend to support the 'don't move anything for a few years!' users.
- There are renters and there are owners. Some people stream all their music and all their movies. Some people lovingly catalog their audio CDs and buy their favorite movies and shows in boxed sets of DVDs. There are advantages and disadvantages to each model. We get it, and we want to support both models.
- Shorter cycles improve transparency. In a short-cycle model, there won't be years of hidden work. Even if you choose to only upgrade / add-to-your-library every couple years, you'll know what we're doing, what features we've been adding and improving, and what content we have been creating. You'll even be able to speak into that process and make suggestions on what to do and what not to do.
- We know that offline access (on both mobile and laptops) is important. While we will certainly create more features that need online access -- for reasons of logistics (often) and physics (sometimes) -- we know that many of you need offline use of your Bible study tools and we will keep it a top priority.
We know that all of our users are interested in a solid, fast, quality product. And we know that most of them are also interested (enough to pay -- which feeds us!) in new tools and content. It's a perpetual balancing act, and one where we're off balance to someone all the time. I believe that shorter cycles and more communication can help address both, leading to a better product that is even more in tune with what you want. (While still delivering the occasional delightful surprise you didn't know you wanted until you saw it!)
What do you think?
-- Bob