To those unhappy with Logos communication during outage

Page 1 of 5 (82 items) 1 2 3 4 5 Next >
This post has 81 Replies | 6 Followers

Posts 26014
Forum MVP
MJ. Smith | Forum Activity | Posted: Sat, Nov 22 2014 9:22 PM

Having been on the inside of several critical failures, I'd like to give another perspective. For those working on the problem there is nothing as annoying as someone from management interrupting every 10 minutes to learn your current status. As you are identifying a problem anything breaking concentration slows down identifying the problem. So please, until the problem is identified, don't expect more than an hourly "we don't know."

Once the problem is identified and the solution determined, take the first estimate of time until things return to normal that is given by the techies and double it ... in the pressure something is bound to go wrong. The one I remember best is a 4th of July drunk driver taking out a power pole. We'd been discussing adding backup power.

If things are going wrong in off-hours or weekends-holidays, don't expect the techies to have the company's understanding of time zones or remember what form of communications the users may try. Techies tend to be extremely competent in their area of expertize ... and highly upwardly mobile if they can place that in a broader context.

And when they finally identify the problem, have implemented a solution and are willing to give you estimates as to what functions will be available when, half the users are going to read the notice and say "I don't understand what you're saying" while the other half are saying "but are you using xxx or yyy did you consider zzz so it won't happen again." Know which half you fall within, laugh a bit at yourself and give the people working for at least 28 hours on this a break.  Monday morning is just fine for the post mortem and future planning - and by then they should be capable of broaden their perspective a bit. It has been highly focused for many many hours.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 2817
Michael Childs | Forum Activity | Replied: Sat, Nov 22 2014 9:33 PM

My, we are a demanding customer base, aren't we? 

"In all cases, the Church is to be judged by the Scripture, not the Scripture by the Church," John Wesley

Posts 26014
Forum MVP
MJ. Smith | Forum Activity | Replied: Sat, Nov 22 2014 9:42 PM

No worse than Payroll or Hospitals.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 279
Sean McIntyre | Forum Activity | Replied: Sat, Nov 22 2014 9:49 PM

Not asking for a report just 'we are aware of a problem and dealing with it'  this did not happen on Twitter for hours.  Also a status page should not be considered an innovation. 

Posts 623
JAL | Forum Activity | Replied: Sat, Nov 22 2014 10:08 PM

MJ. Smith:
Monday morning is just fine for the post mortem and future planning

Yes

"The Christian mind is the prerequisite of Christian thinking. And Christian thinking is the prerequisite of Christian action." - Harry Blamires, 1963

Posts 2016
GaoLu | Forum Activity | Replied: Sat, Nov 22 2014 10:19 PM

I have never visited the inner sanctum of Logos (or the Outer Sanctum for that matter).  I imagine it is a lot like Charlie and the Chocolate Factory.  Seems like, Logos being all big and professional and all could send a friendly Oompah Loompah up the elevator to assure the outside world that smoke belching from the chimney means, not a melt down, but fresh chocolate!

On the other hand I know how frustrating it can be to explain that things aren't going well when you don't even know yet what isn't going well.  I, for one, just don't want to be a Verruca Salt about this event, know what I mean?  

Thanks Logos for your Hard Work.  As far as I am concerned you can each take time to have a piece of cake and cup of tea.  I'll survive this OK.

Posts 406
Paul C | Forum Activity | Replied: Sat, Nov 22 2014 10:45 PM

Yes, but think how traumatic it could have been if DateLife.Com were up and running ... and a few of the subscribers were experiencing testosterone overload.Stick out tongue

Posts 1809
mike | Forum Activity | Replied: Sat, Nov 22 2014 11:08 PM

I cant imagine for those SDA pastor who had to use their Logos ipad or proclaim to preach today. Mustve been their worst Sunday ever. 

I would have been super pissed if had to preach today and my notes was gone. Standing up there on pulpit like a dummy.

logos is playing dangerously 

Posts 406
Paul C | Forum Activity | Replied: Sat, Nov 22 2014 11:14 PM

 ~

Posts 102
Andrew Mercer | Forum Activity | Replied: Sat, Nov 22 2014 11:14 PM

I'm not SDA but am in Australia. Church finished 4 hours ago, so no Proclaim. We fell back onto ProPresenter and I think we'll stay there. All the time spent building the service structure in Proclaim was wasted

Posts 1070
Lee | Forum Activity | Replied: Sat, Nov 22 2014 11:37 PM

I preached from my notebook, I just worked offline. most everything is on my tablet, and i try to review my sermon and make sure everything is ready if i can't sync to logos. 

we have local internet problems so.. even though I like to keep in sync with the servers its not so big a deal.

I just wait and trust that Logos will be back soon.

L4 Bible Study, L5 Reformed Bronze, & L5 Gold, L6 Platinum & Reformed Platinum, L7 Platinum, L8 Baptist Platinum.
2015 rMBP 15" 2.2GHz 16GB 256GB SSD, running macOS Mojave   iPad Mini 4,   iPhone 6.

Posts 124
Wayne Clarke | Forum Activity | Replied: Sun, Nov 23 2014 12:58 AM

MJ. Smith:

Having been on the inside of several critical failures, I'd like to give another perspective. For those working on the problem there is nothing as annoying as someone from management interrupting every 10 minutes to learn your current status. As you are identifying a problem anything breaking concentration slows down identifying the problem. So please, until the problem is identified, don't expect more than an hourly "we don't know."

Once the problem is identified and the solution determined, take the first estimate of time until things return to normal that is given by the techies and double it ... in the pressure something is bound to go wrong. The one I remember best is a 4th of July drunk driver taking out a power pole. We'd been discussing adding backup power.

If things are going wrong in off-hours or weekends-holidays, don't expect the techies to have the company's understanding of time zones or remember what form of communications the users may try. Techies tend to be extremely competent in their area of expertize ... and highly upwardly mobile if they can place that in a broader context.

And when they finally identify the problem, have implemented a solution and are willing to give you estimates as to what functions will be available when, half the users are going to read the notice and say "I don't understand what you're saying" while the other half are saying "but are you using xxx or yyy did you consider zzz so it won't happen again." Know which half you fall within, laugh a bit at yourself and give the people working for at least 28 hours on this a break.  Monday morning is just fine for the post mortem and future planning - and by then they should be capable of broaden their perspective a bit. It has been highly focused for many many hours.

I am the Director of Systems Infrastructure for one of the top-five global IT companies.  We specialize in business consulting, systems integration, and IT outsourcing services.  We have thousands of clients globally and a vast number of those who rely on us for Data Centre services, server, storage and network management of their business systems.  We ultimately support millions of end-users and customers of our Client-base.

So, similar to MJ, I would like to provide my perspective and it is a little different.

And we sometimes have outages (hopefully no one from my executive management team is reading this).  Failed hardware.  Failed operating systems upgrades.  Bad code.  Corrupted data bases.  Application coding problems.  Network failures with various telecomm providers.  Environmental issues.  Nefarious activities.  Human mistakes.  And on and on and on.  No matter what the cause, keeping your clients posted on the issues and updating them with periodic progress reports is so fundamentally necessary in managing these situations and effectively communicating expectations.

Problems happen.  Experience and internal processes and procedures help companies like Faithlife effectively manage these types of issues.  Having an effective way to deal with them and keeping communication lines open should not be an option.  And I am not talking about communicating every nit-picky detail, but keeping people posted about the general nature of the issue and the expected restoration time is the minimal type of communication that should be happening.

Providing internal and external communication updates is absolutely key to help set expectations and keep people informed.  How many people clicked on a Twitter or Facebook link taking them to the free book offered or pricing special, but only to receive a script/runtime error in their browser?  Or a blank web page that tried to load for a period of time?  Or gateway timeouts?

It took hours before the Faithlife ‘sad little sheep’ was posted to the logos.com site and then it disappeared again and then it came back again.  How many Faithlife customers wasted their time checking to see if they had an internet or PC problem?  How many Faithlife End-Users received a script error in their browser and wondered if they might have picked up a virus.  How many proclaim users were struggling with issues?  How many iPad Bible App users deleted the App and re-installed it because that is often the only way to resolve with iPad Faithlife/Logos Bible App issues.

How many of us Faithlife Users had problems with the website with the L5 and L6 releases?  With items available/not available in certain packages and bundles.  Et cetera.

These types of troubles are great opportunities for the Marketing department at Faithlife to kick into gear (yes, even on a weekend) and make some posts and try to alleviate ill will via the various social media sites that Faithlife leverages for sales and marketing purposes.  Eight hours ago was the last time an update to the Logos Facebook page was made and nine hours for Twitter.

A first step to helping ease issues for the Faithlife Users should be to immediately redirect all their various branded (Vyrso, Verbum, Logos, and others) traffic to an status page—this is so incredibly simple to have done and can even be automated.  Logos.com has it sometimes, but I have never seen it used for faithlife.com, vyrso.com, or verbum.com.  In fact, the verbum.com tells people to call into the toll-free number and order Verbum 6.

And MJ, I agree with you, technical folks very frequently assume they are going to provide a temporary fix or resolution for an issue in less time than it will actually take.  That is why IT management needs to be involved and be the ‘voice of reason’.

Is it the techies’ responsibility to provide the customers of the various Faithlife brands of products?  Probably not, as it is not a very wise use of their time.  Is it the responsibility of Faithlife to provide service restoration updates to its customer base through to a fix or resolution?  Absolutely!!

I really, really like Faithlife.  The vast majority of the Faithlife folks I have interacted with are very good and competent at what they do.  I consider both my current and past sales rep (who left the company) friends.  I have made a huge investment in Logos resources and am planning on continuing to expand my library with them.  They are my Bible software vendor of choice—hands down!  This is not a bash Faithlife post, but it is ‘there is a significantly better way to do things’ type of post.

I can truly empathize with the problems that Faithlife has experienced with their infrastructure this weekend because I have been there—both as a technical specialist in the day and as the person that must provide external communications to the Client and Users!  I also know how much better the internal and external experience can be when appropriate levels and channels of communication are leveraged.

I don’t for one moment think that anyone at Faithlife would want to cause anyone of us grief, but it is what sometimes has happened.  For this weekend’s issues, some Users will be negatively affected and others will not even realize an outage occurred.  But for those affected, more effective communication would have made a positive difference and would have avoided wasted time and/or worries from many Faithlife users.

Bottom line 1:  From a technical perspective, should I as a Logos Users expect that Logos.com and the related services be available 7x24?  If so, then Faithlife needs to make some changes to their infrastructure to support such.  If 7x24 availability is not their goal, then I believe that should be communicated so that people will not have an unrealistic ‘unwritten’ expectation.

Bottom line 2:  From a communications perspective, I truly believe there is a great deal of improvement that Faithlife can make with both their internal and external communications—changes that will benefit themselves and their vast User base.

Posts 26014
Forum MVP
MJ. Smith | Forum Activity | Replied: Sun, Nov 23 2014 1:59 AM

Actually Wayne, I agree with you and hope Faithlife learns the value of preplanning including disaster recovery. I also agree that some people will have experienced a significant loss. But I also believe in letting the techies be techies and to express our frustrations and anger to people who can do something about it. But then again, I'm a great believe in the conservation of energy - especially mine. Too many great things to be doing to waste energy where it is ineffective ... but that's just me.

Orthodox Bishop Hilarion Alfeyev: "To be a theologian means to have experience of a personal encounter with God through prayer and worship."

Posts 80
LogosEmployee
Jim Straatman | Forum Activity | Replied: Sun, Nov 23 2014 2:22 AM

Wayne,

Thank you for your analysis. We're already discussion many of the points that you raise, and you can expect to see our operational and communication processes improve in time. For now, I'm updating the following thread with progress.

https://community.logos.com/forums/p/96616/665726.aspx#665726

Jim

Posts 784
Cynthia in Florida | Forum Activity | Replied: Sun, Nov 23 2014 3:22 AM

A viewpoint from a non-technical, side... (Directed at no one in particular).

Honestly, I am shocked, and somewhat embarrassed by the behavior and language some displayed the last couple of days on this forum.  When did having the right to voice an opinion give a license to speak so rudely and without love?  Between the unbelievably distasteful, inappropriate remarks on the "dating" thread and the reaction I have read about Logos being down, I wonder what our Savior thinks about how we Christians are representing Him.

Every day, when my children walk out the door, I say the same thing.  The conversation goes like this.

Me:  "Remember..."

Son:  "I know Mom, Who I am and Whose I am."

Me:  "Who are you?"

Son:  "I am Luke F. and I represent this family and its name."

Me:  "Whose are you?"

Son: "I am son to the King Most High, a physical representative of God in Heaven."

Me: "Okay, sweetie.  Have fun and remember to represent both with honor."

I know it sounds cheesy, but I did it with my daughter (now 24 in grad school) and now my son (who is 16).  My daughter told me that while she would role her eyes every time here at home, she found that now that she is  in college, she still can hear that dialogue in her head every time she walks out the door.

I guess my point is that we need to behave the same on line as we do IRL, which makes me wonder if some of those who speak so rudely in the virtual world behave the same IRL.  I understand that frustrations, such as the site being down, can cause us to be "snappy," but I am also reminded that the core of who we are is manifested more in how we RE-act than how we act.

Just saying...

Cynthia

Romans 8:28-38

Posts 2016
GaoLu | Forum Activity | Replied: Sun, Nov 23 2014 3:49 AM

Some really good advice, Cynthia.  And you are doing something great as a mom to teach your children well.  Thanks.  

Posts 8
Paul Coleman | Forum Activity | Replied: Sun, Nov 23 2014 6:48 AM

I worked in a large WAN and Datacenter for years. If and when a failure occurred we were always inundated with grumpy people and comments -- so much so, that to get the job done and matter resolved, we would actually shut off communication to all but the critical personnel actually working on the problem.

The fact that there are people working on the matter, and the ocassional techie update for guys like me, is all the information that is needed.

For those who continue to complain, this issue is simply a momentary non-linearity in an otherwise smooth operation. Humans deal with such things daily. I am finding it rather amusing the amount of complaints, off-hand comments -- shall we just call them what they are - rude? -- that are cropping up here among a group of people who should be, or claim to be, facing their world with a different perspective than the world.

To the Logos staff -- many thanks for the work you do, even when it necessitates putting your weekend plans on hold, to keep services available.

Posts 3005
Milkman | Forum Activity | Replied: Sun, Nov 23 2014 7:10 AM

Great reply! We have done something like this with our kids as well. Cheesy? Maybe. But like you said, that has gone on with them now that they're older.

Great reply Cynthia.

Cynthia in Florida:

A viewpoint from a non-technical, side... (Directed at no one in particular).

Honestly, I am shocked, and somewhat embarrassed by the behavior and language some displayed the last couple of days on this forum.  When did having the right to voice an opinion give a license to speak so rudely and without love?  Between the unbelievably distasteful, inappropriate remarks on the "dating" thread and the reaction I have read about Logos being down, I wonder what our Savior thinks about how we Christians are representing Him.

Every day, when my children walk out the door, I say the same thing.  The conversation goes like this.

Me:  "Remember..."

Son:  "I know Mom, Who I am and Whose I am."

Me:  "Who are you?"

Son:  "I am Luke F. and I represent this family and its name."

Me:  "Whose are you?"

Son: "I am son to the King Most High, a physical representative of God in Heaven."

Me: "Okay, sweetie.  Have fun and remember to represent both with honor."

I know it sounds cheesy, but I did it with my daughter (now 24 in grad school) and now my son (who is 16).  My daughter told me that while she would role her eyes every time here at home, she found that now that she is  in college, she still can hear that dialogue in her head every time she walks out the door.

I guess my point is that we need to behave the same on line as we do IRL, which makes me wonder if some of those who speak so rudely in the virtual world behave the same IRL.  I understand that frustrations, such as the site being down, can cause us to be "snappy," but I am also reminded that the core of who we are is manifested more in how we RE-act than how we act.

Just saying...

mm.

Posts 8
Paul Coleman | Forum Activity | Replied: Sun, Nov 23 2014 7:24 AM

Redundancy -- As one who also makes presentations, has taught classes in a tech environment, and has led group studies, I know that I am responsible for my own redundant options - I should not assume that a single service will be available at the moment I need it. Thus, my notes, slides, optional information and such, are saved locally, copies uploaded to a second cloud location, and I must also have a secondary Internet connection option readily available.

The complainers here have simply forgotten this simple self sourced basic rule of thumb regarding all things tech. Datacenters are not immune to failures. I once worked in one where both the primary and secondary backup options failed in the same day. Stuff happens. The end user will do well to be thankful they have the sort of tech we do, but always assume responsibility for their own secondary access options.

Posts 279
Sean McIntyre | Forum Activity | Replied: Sun, Nov 23 2014 7:47 AM

Paul Coleman:
The complainers here have simply forgotten this simple self sourced basic rule of thumb regarding all things tech.

A lot of assuming there.

I went to church and everything was fine as I am always prepared for such kind of eventualities. I am super cautious and I even back up google drive to dropbox as well as local backup options. I keep paper notes as well as the ones on my ipad and I always have a traditional Bible on hand.

But does that mean I should not complain?

I make sure that I download the most important resources to my iPad and now I find that even those are inaccessible because the app wants to sign in before I can use them. 

Page 1 of 5 (82 items) 1 2 3 4 5 Next > | RSS