The Proclaim servers have experienced issues at peak times the last two Sundays (3/9 and 3/16) which affected the Proclaim Remote. The increase in response time when rendering slides led some users to be experience an inability to control their presentations.
Last Sunday we fell back to a “known good” version of the code, along with additional logging. We were confident that this would solve the issue of the previous Sunday. It turned out we were wrong but the additional logging helped discover a hardware problem with one of the nodes in the application cluster. Because of the round-robin load balancing this meant that when one node got slow then all consumers suffered the consequences.
We are confident that we have identified the real issue and plan to have it resolved for the upcoming Sunday. I apologize for any disruption this may have caused.
Can you update on the status of this?
We are deploying the slide renderer to new hardware within the hour. Load testing performed this week indicates perf improvements from 2,000 ms to 200 ms at heavy load. The bad node mentioned in the update was taking up to 80,000 ms at peak times so this is a huge improvement. We also have plans for more sophisticated caching which we'll roll out in stages over the next few weeks. This Sunday it will just be the new hardware.
The slide renderer has been deployed to the new hardware. It should be working faster than ever before. Please let us know if you see issues.
I've had issues with the remote, and I'm not sure if it's related.
Does this happen every time you use the remote, or only certain times/trying to control certain presentations? I have not run into this issue in house.
Yes. It happens on my Android phone, and Kindle Tablet.
Proclaim started jumping around and lost my slides for a few moments just as I was reading my opening passage today.... Sure hope this is corrected soon, but for now, I am going to abandon trying... It is too distracting.
Same here started acting up about halfway through the song service and completely died at the time I was to start the sermon. I picked up my iPhone in desperation and it kept wanting me to log in for validation, tried another iPad and it kept failing. I am about out of patience with this program...
Spencer & Scott: Can I ask a question about this? I too have run into problem regarding the remote control feature of Proclaim. Would it not make more sense to build into Proclaim a feature that works over Wi-Fi or network connection on the local level as opposed to everything having to be sent through the servers at Logos which seem to get overloaded quite frequently? I'm not trying to ruffle feathers, but this is how MediaShout and ProPresenter do it, I believe. It only makes more sense to do it this way than to have to run everything through your servers. We are taking a great risk in disruption using the Proclaim remote and I have used it only sparingly because of the issues I have read about. I never run into a problem on Wednesday nights, but I assume it's because the servers are never overloaded on Wednesdays. It seems on Sunday mornings that the issues arise, so I can understand why people are beginning to lose faith in the reliability of the remote. Just my opinion. Don't shoot the messenger, please. :-)
Thanks for the suggestion Josh, I will pass it on. It does seem to be a persistent problem; I have taken the time to test the remote (as well as Proclaim) on Sunday mornings and have run into sluggishness myself. I will talk with Scott about possible solutions soon.
Spencer and Scott, does this sound like it is a do-able solution? Also, if the answer is "yes" what type of development timeline would you foresee?
I cannot give an answer on this at the moment but I will update when I have more information.
Proclaim already supports universal presentation remote controllers. These emulate keyboard presses wirelessly. This is a good option for environments without reliable internet service or as a backup to the Proclaim Remote.
I would suggest if you feel strongly about this feature to give it a vote on UserVoice. It was originally posted in 2011 but it currently it ranked behind a lot of other features.
http://proclaim.uservoice.com/forums/137112-general/suggestions/2336279
Rodney,
While investigating the problems you described I noticed that your presentation from Sunday was edited 21 times while it was On Air. Actions like reordering, adding, removing and editing service items while On Air could cause some of the issues you describe. Each time the presentation is changed the remote needs to download the latest revision and request new slides to be rendered. We are working on improving the experience of controlling On Air presentations while edits are being made but it will be still be a few weeks before we are able to ship these changes.
I recommend avoiding changes to the presentation once you have gone On Air.