Last Friday began badly for me. I checked my Blackberry while still in bed before 6am ET and found that our internal community hosted by Jive Software had been offline for at least an hour, and since there was no communication from Jive it appeared that they had not detected the problem.
Still in bed, I opened my iPad and entered a Priority 1 case with Jive support that voiced my displeasure: "Site down for more than an Hour - where is Jive?" Then while I waited for a response I hopped over to Facebook and resisted the urge to call Jive out publicly, as I had no facts about the situation. I simply posted as my Facebook status:
Ted Hopton does not like bad news when he first checks his Blackberry in the morning. #badstart
It did not occur to me that I was at that moment helping a friend and fellow Jive customer realize that the outage was widespread, as I assumed only my site was affected. You can read that story here: Managing Expectations – The value of Transparency
After half an hour of dealing with email and checking out the news online there was STILL no response from Jive. Feeling pretty testy, I posted a reply to my own P1 support case further expressing my frustration, pointing out that all I need is a short response assuring me that they are working on the issue. It's a very lonely feeling to have a major technology failure on your hands and not be able to rouse the support team responsible for fixing it.
Only after I escalated the issue through executive channels did I learn that Jive had a massive failure at their data center and we were not the only customers affected. That news changed everything, and Jive needed to get it out to all affected customers MUCH faster. I no longer needed to worry that my issue was not getting attention. I knew then that the issue was getting all the attention Jive could possibly give it.
In fact, despite the huge, terrible and inexcusable impact this outage had on me and everyone in my company, my emotions actually shifted somewhat on the spectrum from fury toward sympathy. My thoughts were more along the lines of, "Boy, they really screwed up! This is going to hurt." I felt sorry for them. And with good reason, as the outage lasted all day, giving Jive a big ol' black eye.
So when I ended up being interviewed by Information Week magazine (I was not selected randomly -- IW is part of my company, so the editors knew from my update emails to all employees that the problem was widespread), I was no longer mad at Jive. Disappointed would be a better word.
I'm not sure exactly what I actually said when the reporter called. Maybe I carelessly joked about taking Jive to the woodshed, but what I meant to say was that I WON'T have to take Jive to the woodshed over this incident because it has affected so many customers that Jive will take it extremely seriously and will take whatever steps needed to ensure it doesn't happen again.
For the record, it is my view that Jive completely dropped the ball several ways on Friday:
1) the failure of the failover system was inexcusable;
2) the failure to communicate quickly and proactively with all customers affected was unprofessional;
3) the failure to use social media to get in front of the public view of the situation was an opportunity that a vendor of social media software should not have missed.
Hardware failures can happen, so that's infuriating but understandable. Once Jive finally set up the Webex session for all customers to keep updated on what they were doing to resolve the issue, as well as to let us ask any questions we had in a common place where we could see all of the questions and answers, the information flow was effective and efficient. There was another minor glitch, in that I did not receive some email updates from Jive until Saturday and Sunday (I don't know why they were delayed).
But I also received phone calls from my Jive account manager and support team leader during the day to apologize and make sure I knew everything they were doing, and then again after the site was restored, to make sure I was satisfied that all was finally well.
I love Jive's technology. We're having tremendous success with it and reaping great benefits from using it. I hate it when it lets us down, as software is bound to do from time to time, especially cutting-edge software like Jive's. We have come to depend on Jive as a critical partner in running our business, so Friday was a bad day, but as long as it results in significant improvements then the memory of it will fade. I'm counting on Jive to learn and grow from this experience, so no woodshed's needed this time.
Posts
