Blog: Crisis Management

Compassionate and conscientious email staves off reputation crisis

In a recent article on the Bernstein Crisis Management blog, we discussed a crisis management email from Twitter that was missing a core component – compassion. Now, we’d like to show you an example of a company absolutely nailing it that was brought to our attention by friend and colleague Bruce Bonafede.

Here’s the email that went out to members of Yahoo! Sports’ Fantasy Football league following a Sunday outage – the absolute worst day of the week for their system to crash:

Dear Yahoo! Fantasy Users,

As I mentioned in my earlier email, Yahoo! works hard to provide great experiences for our users every day. For you, we understand that Fantasy is a core part of your daily routine during this time of year. On Sunday we experienced technical issues which caused the Fantasy experience to be inaccessible. We’re sorry. Our entire team worked tirelessly to re-establish the service – however, we had a significant outage. We know this created inconvenience and frustration for millions of our fans, and we feel we owe you an explanation.

So what happened? At Yahoo!, we have giant machines called “filers” that process a lot of the real-time data and stats for us and for you. We do millions of calculations every hour for our games, and normally our machines can handle this with no problem. Recently, we discovered a hardware issue in one of the filers that caused the other one to overload. We replaced some hardware, re-configured the setup, and did some testing. However this Sunday – at approximately 12:15 p.m. Eastern – the new configuration failed. This created an overload on storage capacity and took the Fantasy part of our site down.

We had dozens of engineers from various teams working together to try to determine the cause and fix it. One option was to fall back on another data center, but that would have meant shutting the game down and losing scoring data. We wanted to avoid that at all costs. Ultimately, we were able to move our mobile apps to a back-up data center, free up storage to get the PC version of the game working, and get the mobile apps up in a “read-only” state – meaning you could see scores and data, but you still couldn’t set lineups and interact.

We spent Sunday night and most of Monday looking at dozens of potential causes. Monday afternoon, we stress-tested our system. Everything seemed to be in working order, so we turned on all our mobile app functions in time for Monday Night Football. Everything performed as expected and continues to do so. We’ll have all hands on deck this coming Sunday to closely monitor performance and ensure we can respond quickly in case of any abnormal activity.

We know many of you have questions about your leagues and games, so we’ve also set up a FAQ here that will explain how we are going to handle various scenarios.

Finally, we know we failed you this weekend, and there’s no way to fix that. As a token of our appreciation for your patience and continued support, we wanted to give something back. First, we’re offering a complimentary football weekly Scouting Report for the remainder of the year. Hopefully this will help you down the stretch. The report includes weekly expert analysis, rankings and news to help you make smart decisions. Second, we are offering all our users a 20% discount for the rest the year at the Yahoo! Sports Store.

For any updates, we will keep you informed on @YahooSports and facebook.com/yahoosports.

We won’t stop working to provide you with new and improved features that help you enjoy your Sundays.

Thanks again for playing and for your patience.

Best,

Ken Fuchs
Head of Yahoo! Sports

Prompt, compassionate, honest, informative and interactive. Yahoo! checked off on every one of the Five Tenets of Crisis Communications and it shows in the quality and effect of the email.

Kudos Yahoo!, excellent crisis management!

——————————-
For more resources, see the Free Management Library topic: Crisis Management
——————————-

[Jonathan Bernstein is president of Bernstein Crisis Management, Inc., an international crisis management consultancy, author of Manager's Guide to Crisis Management and Keeping the Wolves at Bay - Media Training. Erik Bernstein is Social Media Manager for the firm, and also editor of its newsletter, Crisis Manager]