Google does postmortems too… at scale.
http://googledevelopers.blogspot.com/2013/05/google-api-infrastructure-outage.html
“Our normal rollback procedure failed…”
Followup with more details here: http://googledevelopers.blogspot.com/2013/05/google-api-infrastructure-outage_3.html
Now that’s the kind of post-mortem I like to see! Thanks @Stephen_Lewis - might be worth a new post linking back to this one.
@EdS I’ve only got access to the mobile client right now, so can’t easily link back - if you’d like to create the new post, you’re more than welcome - otherwise I’ll do it later.
Thanks, but I’ll leave it to you, if that’s OK. It’s a great link!
As we say – PMTF (Post Mortem to Follow)
I’m happy that they decided to share more of this.