From 1998: hitting a 32-bit limit on backup file sizes, unknowingly.

From 1998: hitting a 32-bit limit on backup file sizes, unknowingly. And then losing half your data. And then having to merge three partial copies. With a hard deadline. Includes short video telling the story. See also Oren Jacob’s telling of the story at
http://www.quora.com/Did-Pixar-accidentally-delete-Toy-Story-2-during-production/answer/Oren-Jacob

This link via @Peter_da_Silva

Somewhere else in California right around the same time a very very tired software engineer was clearing his hard drive but not really his hard drive but a network drive containing the entire source database for the company. And as luck would have it, the backup tapes were empty.

This brings up some very bad memories. :slight_smile:

Oh dear! What happened next? (Once you’ve used a filesystem with snapshots, you never want to go back. I’ve done one or two mistaken deletions in my time, and, I think, just one bulk restore from tape in anger.)

Also a very similar solution. Took many days but an essentially full restore was made patchwork from a very old tape, stuff on people’s computers and stuff they took off-site.

This was the second time that something like this happened at a company I was working for. I bet this was all too common back then.

It’s a horrifying thought that a business can fail through one mistaken command, but I wouldn’t be surprised.

Yes, it’s the digital equivalent of burning down the factory.