“Helplessness: that dull, sick feeling of not being the one at the reins. When did you last feel like that –- and what did you do about it?”

I rarely have this feeling. Almost always I can either acquire control or I can find some way to escape the situation. There is one time, not really a matter of helplessness, but one of catastrophic failure that recently happened to me that I can write about. Several months ago I had a server, a Dell uber-tower that was 9 years old and suffered a total systems failure while I was actively trying to backup files on that server because I just didn’t trust the tape system to work properly. Turns out I was just a little bit psychic I guess, because half-way through my attempt to backup the machine, there was a catastrophic fault on the servers motherboard which pretty much hosed the entire machine.

I needed to get the data off the tapes and get a new server set up as quickly as I could. The realization of the failure hit me right in my gut. It’s where cancers always feel stress, like a knot in your stomach. Not nausea, but it felt just like I had been punched right in my gut and was short on air.  I arranged to get a new server up and running and got the system back on its feet but needed some of those files on the tapes in order to rescue everything. I called all around and nobody had that old technology still, so I had no choice but to resort to data rescue services. They got the tapes, and after a protracted back and forth regarding them I received the files that were written to the tapes on several DVD’s. Turns out that Backup Exec lied about making successful backups all along and that all my tapes had filenames and directory structures, but no actual data to any of the files. They were all zero kb. In many regards I had spent a lot of money for DVD’s with just the headstones of the files I needed, and no bodies.

Thankfully I had mirrored the system to another server a few months back and I was able to rescue a majority of the data that I really needed and only lost about two months of my coworkers work. It was bad, but it wasn’t heart-attack bad.

Since then, I resolved that I would never again trust any data to tapes, and thankfully when that server died it took the last tape drive with it, so there are no more tape drives in use in my office. Tapes are very 20th century things and there are better ways to store and backup data now.

I will never forget that feeling of being punched in the gut. Thankfully now the relevant systems are under contracts and warranties and letters of agreement and whatever else we could find to properly cover-my-ass and ensure that this would never again happen to me. It might happen to someone else here, but it won’t be my fault, ever again.