If I wasn't absolutely pissed last time the network went down because of my family, this time I am. I was only able to reach step two of five during the last outage. Here's what this means.
All of my servers and networking equipment is not where it should be. It has been moved to where it is physically accessible to me (in a wheelchair) and the rest of my able-bodied family. Little did I know that the outlet they had all of my servers and our networking equipment on was operated on a single light switch. One flick while I was taking one of my sacred naps brought everything to a crashing halt.
I suspected they would do something really fucking stupid again, so not everything was permanently damaged. The network bounced back without manual intervention, no small feat for its complexity. All but one of the servers were able to POST and carry on.
Unfortunately, we lost the back up server. Ordinarily, I wouldn't be very concerned; that's the easiest server to manage without! However, this is the second period of downtime in less than a month. There is an obvious trend here. Every second my machines are in this high-risk environment (surrounded by pets, neighbors, family, etc) is just a total crash waiting to happen.
The last outage was painful. There was a very long period when I was immediately post-op, during which storage servers bit the dust. I could tolerate more storage servers going down. I could not tolerate any services going down.
Here is the emergency plan. Just like last time, I'm not even sure I'll finish it due to time, injuries, and money.
- Begin short, limited backups to my laptop and external hard drives
- Attempt to replace drives in back up server
- Hold candlelight vigil for fallen server
- Hang on for the now three month overdue stipend to replace machines
- Practice wheelchair combat to threaten family
- Send bill for destroyed servers to the single individual (who will not be named) who is solely responsible for both outages
Hope everyone enjoys the rest of their week.