Some of you may have noticed that the freshbrain.org site was down for quite a while this weekend. More than 12 hours. We are very sorry if you were impacted by this outage.
Let me quickly fill you in on what happened.
The network.com team applied a simple firmware update to a switching system. This update was suppose to have done nothing to the environment. It was not suppose to impact any of the systems within the datacenter.
Well - turns out that it did.
The update caused some of the systems in the datacenter to be unreachable. The challenge the support team faced were twofold. One, they didn't didn't know the update was applied so they didn't think to check there first. And two, some but not all of the systems were unreachable. Therefore they believed the problem was elsewhere.
To make a long story short - this was a great learning experience for us. This is a part of why you do a Preview Release. It is about learning so that you don't have these kinds of problems with the real release.
Again, we are very sorry and will take action to insure that this does not happen again.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment