This is just too bizarre not to post. I got this in an e-mail today.
Favyen Bastani
11:28 AM (5 hours ago)
to lndynamic-anno.
Hi all,The datacenter (Toronto) will be performing emergency maintenance at 3:30 pm EDT today afternoon due to raccoon in the electric system. No downtime is expected (although power will be going offline, the backup generator should take over). The details provided by the datacenter are copied below.
Best,
Favyen Bastani
Luna NodeEmergency Facility Maintenance
Dear Cogent Customer,
Please be advised, during a routine visual inspection in the building’s main electrical room, Cogent’s electricians found an issue with Cogent Main Switchboard #2. Location : 245 Consumers Rd. 300 Toronto, ON A raccoon has gotten in through the hydro vault and has been electrocuted and is lying on top of the bus bars. This, of course, poses a major electrical hazard to the equipment. This will require further investigation and removal of the raccoon, as there is already a burning smell coming from the board.
Unfortunately, this board is one of the main feeds to the UPS Unit for the data center; however, there is a tie breaker that the electricians can swing. But this cannot be attempted until the utility isolation of the incoming bus bars where the racoon is for fear of imposing a potential dead short. Once they get utility isolation from the M1 feeder, the tie breaker can be closed, transferring power through the M2 feeder to further investigate this situation
The electricians are confident this can be done and the data center/facility load will be transferred to Generator backup without any
outages to our customers. The generators will carry the load so that the switchboard can be taken offline and can be inspected safely.Cogent’s Electrical contractor has contacted Toronto Hydro (Utility provider) for a dispatch, as they need to be present for this work. As of now, they have provided an arrival time of 3:30pm EST; at this time or anytime after, this maintenance work will begin. The Cogent NOC will receive regular updates during this maintenance work from the field team.
This, of course, is an unforeseen issue that we are attempting to correct with no downtime to customers. We apologize for any inconvenience, but this is obviously an urgent matter that needs to be rectified immediately and cannot wait for the standard Cogent “maintenance window”.
Our network operations engineers closely monitor the work and will do everything possible to minimize any inconvenience to you. While it is important for Cogent to perform maintenance such as this to ensure the quality of our network, we do everything possible to minimize any inconvenience to our customers. We appreciate your patience and welcome any feedback.
Sincerely,
Customer Support
Cogent Communications
support@cogentco.com
877-7COGENT
A few hours later, I got a follow-up:
This maintenance was completed without incident.
Great Jorb! (Racoon R.I.P.)
