Calls to 911 weren’t going through. Some computer screens at emergency operations centers went blank. Backup systems engaged — and in some cases, backups to the backups had to be activated.
Officials in the counties — Armstrong, Butler, Cambria, Indiana, Somerset and Westmoreland — undertook various emergency measures, posting alternate phone numbers through social media, asking local TV stations to alert the public and contacting facilities that have frequent contact with 911, such as nursing homes, to give them a heads-up.
“We toned out every fire, EMS and police agency in the county, had them man their stations and then put out an emergency alert for the community and had them contact us on our business line,” Steven Bicehouse, director of Butler County’s Department of Emergency Services, said Thursday.
Thanks to built-in redundancies, no one suffered any harm as a result of the outage, according to authorities in most of the counties, which were back up and fully operational within about three hours. But the disruption — which apparently affected not only CenturyLink’s main system but its automatic backup as well — has left 911 officials without a concrete explanation about what caused it and why.
“Our main system experienced a glitch. There was a malfunction. So what it did ... it was supposed to do a failover to the other server. Somewhere along those lines it did not fail over,” said Nicholas Caesar, deputy director of the Westmoreland County Department of Public Safety. “As the system’s designed, it’s not supposed to perform like that. We’re trying to find out why the redundancy ... didn’t kick in.”
The way Mr. Bicehouse explained it, the main system locked up but did not completely fail. That confused the backup system, which is supposed to take over only in the event of a full failure.
“Because it never went down, it locked up, [the backup] never picked up,” Mr. Bicehouse said.
Some 911 officials said the problem was worrisome.
“Is it something that is concerning and serious? Absolutely. But is it something unexpected? No, it’s not,” Mr. Bicehouse said. “We have demanded — not asked, but demanded — from the vendor a solution and root-cause to what happened, and how they’re going to fix it. We are all concerned. But at this point we have confidence this will get fixed, and we’ll develop a solution moving forward, and we’ll continue to do business.”
The Pennsylvania Emergency Management Agency, which described the situation as a “cause for concern,” monitored the event as it unfolded and will continue to examine the issue.
“It is important to rectify these types of problems immediately when they happen,” agency spokeswoman Ruth Miller said, “but given the effectiveness of the redundant systems and the urgency with which phone service providers responded to the need for technical repairs, we have no reason to believe public safety services were jeopardized in any way during this incident.
Various counties share 911 systems as part of a push toward regionalization, which cuts costs and enhances efficiency.
CenturyLink this week issued a brief statement that blamed the outage on a “technical issue with one of our vendors.”
Company spokeswoman Debbie L. Keyser said CenturyLink would not answer questions about the outage until an internal review was completed.
———
©2015 the Pittsburgh Post-Gazette
Visit the Pittsburgh Post-Gazette at www.post-gazette.com
Distributed by Tribune Content Agency, LLC.
