As reported by Government Technology, the first outage occurred Monday, May 16, during a routine network upgrade and lasted two hours. State workers were unable to access the state network and their e-mail. State officials said immediately afterward that it was one of the most widespread and longest system failures in the state government’s history.
The second incident the afternoon of Wednesday, May 19 was even more serious. Kurt Weiss, the spokesman for the Michigan Department of Technology, Management and Budget, said that ironically the second incident occurred while performing a disaster recovery test on the state’s mainframe. During the exercise, fiber broke between the test environment and the production environment, causing the mainframe to shut down.
As a result, computers crashed in the state’s 131 Secretary of State branch offices, Weiss said. (In Michigan, Secretary of State offices include services that are equivalent to a department of motor vehicles.) For two days, citizens couldn’t renew drivers’ licenses or get new licenses and other services.
“The price of gas is high, people’s time is precious and people were driving to their Secretary of State office only to see a sign that said ‘Computers Down,’” Weiss said. “So they didn’t close the offices, but they weren’t able to — once folks got in there — do anything because our technology was down.”
The mainframe was up and running by Wednesday night, but computer applications were still inoperable. Secretary of State employees couldn’t work in the applications because files had been corrupted during Wednesday’s outage. On Thursday, corrupt files were repaired.
To make up for lost time, the state extended all branch office hours on Friday, May 20, and plan to on Monday, May 23. The department’s seven Super Centers will be open until 2 p.m. on Saturday, May 21. Secretary of State officials said they will waive late fees for anyone affected by the outage.
For each of the two outages, the state’s Department of Technology, Management and Budget will conduct a root cause analysis and document the lessons learned and submit it to state CIO David Behen.
“We understand that this has been an inconvenience to our customers and we want to do everything we can to assist them,” Secretary of State Spokesman Randall Thompson said in a statement. “Unfortunately it will take some time for us to recover once the system is up.”
