The region, which is home to Aberdeen, the county seat and the state’s third-largest city, had faced an unidentified communications issue for “a little over a year,” according to its Chief Information Officer Paul Sivertsen.
The delay didn’t exclusively impact one user or group, one account or one type of account, Sivertsen told Government Technology earlier in April. Instead, it randomly affected all departments and impacted not only the emails sent from Google accounts but the usual follow-up “delayed sent” messages that tell an intended recipient a communication has been delayed. The delayed emails, the CIO said, were arriving anywhere from 12 to 36 hours later.
The county had difficulty making contact with officials at Google. However, in an email that followed an April 17 article on GT’s website, Sivertsen indicated the county had reached the technology company and resolved the problem.
It had been “an issue with their blacklisted IPs,” or Internet protocol addresses, Sivertsen said via email, describing the article as “the missing piece to this puzzle,” which was “contact with Google.”
Sivertsen indicated he heard from Google on April 18, and the interaction identified and appears to have fixed the issue of blacklisting, which his agency had previously suspected.
In a LinkedIn message, Donna McIntire, Google customer engineer manager, public sector, North America, said the county’s firewall had used a real-time blacklist (RBL) monitor that had blocked certain Google-based IPs.
The issue isn’t believed to have resulted in any confirmed interruptions of county services. However, earlier in April, The Associated Press reported it had impacted emails from livestock exhibitors, entertainment buyers, fair board members and others that were sent to county fair managers — and had nearly interrupted veterans services to at least one veteran.
An email blacklist is typically a database updated in real-time, which determines if someone at an IP address is phishing recipients or doing something that could be considered a scam. On its end, the CIO said, Brown County has “security measures to keep this type of email from getting to our end users and to protect our systems.”
The county had previously been adding Google IP addresses to its whitelist of allowed senders, but Google’s more than 1 billion active monthly users made that an ongoing task. In working with the Google representative, Sivertsen said “we sent test Gmail messages to our (Brown County) address,” tracking those that were delayed or retried and finding out why.
“At this point, they could see the messages that were being sent to the delay/retry queue all had one of their blacklisted IP address(es),” Sivertsen said, indicating he was subsequently able to whitelist the address and others.
The CIO said that while there’s no metric for the agency to determine whether it is receiving 100 percent of emails sent to it, he added, “I can say, at this point, I have seen 100 percent reply rate from the test emails I have been involved with. Google is not seeing any messages addressed to Brown County being sent to the delay/retry queue at the time of testing.”
