Customers’ personal, credit and debit card data were exposed when they paid for utility bills, business licenses, permits, and in some cities, parking fines.
Although forensic investigators were able to identify malware as the hacking method, it’s not so easy to explain why potential targets weren’t notified and cardholders themselves were often left in the dark for weeks.
Medford, through Deputy City Attorney Eric Mitten, reported Monday it had suffered data breaches through Click2Gov as early as five months ago.
The data breach came in two waves — Feb. 18 to March 14 and March 29 to April 16; the initial alarm was sounded by a financial firm that notified the city on April 19.
Forensic investigators from Kroll, a Manhattan-based, technology-enabled intelligence and information management firm, determined on June 5 that malware was used to gather payment card information, including credit or debit card numbers, cardholder names, expiration dates and CVV codes. Social Security numbers were not affected.
Medford sent a letter July 23 to 1,842 potentially affected cardholders.
Mitten said Monday the city didn’t want to alarm people who weren’t affected by the breach.
“We’ve been working closely with the forensic investigators to get more specific information and to pinpoint who may have been affected,” Mitten said. “If we would have announced it immediately, we wouldn’t have known who was and who wasn’t affected; we recommend everyone monitor their credit cards, regardless.”
Superion, the Lake Mary, Florida, developer of the Click2Gov system, says it took steps to notify its customers in 2017 of potential problems.
“Last year, we reported that a limited number of on-premise clients had identified suspicious activity on their servers that are used to host Superion’s Click2Gov product,” said spokesperson Carol Matthieu in response to a series of email questions. “Upon learning of the activity, we took proactive steps to quickly notify all Click2Gov customers as early as September 2017.”
Matthieu said Superion, which also sells emergency management software throughout the country, launched an investigation and engaged a forensic investigator to assess what happened and determine appropriate remediation steps.
“For security and confidentiality reasons we cannot disclose any information about our customers, their environments, or their security,” Matthieu said. “We have assisted as many customers who would allow us, by providing best-practices advice and helping them with the application of patches in order to update and better secure their networks.”
She said Superion has deployed the necessary patch to software and assisted customers in the application of patches. However, Superion does not control its customers’ networks, she said.
“It is important to note that these security issues have taken place only in locally hosted on-premise networks in certain towns and cities,” Matthieu said. “Not a single client in Superion’s data centers or in the Superion Cloud has faced these issues, even when they are using the same software product. At this time, we have no evidence showing that it is unsafe to make payments utilizing Click2Gov on hosted or secure on-premise networks with recommended patches and configurations.”
Mitten wrote potentially affected customers that a hacker gained access to portions of the city’s website and installed software “that was designed to capture payment card information as it was inputted on the website.” The website was temporarily disabled until it was re-secured, he said.
Medford City Manager Brian Sjothun said he was unaware of other cities having their Click2Gov systems hacked.
While declining a direct interview, Sjothun responded to emailed questions.
“It was not apparent to the city of Medford whether or not other municipalities were affected by the same situation,” Sjothun wrote. “The city of Medford has focused on addressing the circumstances of our customers. We have not focused on circumstances in other municipalities.”
Asked if he thought the software patch was effective, Sjothun offered: “We have instituted measures designed to help prevent a future incident.”
Sjothun said select members of the city administration were brought into the loop on April 19 with the city council and mayor notified the following week.
“What we had been told about the cyber stuff is that it is somewhat inevitable,” said Councilor Kim Wallan. “We take as many security steps as possible, but we might not have taken enough. No matter, you have to be ever-vigilant; I’m glad it wasn’t bigger.”
Wallan said criticism goes hand in hand with such issues.
“I’m not frustrated with the way it was handled,” she said. “When something like this happens, you’re going to get criticized; it’s the nature of the beast. It cost the city tens of thousands of dollars for people to pay bills this way, instead of checks or cash.”
Sjothun said the city was covered by cyberinsurance.
Medford police Chief Randy Sparacino was notified about potential concerns on April 23, Deputy Chief Scott Clauson said.
“It was later determined to be a high level ‘hacking’ type breach,” Clauson said. “This is something outside of our expertise at a local law enforcement level. While we do house the Southern Oregon High Tech Crimes Task Force and have our own Financial Investigation Section, this type of crime is not something we are forensically trained to deal with. Our expectation is that the vendor will investigate and quickly resolve at a software programming level.”
Clauson added the department will take reports for anyone that has had card information compromised and needs documentation.
Bend IT Director Randy James, whose city presently uses a variation of the Click2Gov system, said unknowns lurk at every turn.
“You never know when you’ve actually been compromised,” James said. “You never can say if you’re absolutely safe.”
He points out the malware hackers can insert themselves into systems, and then take a break, as was the case in Medford.
“They lie low after the initial penetration to avoid all protocols,” James said. “It’s not a matter of being impacted but how you are going to deal with it.”
Though Bend’s website has not been compromised, the city will no longer subscribe to Click2Gov once its five-year enterprise platform makeover is completed, he said.
“We’re going with a different vendor,” James said. “But it’s more a result of the evolution of our system.”
Superion is just the latest name for the organization running Click2Gov, he said.
“They’ve been acquired several times over the past six or seven years,” James said. “It’s not atypical in the technology world for bigger vendors to fill in a portion of what they need by acquiring an existing company rather than build their version.”
James said zero-day attacks — where hackers take advantage of a security vulnerability the same day it becomes generally known — are on his radar, but he wasn’t aware of Medford’s issues until it hit the news wires.
“We watch and evaluate our systems and make appropriate security adjustments as the industry dictates,” he said. “This vulnerability hasn’t had a long-term exposure, so in reviewing it people are a little slower recognizing this one. Malware like this finds the vulnerabilities in the software, such as missing security protocols.”
Given the cyberthreats from every direction, James said it’s not a matter of if, but when systems will be compromised.
“It’s sort of like playing whack-a-mole,” he said. “You get one and another pops up, you just review it again and again. You respond to one threat and then another. When you go after one with a patch, it might introduce another. Sometimes you just have to pick and choose priorities in finding the best way to protect your organization.”
While it took Medford nearly seven weeks to go public after the breach was confirmed, Bozeman, Montana, continued to hold its peace well after complaints began surfacing in fall 2017. It wasn’t until July 16 that Bozeman notified 3,000 customers of the hack.
Following several complaints, Bozeman took down the utilities payment page and hired computer forensics firm Lake Missoula Group to investigate.
The local investigators found no evidence of unauthorized activity, and the city went on with business as usual. City Manager Andrea Surratt told the Bozeman Daily Chronicle the city didn’t notify all users at that time because there wasn’t any evidence available.
Superion, however, probed the matter and found evidence of fraudulent activity on the site, but the Florida firm didn’t notify Bozeman until July 3.
The delayed response led Bozeman advocacy firm Pirl to file a Freedom of Information Act to obtain a “complete list of names of people whose information was exposed, all correspondence that discusses the breach, and all documents about how the city found out about the breach and how it’s planning on remedying it,” according to the Daily Chronicle.
Chief Operating Officer Joseph Chyatte said he was unaware of similar pushes for transparency.
“What’s disconcerting to me is that the city admitted receiving complaints of fraudulent transactions for a year and a half and didn’t tell the public until this month,” Chyatte said in a Friday interview. “Our position is, a government works best when it’s transparent. There is some obligation to notify residents when this kind of thing happens.”
Only a handful of municipalities in Oregon used Click2Gov, and chances are fewer will opt for a system plagued by hackers.
Ashland Administrative Services Director Mark Welch said the billing system used by his city doesn’t involve customer financial information going through city servers.
“We don’t touch any customers’ financial information and our servers never see it,” Welch said. “Pretty much for this reason; it can be a big nightmare.”
Here is a list of reported Click2Gov data breaches. Others may still be in the investigation period, or have not been publicly reported.
- Oxnard, California, March 26 to May 29, 2017
- Lake Worth, Florida, April 3, 2017, to Jan. 22, 2018
- Goodyear, Arizona, June 13, 2017 to May 5, 2018
- Oceanside, California, July 1 to Aug. 13, 2017
- Bozeman, Montana, July 1, 2017, and Oct. 24, 2017
- Beaumont, Texas, Aug. 1-24, 2017
- Ormond Beach, Florida, Aug. 14 to Oct. 4, 2017
- Fond du Lac, Wisconsin, August to October, 2017
- Wellington, Florida, Nov. 28, 2017 to June 4
- Okaloosa County, Florida, December 2017 to March 2018
- Midland, Texas, December 2017 to June 2018
- Thousand Oaks, California, Jan. 4-10, 2018
- Medford, Oregon, Feb. 18 to March 14, 2018, and March 29 to April 16, 2018
- Midwest City, Oklahoma, May 25 to June 21, 2018.
Superion is a Vista Equity partner. What potentially muddies the water for Click2Gov customers is a merger announced this week put together by Bain Capital and Vista Equity Partners. The multibillion-dollar private equity firms have combined Superion with San Diego-based TriTech and Alpharetta, Georgia-based Aptean to create a company providing software and services for public safety and government management in about 5,500 communities and jurisdictions, covering about three-quarters of the U.S. population.
©2018 the Mail Tribune (Medford, Ore.) Distributed by Tribune Content Agency, LLC.
