Minnesota government computer systems: Old, at risk and pricey to fix
ST. PAUL-Many of the state of Minnesota's data centers are vulnerable to both intruders and water damage. The state's crack cybersecurity team doesn't have a night shift. And the only people who know how to maintain code handling billions of doll...
ST. PAUL-Many of the state of Minnesota's data centers are vulnerable to both intruders and water damage. The state's crack cybersecurity team doesn't have a night shift. And the only people who know how to maintain code handling billions of dollars in state transactions are near retirement - or long past it.
"Things haven't changed that much," said Mike Arlett, a retired 79-year-old programmer who still comes into work periodically because he's one of the few people left who can update the venerable COBOL code he's worked on for decades. "It's basically the same as it was many, many years ago. It's just running on a bigger, faster platform."
These cybersecurity risks and antiquated technology are front and center at the Legislature this spring, where lawmakers are considering whether to spend more than $100 million bringing the state's software into the current decade.
Members of both parties agree it's a priority to provide secure and modern technology for a state government that does just about all of its work on computers. But it's not the only priority as lawmakers decide how to split up a projected $1.65 billion surplus between tax cuts, schools, roads and bridges, technology, and a range of other popular programs.
That's not a new problem, which is part of why the need has gotten so big.
"With the politically driven budget process, it gets very hard to schedule (technology upgrades) on a proper basis," said Steven Bellovin, a professor of computer science at Columbia University who has researched cybersecurity. "About the time when you say it really should be done, it's rarely urgent to do it right then. It makes perfect sense to postpone it once - but that keeps happening."
Gov. Mark Dayton and the department of Minnesota IT Services have requested more than $125 million for technology upgrades in the next two-year budget. Lawmakers are likely to approve at least some of it - but how much remains an open question.
The requests primarily fall into two areas: $74 million to improve the state's cybersecurity and $51 million to upgrade antiquated hardware and software.
The most urgent tech problem facing Minnesota is cybersecurity.
Recent events from Target's massive data breach to the hacking and leaking of Hillary Clinton's campaign chairman John Podesta's emails to an attack last year that knocked out the internet for much of the East Coast highlight the potential risks from lax cybersecurity.
And Minnesota's computer systems present plenty of targets for criminals. State computers process billions of dollars of transactions each year and store data on students, vendors and benefit recipients.
"A major breach would require our state to cover millions of dollars in identity theft costs and lead to millions of dollars in consumer fraud losses," Thomas Baden, Minnesota's chief information officer, said earlier this year. "It would also diminish the trust Minnesota has in government."
Cyberattacks today are much faster than a decade ago, said Christopher Buse, the state's chief information security officer. "They're certainly more sophisticated. They're more targeted, and they're more frequent."
The state responds to these attacks from a dimly lit operations center tucked inside an office building near the Capitol. From there, a core team of nine cybersecurity workers monitors a feed of network data and responds to attacks - pre-emptively if possible, reactively if necessary.
But Buse said he feels he could do a much better job protecting the state's data with more resources.
Dayton has proposed $74 million to bolster Minnesota's cybersecurity. That includes adding more staff, consolidating data centers that a recent audit found weren't secure from potential intrusion or damage, and making general upgrades to the state's software.
"A very high percentage of successful attacks exploit flaws for which a fix is available but hasn't been deployed," Bellovin said.
Another shortcoming that more money could fix: the team has no night shift. Computer attacks can come at all hours of the day and from anywhere in the world. But Minnesota's Security Operations Center is only fully staffed during day hours.
Skimping on security is foolish, warns Bellovin.
"Your best touchstone for this is what the private sector is doing, because they've got a bottom line, they've got profit and loss, and they are spending more and more on security because they have to," Bellovin said.
Minnesota has fewer than 60 total cybersecurity staff working for all of state government. Minneapolis-based U.S. Bank, whose annual revenue is roughly on par with Minnesota's annual general fund revenue, has more than 550 cybersecurity staff.
Arlett's experience coming in from retirement to maintain decades-old code is far from unusual. Many of the state's computer systems are really old.
"A couple of these systems predate the internet," Baden said. "Several of these systems predate the mouse."
There are 123 state government applications running on old, outdated or dead computer languages, said a representative for MN.IT, the state information technology agency.
There's the Medicaid claims system running code that dates to the 1970s, an equally old Department of Education mainframe, and the statewide accounting system scheduled to go out of date in January 2018.
This poses less-dramatic problems for the state than cybersecurity vulnerabilities. In fact, by virtue of being obsolete, some of these systems may actually be safer than more modern software - since no one targets them.
"There's not a lot of hackers who understand 1970s-era COBOL at this point," Bellovin said, referring to the computer programming language.
But old systems pose other problems. The same obscurity that makes them hard for hackers to breach also makes them hard for state workers to maintain, because students don't learn old computer languages like COBOL anymore.
"Personnel wanted to know, how come we didn't have any entry-level people?" Arlett said. "Well, nobody trains them. All we have are, basically, senior people with 10, 20, 30 years experience."
These senior-level staff are more expensive. And around 90 percent of them will be eligible for retirement within five years. It's a similar story at many other state projects.
If a system's hardware is old, that can cause problems when parts break that aren't made any more. Bellovin mentioned an example of how the FBI was once reduced to buying spare parts on eBay.
"The upgrades of these obsolete systems is going to save money," Bellovin said. "You are going to spend a little bit of money now - you're going to save money going forward."
In Minnesota's case, though, it's more than a little bit of money. Dayton is requesting $51 million in upgrades to the state's computer systems.
But Bellovin warned that these upgrades, while a good idea, would likely take more time and money than projected.
"The conversion is likely to be a disaster," he said. "Software development is very hard to manage. If you contract it out, it's harder unless you have very good management of the whole process."
Minnesota has seen several troubled technology rollouts in recent years, most notably the debut of MNsure.org. In a statement, MN.IT Deputy Commissioner Jesse Oman acknowledged that tech upgrades "can be challenging" but said Minnesota "needs to step up to the challenge." Making upgrades now, Oman adds, would also make future conversion projects easier.
BUT HOW MUCH?
Dayton's $125 million request is expensive, both in its own right and as a share of the projected $1.65 billion budget surplus. And there will be lots of rivals for the cash.
Republicans like Senate Majority Leader Paul Gazelka say they want "as much as possible" of the surplus for tax cuts - while also funding education and transportation. Dayton's pushing hard for money for conservation, preschool, higher education, rural broadband and other topics.
But in Minnesota's divided government, tech upgrades are a matter that has support in both parties - even if people don't always agree on exactly how much to spend. Even as Dayton pushes for more tech funding, Republican House Speaker Kurt Daudt of Crown is pushing an "technology initiative aimed at modernizing state government."
Complicating matters is that it isn't just one decision about how much to spend on IT. Each part of the budget - education, health and human services, public safety, environment and natural resources and more - goes through a different committee. And each committee gets to decide how much of its budget share goes to technology.
Some key lawmakers are looking skeptically at putting more money into IT. Rep. Sarah Anderson, R-Plymouth, said she's not convinced MN.IT has achieved all the efficiencies it should have by consolidating state IT functions into a single agency. That makes it "more challenging" for her to support millions of dollars in additional funding for cybersecurity even though she agrees that improving the state's cybersecurity is a key goal.
"We're not getting the full information, and we need to have that to make sure we're not making bad decisions," said Anderson, whose budget committee is considering a $22 million cybersecurity request from MN.IT.
Oman said MN.IT is saving around $15 million per year through consolidation.
"We agree with Chair Anderson on the importance of making Minnesota more efficient," he said. "This proposal will help get the state there faster."
Sen. Michelle Benson, R-Ham Lake, said the key to securing funding is a "champion" for it on legislative budget committees. She oversees the Senate's health and human services budget committee and said she'd try to be that champion - but can do only so much.
"I have a big budget I have to manage," Benson said. "I will try to make room for IT and security priorities within that budget. I hope that other chairs look at it similarly."