What do these stories have in common? Success here derives from access to data, or big data as it’s sometimes called. The examples above come from the Data Quality Campaign, a nonprofit effort driving education outcomes through hard numbers.
Data matters. While state administrators have long had access to school district statistics, recent years have seen greater reliance on hard numbers as a means of driving education policy. CIOs have found that the ability to gather and analyze school stats gives them greater sway in the shaping of school policy and practice.
As is common practice, the CIO’s office in Georgia’s Department of Education annually aggregates student performance data as a requirement for tapping into federal funding. Besides giving the school system access to government financial support, aggregation at the state level serves another purpose: It relieves the IT burden at the district level.
“It costs money to hold onto historical information,” said Robert Swiggum, CIO of the Georgia Department of Education.
The aggregated, or longitudinal, data set also gives educators a 360-degree view of a student’s performance at the push of a button. Rather than sift through and collate individual stats on attendance, performance, test scores and other metrics, “they can use the [longitudinal] system as an easy-to-use format that can be sorted, that can be filtered,” Swiggum said. “It helps them deliver the personalized information, something that is almost impossible to do without the data.”
Such data drives the big picture too. “We write a report for our legislature every year to show them how all the schools are doing, by district, so they can see that the money they are appropriating is being used well,” Swiggum said. Curriculum developers can scan the data, and a public-facing portal gives parents and others a glimpse into a district’s progress.
Mark Masterson, CIO of the Arizona Education Department
In Arizona, an expanding longitudinal data system aims to influence state education policy. “The decisions that need to be made are around the school payment model from the legislature,” said Mark Masterson, CIO of the state’s Education Department. “Education is a $7.5 billion business in our state. So we need to ask: Are we paying the right amounts to the right people?”
Arizona’s core education data system came into being in 2008, developed at first by a small cadre of researchers on a $6 million budget. Since then the system has grown dramatically, and by the end of 2014 should encompass every district, teacher and principal in the state. Among Arizona’s 300 local educational authorities, some 500 tap into the system every day, and that figure is growing by 50 to 60 users a week, Masterson said.
The system is largely automated, with teachers filing student data into a state information system as often as once a day, though the law gives them 20 school days to update records. A statewide aggregator extracts that information daily and generates weekly and monthly benchmark reports.
While it may sound simple, this seamless exchange of data is not easily accomplished, as there are few truly interoperable interfaces in play. “The challenge of doing this in education is that there is no standard electronic data interface,” said Masterson. “When you walk into Wal-Mart, their systems are talking to suppliers every day. When you pick up something off the shelf, the supplier is already delivering the next one. So why can’t we do the same with children?”
Each district chooses how to manage its electronic data, and the result is that 15 vendors presently help to manage those systems, leaving Masterson the task of forging digital connections between all those enterprises. “We depend on having good, clean data, so this has been extremely painful. If I was a retail business in this circumstance, I would go out of business,” he said.
To smooth the process, Masterson does much of the procurement work for districts, who have neither the budget nor the expertise to tap into needed resources on their own. The state has spent $32 million over the past six years getting all the pieces in place.
When all the processing is done, the net result can be — and usually is — an enhanced educational experience. In the past, it might have taken a teacher five days to pull relevant academic records for 35 students. “Now it takes her five seconds,” Masterson said. “You give those teachers back their time, so they can do lesson planning.”
Longitudinal data often has direct impact on pedagogy: If a district scores consistently low in fifth-grade math, that’s something that can remedied. Data may also have an impact on individuals, for instance, when a teacher can see at the start of the year that a particular student has a long-term trend of high absenteeism.
Such data also may influence policy at higher levels, showing state leaders which schools are in need of greater funding or which are reaching their capacity in terms of student population.
In Delaware the technology gurus in the Department of Education are looking beyond primary education and even beyond high school. They hope longitudinal data can help drive the best results when it comes to student enrollment and success at the college level.
With 130,000 students in its system, the state is vigorous in its college prep agenda, setting aside funds for every high school student to take the SAT. It even has a formal relationship with the College Board, which helps merge SAT results into the schools’ own data systems.
By having ready access to SAT data, “it shows us something about the effectiveness of the schools,” said Pat Bush, director of the technology resources and data development group at the Delaware Department of Education. “Having that information as a teacher can be very valuable as a way to see where kids are and even how you can challenge them. If a child has solid engineering and math skills as demonstrated by the SAT, I as a teacher would probably try to provide supplemental materials in those areas so that that child could do more advanced work.”
Much has changed in the 10 years since the state began aggregating district data into a longitudinal view. Data has become much easier to access, Bush said, but as with any technological evolution, this has brought its own set of challenges.
“For every kink that gets worked out, there are three or four new ones,” he said. “It may be operating system updates, database technologies or reporting technologies, as well as the landscape of what data is available to us. We are always looking to stay current, but not too current. We need to balance the technology with the cost of ownership and the potential benefits.”
Advocates for data-driven education suggest a number of positive outcomes that might arise from longitudinal systems. At the Data Quality Campaign, Director of State Policy and Advocacy Paige Kowalski points to several examples:
Today more than 40 states produce a high school feedback report, showing what percentage of students go on to a two- or four-year college. Longitudinal data can drill a level deeper, for instance, by showing how many of these needed remedial courses upon entering college. “If we see that they are not being successful in their next step, then we can ask what we can do better to prepare them for that next step,” Kowalski said.
Chicago had been taking a snapshot of performance in the ninth grade to determine whether students were on track to graduate. But that was only one part of the picture. By continuing to track data through 10th and 11th grade, the school system has been able to keep tabs on those who drift off course and to offer remediation.
Longitudinal data also can help to tell a school system from an early point whether a student will likely drop out. “Kids drop out one day at a time. It begins in the first grade. There are indicators and behavior patterns, whether it is attendance or test scores or being held back. If you can put those together through a longitudinal history, then you can think about how to work with that child’s unique individual needs,” Kowalski said.
For some, the key word there is “individual.” It is of value to take an aggregated picture of a school district’s performance — few would debate the point. But when it comes to individualized data, people get squeamish. Some worry that identifiable information could compromise a student’s right to privacy, or even be used for nefarious purposes, such as using a high school senior’s identity to secure credit cards and take out loans.
Advocates and practitioners of longitudinal data say these concerns are largely unfounded, but they acknowledge that it takes planning and insight to ensure that such data stays where it ought to be.
Lonnie D. Tague
Kathleen Styles, chief privacy officer at the U.S. Department of Education
At the U.S. Department of Education, Chief Privacy Officer Kathleen Styles kicks off the discussion with a point of law, specifically the Family Educational Rights and Privacy Act (FERPA). The law says in essence that student information can’t be shared without the parent’s written consent. When it comes to longitudinal studies, data miners must comply with this basic safeguard.
Styles said she gets 12,000 calls and 2,000 emails a year from schools looking for help with FERPA as well as 300 to 400 complaints from people who feel the law has been violated.
In general, states do a solid job of securing longitudinal data, she said. That’s because most systems were constructed fresh, rather than as add-ons to legacy systems that may not have been built with privacy as a goal. “They were deliberately created with this in mind,” said Styles. “In the initial stages, these data security plans were already being put in place.”
In the vast majority of cases, data is stripped of identifying content before it is aggregated into a longitudinal format. Moreover, the people who hack Target for credit card information may simply be less interested in breaking the law just to tap into a fifth-grader’s history test scores.
Nonetheless, state education CIOs say they take careful steps to ensure privacy. There are many ways to go about it: Georgia keeps its information in a hardened data center behind multiple firewalls, in an encrypted format. For many, though, the privacy agenda largely comes down to matters of procedure.
Masterson, for example, focuses on role-based policy, narrowly defining the rights of access to individuals within the system. It can take obvious forms: Only parents can view their children’s scores, and they cannot view anyone else’s. But there may also be subtle distinctions. The principal or the district may see data from Teacher A or Teacher B, but these teachers cannot view data from each other’s students. “The challenge is in making sure the right people see only the right data,” Masterson said.
Looking ahead, some say longitudinal data still has some distance to travel before it reaches its final form. Most notably, there is a jurisdictional sticking point, said Brian T. Prescott, director of policy research at the Western Interstate Commission for Higher Education, a group that works with multiple states to leverage the value of longitudinal data.
Education is a state responsibility, and so it is the states that collect and analyze this crucial data. There’s a hazard there. “State borders are a hard and fast line for policy and data,” Prescott said. “If a student leaves the state, that record gets left behind. There is lost information in the process.”
To gain the full value of longitudinal analysis, “you have to break through the silos that have grown up around these data systems,” he said.
