MetroLab’s Executive Director Ben Levine sat down with Dan O’Brien, assistant professor of Public Policy and Urban Affairs and Criminology and Criminal Justice and co-director of BARI; Nancy Hill, professor of Education at Harvard Graduate School of Education; Eleanor Laurans, chief financial officer at Boston Public Schools (BPS); and Colin Rose, assistant superintendent of Opportunity and Achievement at BPS, to learn more.
Ben Levine: Could you please describe what the Boston Public Schools Opportunity Indexis? Who is involved in this project?
Dan O’Brien: The Opportunity Index (OI) is a place-based metric that captures inequities in academic achievement that arise from factors outside of the control of schools that Boston Public Schools (BPS) and the Boston Area Research Initiative (BARI) constructed to inform funding and programming. In addition to considering factors that are traditionally used in funding decisions, like special education status and eligibility for free-or-reduced-price lunch, the OI is distinctive in that it takes seriously the consequences of neighborhood effects (i.e., the impacts of the events and conditions in one’s residential context) for academic achievement. For this reason, elements like neighborhood crime, socioeconomic status and the academic attainment of local adults contribute to the calculation of the OI.
Nancy Hill: The OI, as Dan described, is particularly useful because it allows us to quantify the assets and challenges that children carry with them “in their backpacks.” Importantly, it evaluates not only the level of opportunity for students at all schools in the district, but also the sources of that opportunity. This gives principals and headmasters the flexibility to pursue and implement resources and programs that will best support the specific needs of their students.
Levine: Can you describe what motivated you to develop this tool?
Colin Rose: Closing opportunity and achievement gaps is the Boston Public Schools’ driving priority. It is the catalyst for the hard work we are doing to replace the structures, practices and mindsets that perpetuate inequities. Every student, in every classroom, in every school of the BPS system should have the same unconstrained opportunity to achieve the greatness within him or her. We know that students come to school with differing experiences, opportunities and needs, but as a district, we historically have lacked a nuanced way to account for these differences and mitigate inequities that are out of schools’ immediate control. The Opportunity Index is an innovative tool created to help quantify these differences so we can make decisions and allocate resources more equitably, helping to close gaps in opportunity.
Levine: What data comprises the index?
Eleanor Laurans: The Boston Public Schools Opportunity Index is a composite index that incorporates a range of data representing factors that are outside of the schools’ control, yet are predictive of students’ academic outcomes. The data includes “place-based” measures related to students’ home neighborhoods — as defined by Boston’s 177 U.S. Census tracts — as well as measures specific to individual students and their families. By rolling multiple measures into a single, more accessible metric, BPS is better equipped to direct resources and supports to the schools and students who need them most.
Levine: How did your partnership with BPS inform the project from the start?
O’Brien: BPS approached us in summer 2016 as part of a broader “listening tour” as they sought expert advice on the methodological and conceptual considerations that would need to go into the construction of an Opportunity Index. They built a pilot version on their own based on those conversations that they announced at our annual conference in spring 2017. This led to sufficient internal buy-in that they decided to undertake the same project in a more methodologically rigorous manner, and they invited us to partner with them in this effort.
Hill: Once we started the project in earnest, it was thoroughly an effort in co-creation, with a natural cycle in which joint brainstorming generated a set of data-based steps that we then executed on, followed by discussion of the results and how they would guide next steps. The project was guided primarily by BPS’ desire for an OI that accounted for both individual- and neighborhood-level factors that impact student academic achievement. BARI’s investment was in the opportunity to help them produce this cutting-edge tool and in the novel discoveries that might arise in that process.
Levine: What was the most surprising thing you learned?
Hill: The most striking part of the project — though not necessarily the most surprising — was the conflict between objective, data-driven insights and legal and institutional limitations. This sort of thing comes up often in BARI’s research-policy partnerships, but it took a particularly prominent form here around the question of race. In statistical models, race is a rather strong predictor of academic achievement, even when controlling for poverty status and other correlated factors. But it was impossible to include it as a part of the OI because of legal limitations around targeting funding specifically on the basis of racial characteristics of students. This led to a number of passionate conversations that revealed various ethical perspectives, from the aforementioned legal limitations to the ethics of how to best address these clear inequities between racial groups (“What good is a tool that ignores these issues?”) to the norms of proper statistical procedure (“How can we leave out meaningful predictors?”).
Levine: Where will this project go from here?
O’Brien: The OI will lead to partnership along two lines. First, BARI continues to be engaged in the first-year deployment of the OI and will be involved in the annual re-calculation of the OI so it can be used to inform the budget. As the OI gains internal momentum, BARI will be available to consider its extension to other applications. At a second level, the project has created a level of trust between BARI and BPS that can be applied to other efforts. This is often cited as one of the keys to effective urban research-policy collaboration — it’s about partnerships, not projects. This is already visible in BARI’s current effort to evaluate BPS’ school choice and assignment system.
The impact of the five components of the Opportunity Index on academic achievement in elementary, middle and high school students.
About MetroLab: MetroLab Network introduces a new model for bringing data, analytics, and innovation to local government: a network of institutionalized, cross-disciplinary partnerships between cities/counties and their universities. Its membership includes more than 35 such partnerships in the United States, ranging from mid-size cities to global metropolises. These city-university partnerships focus on research, development, and deployment of projects that offer technologically- and analytically-based solutions to challenges facing urban areas including: inequality in income, health, mobility, security and opportunity; aging infrastructure; and environmental sustainability and resiliency. MetroLab was launched as part of the White House’s 2015 Smart Cities Initiative. Learn more at www.metrolabnetwork.org or on Twitter @metrolabnetwork.
