Tag: outcomes

Aspirations and Academic Achievement: The Spillover Effects of Beca 18 on Educational Outcomes of Younger Students

Elena Costarelli, Rosamaría Dasso Arana, and Bárbara Sparrow Alcázar analyze the effect of being near a Beca 18 beneficiary -a new scholarship program for high school students- on the academic achievement of second grade children.

Editor’s note: This post is part of a series showcasing Barcelona GSE master projects by students in the Class of 2016. The project is a required component of every master program. 

Authors:
Elena Costarelli, Rosamaría Dasso Arana, Bárbara Sparrow Alcázar

Master’s Program:
Economics

Abstract:

Using administrative data from the Ministry of Education in Peru, we analyze the effect of being near a Beca 18 beneficiary -a new scholarship program for high school students- on the academic achievement of second grade children. Previous literature suggests that information about the potential returns to education plays an important role on students’ achievement. Our hypothesis is that having a fellow nearby might change the perception of younger children and of their parents about returns to education, thus leading them to invest more on it. We use a difference in difference approach to test this hypothesis in a school panel data setting. As we are interested in the effect of information transmission, we use GPS location data to identify which schools are near a Beca 18 beneficiary. We test for several distance specifications with consistent results. Our findings suggest a positive spillover effect of the program on younger children in both reading and math performance.

Introduction:

Access to tertiary education is a well-known motivation for students to perform better at school. Good students are usually the ones that are able to attend better universities which in turn allows them to improve their living conditions. This is true in most developed countries, where access to good quality higher education is a possibility for most students. However, in the case of many developing countries, market failures and government limitations do not allow students to consider this possibility.

When facing the decision of educating their children, many families may consider it to be an unprofitable investment. Little information about the benefits of education in terms of higher future income and the lack of success stories among people close to them may all contribute to this perception. In this regard, the impact of a program that makes access to tertiary education may possibly affect the way people value education in a significant way. If parents and children are aware that been a good student may have a tangible future reward, their investment decisions may change.

Access to tertiary education is greatly limited to children from poor families in Peru. To address this issue, the Peruvian government recently created the Beca 18 program. Beca 18 is a merit/need based scholarship program that targets students applying to higher education institutions such as universities and technical institutes. The program gives selected students the opportunity of attending the best tertiary schools in the country. Before the program existed, even access to public universities was very limited. Beca 18 can be considered as one of the first real opportunities for children of low resources in Peru to access high quality tertiary education.

Current literature suggests that there is a positive relationship between policies that increase the perceived returns of education and educational outcomes among children. There is also evidence that supports that future access to scholarships and merit based programs may encourage better school performance. In this paper, we will analyze the impact of Beca 18 on the school performance of second grade children. To do this, we use test score data from the Ministry of Education and administrative records from the Beca 18 program. Using a difference in difference approach, we were able to identify a positive impact of being near a program beneficiary on both math and reading proficiency outcomes. 

Conclusions:

Our results suggest that the Beca 18 program has relevant spillover effects on the educational outcomes of younger children. Guided by our conceptual framework, we would expect this result to be a consequence of the fact that children and parents exposed to Beca 18 beneficiaries update their information about perceived returns to education, leading them to invest more time and resources in obtaining better educational results.

We also find that effects on math test scores are stronger and more robust to several specifications than effects on reading test scores. This result is consistent with findings of other studies suggesting that math scores are more quickly affected by changes in study behavior. We also find that the effect of being near a beneficiary decreases as the distance to the school of the beneficiary increases. This result is consistent with our hypothesis that the improvements in educational outcomes are a result of information transmission.

Another result worth discussing is that we found that the effect of the program is larger when the number of beneficiaries nearby increases. This suggests that investment decisions are affected by how likely it is to get the scholarship. It may also suggest that the investment decision may vary if there are more people acting as role models.

Overall, our results are relevant from a policy perspective. We present evidence that the program has relevant spillover effects that should be considered when evaluating its benefits. As public programs in Peru are under continuous scrutiny, further evidence that supports the program’s effectiveness is of greatly useful to ensure its continuity. Also, as our results suggest that the number of beneficiaries matter for investment decisions, the expansion of the program could lead to even greater spillover effects.

It is still important to note that the effects found here are not the main intended effects of Beca 18: the scholarship program was designed as a supply side policy intervention. Our findings support the idea that this program can have important demand side effects worth considering. We would also expect for these effects to increase over time: the success stories of current beneficiaries in the labor market could lead to an even greater increase of the expected returns of education.

Defining data for decision-making

authorBy Benjamin Anderson ’15 (Master’s in Economics of Public Policy).

Ben is a Data Strategist for Made in Durham, a non-profit organization in North Carolina (United States) that works to improve education and career outcomes for local youths.

This article originally appeared on Made in Durham’s website.

In the past few weeks, there have been a barrage of media reports about educational achievement and, more generally, life outcomes for the youth of Durham.

The positive news is that these issues are receiving attention, but the downside is that the reports may be more harmful than helpful. At its best, data optimizes decision-making, but at its worst data can be deceptive and divisive.

Specialized knowledge is required to leverage data for decision-making, whereas selectively reporting figures requires some effort but no expertise. In the latter scenario, the ambiguity of statistical assumptions predisposes the audience to personal, as well as, framing bias. Those who go through the effort to produce data often have an agenda, and therefore, have incentives to make claims which imply causes and solutions. Data is dangerous when misused. It can create tension, undermine trust and unity, and result in costly adverse decision-making.

One key characteristic of amateur statistics, aside from lacking an experimental design, is that they do not account for the fact that outcomes are a function of many different variables. For example, schools clearly play a crucial role in influencing academic attainment, but a report drawing relative comparisons between attainment outcomes within or across cities usually implicates unidentified failures of one school district versus another while all but ignoring the effects of transportation, affordable housing, food, healthcare, and social support accessibility, as well as people’s different lived experiences, including traumatic exposure of various kinds.

Reactivity to outcomes is strongly linked to bias and emotion. Making decisions about problems and solutions based exclusively on outcomes is the logical equivalent to going with your gut.

Descriptive statistics alone have a tendency to reinforce what we already think we know rather than helping us to gain an objective understanding of the issues because we often overestimate our understanding of the context. Shards of truth may be buried in our presumptions or between the different storylines, but other times the truth isn’t within sight.

If one wanted to know what public schools are doing right and what positive changes could be made, the reported outcomes would not meaningfully increase understanding. This would be like a college basketball coach using the Ratings Percentage Index (RPI) to make game plans. The RPI is simply a function of outcome variables that are influenced by other, more influential variables over a team’s success, such as shot selection, rebounding, ball control and many others.

Similarly, objective inference about the determinants of academic achievement are impossible when we simply have some measure of the output, like grade level proficiency, graduation rates or achievement gaps. Summarized outcomes do not even begin to untangle the multifaceted causal factors of student achievement, or even point to which factors are within the schools’ control and which are shaped by other institutions that govern infrastructure, real estate development, credit markets and criminal justice.

Good intentions often lead to unintended consequences. Calculating outcomes or deriving slightly new definitions of them does not enhance the cultural or intellectual competence of our community, its citizens or the institutions within it.

This is troubling because the extent of harm done with every report that subjectively frames and selectively reports data will never be known. A symptomatic obsession can enable data to have a negative social impact, leading to the proliferation of economic and racial segregation, adverse selection of people and funds from public schools, victim blaming and the marginalization of objectivity. The focus needs to shift from symptoms to solutions.

Data should be collected and analyzed in a way that enables us to separately identify effects on outcomes, including those determinants within the school’s control and those outside, so that all can be addressed in order of impact and feasibility. Robust evaluations should yield insight, pointing out specific causal factors that affect outcomes that the schools, nonprofits policy and citizens can address.

Applying a scientific lens to social issues transforms data from punitive to instructive. Careful investigation using valid quantitative methods can help us gain an understanding of the inferences that the data will and will not permit. Through empirical analysis, we have the opportunity to disentangle the effects that different factors have on certain outcomes. This is powerful because it enables us to create informed strategies.

Subsequently, when we know how our potential actions will affect an outcome, a cost-benefit analysis can help decide which evidence should be brought to action. Operating in the public and nonprofit sectors, the cost-benefit analysis goes beyond fiscal considerations to examine social returns. Combining these empirical tools puts us in a position to optimize social welfare. Data or analysis vacant of these characteristics will result in suboptimal decision-making.

An empirical basis for decision-making that respects the complexity of determinants on outcomes and the tradeoffs between various actions or lack of action should be utilized at all levels – from the systemic to the programmatic. A symptomatic focus and a preoccupation with a single area will not result in systemic improvement. As institutions, organizations and programs, our goal should be to improve, which can only be achieved through learning.

Durham has great potential to grow while enhancing the well-being of all, including the most marginalized. Continuous improvement requires the commitment of people in the public, private, and social sectors to work together.

Part of analytical integrity is the acknowledgement that sometimes our data tells us nothing at all. If we truly care about addressing systemic issues, lack of information is a strong argument for why we should build more robust datasets that incorporate variables across institutions and the socio-economic environment. This requires a willingness to coordinate and to learn. Importantly, these actions imply the willingness to change.

The Made in Durham partnership exists to address issues of the highest importance. It is the job of data is to increase the role of evidence in the partnership’s decision-making, and because of the gravity of these decisions, I also feel an ethical accountability to this work.

If we aren’t asking the right questions, data can lead to costly decisions that undermine improvement. As members of the community, we should all be able to ask the right questions to hold decision-makers accountable to analytical standards that drive improvement.

Regardless of what the outcomes show now, or anytime in the future, what we should be asking is: what are the causes of these outcomes, what are their magnitudes, and thus, what can we do to improve.