«RE S E AR CH RE P O R T Varsity Blues Are High School Students Being Left Behind? Kristin Blagg Matthew M. Chingos May 2016 AB O U T T HE U R BA N I ...»
EDUCATION AND TRAINING
RE S E AR CH RE P O R T
Are High School Students Being Left Behind?
Kristin Blagg Matthew M. Chingos
AB O U T T HE U R BA N I NS T I T U TE
The nonprofit Urban Institute is dedicated to elevating the debate on social and economic policy. For nearly five
decades, Urban scholars have conducted research and offered evidence-based solutions that improve lives and strengthen communities across a rapidly urbanizing world. Their objective research helps expand opportunities for all, reduce hardship among the most vulnerable, and strengthen the effectiveness of the public sector.
Copyright © May 2016. Urban Institute. Permission is granted for reproduction of this file, with attribution to the Urban Institute. Cover image by Nancy Honey/Getty Images.
Contents Acknowledgments iv Executive Summary v Varsity Blues: Are High School Students Being Left Behind? 1 How to Use NAEP Data 3 Data on High School Achievement Are Limited 7 Hypotheses for Fade-Out of NAEP Achievement Gains in High School 9 Cohort-Adjustment Hypothesis 10 Marginal-Graduate Hypothesis 11 Senioritis Hypothesis 14 Measurement Hypothesis 15 The High School Conundrum 17 Conclusion and Recommendations 18 Appendix A. Data and Methodology 20 LTT NAEP Variables 20 LTT NAEP Adjustment Methodology 21 Main NAEP Variables 21 Main NAEP Adjustment Methodology 23 Main NAEP Robustness Analysis 23 Notes 29 References 31 About the Authors 33 Statement of Independence 34 Acknowledgments This report was funded by an anonymous donor. We are grateful to them and to all our funders, who make it possible for Urban to advance its mission.
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders. Funders do not determine research findings or the insights and recommendations of Urban experts. Further information on the Urban Institute’s funding principles is available at www.urban.org/support.
The authors thank Tom Loveless and Martin West for helpful comments on drafts of this report.
IV ACKNOWLEDGMENTSExecutive Summary Efforts to improve the academic quality of US schools often leave high schools behind.
Federal accountability policies require annual testing in math and reading for grades three through eight but only once in high school, where schools are also held accountable for increasing graduation rates. The relative lack of attention to secondary schools has coincided with disappointing national student achievement results for high school students, even as achievement among elementary and middle school students has risen significantly since the 1990s.
Has high school quality stagnated, or even deteriorated, as student achievement has increased in elementary and middle schools? Or are there other factors, such as changing demographics or declining student effort, which explain stagnant high school achievement? This report addresses these questions using student-level data from the National Assessment of Educational Progress (NAEP). We examine the results of nationally representative math and reading tests that have been administered since the early 1970s to better understand why the academic gains posted by elementary and middle school students have not persisted into high school.
The data strongly suggest that stagnant achievement among high school students is a real phenomenon. This result is consistent across different versions of NAEP and with other achievement tests and does not appear to result from changes in who is taking the test (e.g., as a result of rising high school graduation rates), flaws in test design and administration, or declining student effort.
Understanding why students are leaving high school with math and reading skills not much better than their parents awaits better data and additional research. We recommend several improvements to NAEP, including the regular assessment of high school students across the nation and in each state, as is done for younger students. There is also a critical need for researchers and policymakers to renew their focus on high schools and ensure that the academic gains that elementary and middle schools have produced are not squandered.
EXECUTIVE SUMMARY V
Are High School Students Being Left Behind?
Elementary and middle schools have long been the focus of education reform efforts. In Hamilton, Ohio, at the 2002 signing of the No Child Left Behind (NCLB) act, President George W. Bush stressed that states would be called upon to “design accountability systems to show parents and teachers whether or not children can read and write and add and subtract in grades three through eight.” Bush was speaking to a group of high school students at the time, yet high schools have received significantly less attention in federal policy. NCLB promoted annual test-based accountability for elementary and middle schools, but subjected high schools to much looser testing requirements. States were required to assess students’ mastery of math and reading standards at least once between 10th and 12th grade, a continuation of requirements from the previous reauthorization of the Elementary and Secondary Education Act in 1994. In lieu of additional annual testing, high school graduation rates were incorporated into the federal school accountability measures.
When Congress reauthorized NCLB as the Every Student Succeeds Act in late 2015, little changed in federal policy regarding high school. High schools are still required to test students once in math and reading using either a state-created or nationally recognized high school assessment test. In addition, states must include graduation-rate goals for high schools in their accountability systems.
The lack of attention on high schools, relative to elementary and middle schools, has coincided with disappointing student achievement results for high school students, even during (and following) periods of substantial increases in achievement among elementary and middle school students. These trends are alarming, especially as many students are entering higher education unprepared for college-level work (Sparks and Malkus 2013). The goal of this report is to assemble the key data sources on high school achievement and empirically examine competing explanatory hypotheses for the stagnant trends.
We draw on the National Assessment of Educational Progress (NAEP), widely regarded as the Nation’s Report Card. NAEP comprises a set of nationally representative assessments of student achievement in elementary, middle, and high schools. Since 1990, mathematics and reading assessments, known as main NAEP assessments, have been conducted about every two to four years for 4th- and 8th-grade students, and roughly every four years for 12th-grade students. In addition, a second set of tests, the long-term trend (LTT) NAEP assessments, have been administered in mathematics and reading to 9-, 13-, and 17-year-olds roughly every four years since the 1970s.
On both the main and LTT assessments, elementary and middle school students have registered substantial achievement gains over time. However, national high school student achievement (12thgrade students and enrolled 17-year-olds, most of whom are in 11th grade) has remained largely stagnant. For example, while 9- and 13-year-olds saw increased performance on the LTT mathematics assessment by 0.71 and 0.54 standard deviations from 1978 to 2012, the performance of 17-year-olds inched upward only 0.17 standard deviations (figure 1). Even more worrying is that achievement for younger students has increased markedly since the early 1990s while high school achievement has been flat.
FIGURE 1 LTT NAEP Mathematics Score Changes, in 1978 Standard Deviations Score change on LTT NAEP since 1978 0.80
This trend has puzzled education researchers and policymakers alike. Given the positive trends in student achievement among younger students, we should expect that at least some of those academic gains would persist into high school. Has high school quality stagnated, or even deteriorated, as student achievement has increased in elementary and middle schools? Or are there other factors, such as demographics or declining student effort on low-stakes exams, that explain stagnant high school
2 VARSITY BLUES: ARE H IGH SCHOOL STUDENTS BEING LEFT BEHIND?achievement? We address these questions using student-level, restricted-use main and LTT NAEP data, which allow us to investigate the underlying trends in student populations that may affect overall student achievement.
The infrequency of NAEP administration makes it difficult to track student achievement at the high school level—an important problem in its own right. But the available data strongly suggest that stagnant achievement among high school students is a real phenomenon, and it cannot be explained away by rising high school graduation rates or problems with how the tests are designed and administered. The NAEP tests for high school are not perfect measures of academic achievement, but they are consistent across tests (main versus LTT) and with the results of other national achievement tests.
How to Use NAEP Data NAEP is too often treated like a public opinion poll, the results of which are tea leaves that can be read by analysts and commentators interested in how school quality varies across places and over time. But NAEP reflects much more than just the quality of schools. In particular, NAEP scores reflect the underlying population of students taking the test. Schools in Massachusetts serve a different student population than schools in Mississippi, and the student population in 2016 looks different from the student population in 1990 or 1980.
Given the changes in the nation’s student population over time, an assessment of NAEP-score trends should also account for underlying demographic trends. Such an adjustment allows national trend scores to reflect how demographically similar students perform academically over time, assuming that the relationship between demographics and test performance is constant over the period analyzed.
The adjusted trends do not isolate school quality, as they still reflect changes in non-education policies, culture, and unmeasured student characteristics, but they are a marked improvement over simply eyeballing the raw scores.
The US student population has changed substantially since the start of NAEP testing in the 1960s and 1970s, with most of these changes occurring since the start of main NAEP testing in the early 1990s (figure 2). In particular, the nation has seen an increasing Hispanic share of the population (and decreasing white share). In addition, the percentage of students identified as eligible for free and reduced-price lunch and as eligible for special education services has risen considerably over time.
Appendix table A.1 shows that the increase in the Hispanic share was accompanied by increases in the
Note: Demographics averaged across 4th- and 8th-grade main mathematics and reading NAEP tests. We do not use 12th-grade data because 12th-grade students are assessed less frequently Some of these trends are the result of national demographic shifts, but others may be at least partly driven by policy changes and school decisionmaking that affect how demographics are measured. For example, over the past decade, eligibility criteria for the National School Lunch and School Breakfast programs have undergone significant change. States now directly certify students from households who receive Supplemental Nutrition Assistance Program benefits, and, starting in 2010, districts began to adopt community eligibility, allowing schools to grant free lunch to all students if 40 percent or more would qualify through direct certification (Moore et al. 2013; Neuberger et al. 2015, note 2).