A quarterly magazine of urban affairs, published by the Manhattan Institute, edited by Brian C. Anderson.
• • • • • • • • • • • • • • •
New Yorks Lake Wobegon Effect
The states rosy test scores dont square with reality.
26 June 2008
If New Yorks commissioner of education, Richard Mills, is to be believed, one of the great success stories in the history of American public education is unfolding in the Empire State. The commissioner has released 2008 state test results showing that a stunning 97 percent of the 708 third-graders in upstate Warren County are achieving proficiency in math. Only five of the countys third-graders scored at level 1, defined by the test protocol as reflecting serious academic difficulties. The states other third-graders arent doing quite as terrifically as those in Warren County, but theyre pretty closewith 90 percent demonstrating proficiency in math and only 2.3 percent of all children scoring at level 1. For all New York students in grades three through eight, the pass rate in math this year is 81 percent, up 15 percentage points since 2006. Pass rates in reading have also risen dramatically over the past two yearsup 7 percent for the state overall; 6.9 percent for New York City; 12.4 percent for Buffalo; 8.2 percent for Rochester; and 8.1 percent for Syracuse.
If these test results, particularly in math, are a true reflection of New York students mastery of subject matter, educators should be rushing here from all over the country to learn how our teachers manage to do it. U.S. Secretary of Education Margaret Spellings might also consider visiting the town of Lake George, in the center of Warren County, to congratulate the children there for being the first to achieve the No Child Left Behind Acts goal of total proficiencyand six years ahead of the target date of 2014, at that. Lake George could then qualify as the real-life version of Garrison Keillors fictional Lake Wobegon, where all the children are above average.
Alas, New York States and Lake Georges test scores are not to be believed. Spectacular improvements of the kind Mills claims just dont happen within the space of two years for a cohort of over 1.2 million students in hundreds of school districts with very different demographics. Some extraordinary individual schools do sometimes beat the odds, but nothing like these across-the-board gains has ever been demonstrated in any independent achievement tests, such as the SAT or ACT. Moreover, almost none of the dramatic improvements in the state tests show up in the most recent tests administered by the federal National Assessment of Educational Progress (NAEP), otherwise known as the nations report card. NAEP scores in fourth- and eighth-grade reading and eighth-grade math in New York State remained flat from 2005 to 2007. There was a modest improvement in fourth-grade math, but nothing close to the gains reported on the state tests. (The next NAEP test is due in 2009.) The inescapable conclusion is that a significant portion of New Yorks improvement in test results from 2006 to 2008 was created by the test factor itselfthat is, by the degree of rigor (or lack thereof) of the specific state tests administered to the children in those years.
Knowing that last years NAEP results sparked some critical commentary (including in the New York Times) about the state tests validity, Mills tried to preempt skepticism at his press conference announcing the 2008 results. One of the slides in his PowerPoint presentation was titled ENSURING THESE RESULTS ARE ACCURATE and claimed that New Yorks testing system passed rigorous peer review by [the] U.S. Dept of Education. But this rigorous peer review, which all 50 states now undergo under the No Child Left Behind Act (NCLB), is less impressive than it sounds. I was told by a federal education department official that the review covers only the general process used by the states in establishing a reliable system of standards and assessment. It does not constitute a federal seal of approval for the accuracy of any states particular tests.
Further, the federal review, though well-intentioned, appears to be producing its own Lake Wobegon effect. The review has already approved 32 states, is about to approve three more, and is still examining the remaining 15 but will eventually approve them, the official told me. Sometime soon, then, the U.S. Department of Education will certify that all 50 states, plus the District of Columbia, have reliable systems for administering the tests that are at the heart of NCLBs accountability measures. But many state tests will show results that, like New Yorks, suggest grade inflation, and there will still be a yawning gap between what many states report about their students academic abilities and what the NAEP tests reveal for those states. All of this will produce more fodder for NCLBs critics on both the left and the right.
The premise of NCLB, as of so many current education reform efforts, is that schools must serve the interests of children, not the interests of the adults who work in the system. But in a classic case of unintended consequences, the widespread test inflation produced by NCLB is serving only the interests of the adults. New York education officials like Mills, New York City mayor Michael Bloomberg, and his schools chancellor, Joel Kleinalong with teachers union leaders like Randi Weingartenadvance their varied agendas in the glow of inflated test scores. But the children are the big losers. Sometime in the next decade, the white children of Lake George and the black children of New York City will come face to face with reality. On a high school math Regents testor on an SAT test, or in a college remediation coursethey will discover that they are not quite as proficient as New York State once assured them.
Sol Stern is a contributing editor of City Journal and the author of Breaking Free: Public School Lessons and the Imperative of School Choice.