Dana Goldstein

Policy Implications of the New Teacher Value-Added Research

cross-posted at The Nation

Last month, economists at Harvard and Columbia released the largest-ever study of teachers’ “value-added” ratings—a controversial mathematical technique that measures a teacher’s effectiveness by looking at the change in his students’ standardized test scores from one year to the next, while controlling for student demographic traits like poverty and race.

Raj Chetty, John Friedman, and Jonah Rockoff analyzed the test scores and family tax returns of 2.5 million Americans over a twenty-year period, from 1989 to 2009. The team concluded that students who have teachers with high value-added ratings are more likely to attend college and earn higher incomes, and are less likely to become pregnant teens.

In a rare instance of edu-wonk consensus, both friends and skeptics of standardized tests are praising the study as reliable and groundbreaking. Indeed, these findings raise several interesting questions about how to evaluate and pay teachers—one of the most controversial topics in American urban politics. In his annual State of the City speech last Wednesday, New York Mayor Mike Bloomberg cited the new research as he promised annual bonuses of up to $20,000 for teachers rated “highly effective,” based partially on value-added measures and partially on principals’ judgments. In a move that befuddled many casual observers of the education debate, the New York City teachers’ union, the United Federation of Teachers, immediately opposed the proposal.

If we now know teacher effectiveness has a real, measurable impact on both student academic achievement and life outcomes like teen pregnancy, why aren’t teachers’ unions supporting plans to pay teachers with high value-added ratings more money? Pundits like Nick Kristof and the Daily News editorial page have jumped in to claim that the new research justifies merit pay plans like Bloomberg’s, and the oneinstituted by former chancellor Michelle Rhee in Washington, DC.

The policy implications of the Chetty, Friedman, and Rockoff paper are, however, far from clear. As the researchers note in their conclusion, their study was conducted in a low-stakes setting, one in which student test scores were used neither to evaluate nor pay teachers. In a little-noticed footnote (#64) on page 50, the economists write:

even in the low-stakes regime we study, some teachers in the upper tail of the VA [value-added] distribution have test score impacts consistent with test manipulation. If such behavior becomes more prevalent when VA is actually used to evaluate teachers, the predictive content of VA as a measure of true teacher quality could be compromised. [Emphasis added.]

The importance of this caveat cannot be overstated. As I’ve written in the past, there is evidence of increased teaching-to-the-test, curriculum-narrowing and outright cheating nationwide since the implementation of No Child Left Behind, which put an unprecedented focus on the test scores of disadvantaged children.

Despite these concerns about testing, the United Federation of Teachers has agreed in principle to a new evaluation system that depends in part on value-added; a similar system, after all, is already in place for determining whether teachers earn tenure. Negotiations between the union and the city are stalled not because, in the words of the Daily News, the union has “placed protecting the jobs of incompetents over the future financial well-being of children,” but because the union would like teachers who receive an “unsatisfactory” rating under the new system to have the right to file an appeal to a neutral arbitrator. Currently, the city Department of Education determines whether to hear appeals of teacher evaluations, and it rejects 99.5 percent of the appeals filed.

Given the widespread, non-ideological worries about the reliability of standardized test scores when they are used in high-stakes ways, it makes good sense for reform-minded teachers’ unions to embrace value-added as one measure of teacher effectiveness, while simultaneously pushing for teachers’ rights to a fair-minded appeals process. What’s more, just because we know that teachers with high value-added ratings are better for children, it doesn’t necessarily follow that we should pay such teachers more for good evaluation scores alone. Why not use value-added to help identify the most effective teachers, but then require these professionals to mentor their peers in order to earn higher pay? That’s the sort of teacher “career ladder” that has been so successful in high-performing nations like South Korea and Finland, and that would guarantee that excellent teachers aren’t just reaching twenty-five students per year but are truly sharing their expertise in a way that transforms entire schools and districts.

Beyond Pink vs. Blue: Why Gendered Toys Really Matter

Peggy Orenstein writes in the New York Times today about how various toy stores and toy manufacturers are navigating the minefield of gender and play. Parents of young children often marvel that, despite their own egalitarian intentions, their kids are the ones who police traditional gender norms. Indeed, as Orenstein notes, studies of primate and human toddlers found that while both sexes enjoy stuffed animals and books, boys prefer cars and balls, while girls are drawn to dolls. I myself have an embarrassing childhood memory of being distraught when given the gift of a remote-control airplane; my parents had to remind me to say thank you and then encourage me to play with it—and, of course, it turned out to be a lot of fun.

Orenstein points to research finding that children raised in households that practice and preach gender egalitarianism make better romantic partners as adults. But there are other reasons to encourage girls especially to play with stereotypically male toys. Research shows that boys get their first computers at younger ages than girls, and are more likely to become expert at video and computer games and to play with toys (like Legos) that develop spatial reasoning skills. This matters because all of these childhood activities are correlated with eventually pursuing careers in science, technology, engineering and math—the STEM sector, which over the past decade has created three-times as many jobs as non-STEM fields. According to the Commerce Department, though women currently hold less than one-quarter of all STEM jobs in the economy, those who do work in STEM fields earn 33 percent more than women in non-STEM jobs.

Laura Reasoner-Jones is a Virginia elementary school computing teacher who enters teams of girls in  FIRST LEGO League, in which children compete to construct and program robots. She says parents should actively encourage their daughters to get over the “ick” factor many girls associate with traditionally “boy” activities, such as interacting with machines and building things. “Girls should be encouraged to go out and take apart the lawn mower; take the grass off the blades and see how it works. Parents can start with that.”

Sylvia Martinez, an expert on educational technology, has written about how all children need to reinforce math and science concepts through “tinkering”—interacting with the physical world, as opposed to just learning at their classroom desks. (For example: collecting water samples to test pH levels, or reinforcing math concepts by learning basic computer coding.) It doesn’t work, Martinez says, “to explain everything to kids without them having any basis in experience. I’m trying to expand the idea of ‘tinkering.’ It’s not just going down to the basement and playing with stuff. You can play with data, ideas, equations, programming.”

Parents can foster this type of experimentation at home, but schools should also do their part. The problem is that in an age of increased focus on standardized test scores in reading and math, many schools are canceling computing and science courses or cutting down lab time.

“We’ve created math and science in school as very abstract,” Martinez says. “We’ve taken away a lot of hands-on experiences from kids in favor of testing. We’ve reduced a lot of science to vocabulary, where kids are being given vocabulary tests about the ocean instead of going to the ocean or looking through a microscope at organisms. If we taught baseball the way we taught science, kids would never play until they graduated.”

When schools fail to spark children’s interest in science, math, and computing, the result is that populations that have historically been drawn to those fields—the sons of college-educated parents—continue to excel, while girls and low-income kids lag behind. The toys kids play with at home matter, and so do the lessons children learn at school; in order to overcome overwhelming cultural conditioning to the contrary, both parents and educators should actively send the message that all children will have fun and learn a lot when they “tinker” in the physical and electronic worlds.

cross-posted at The Nation

The Trouble with National Social Studies Standards, and how the Common Core Might Help

Education Week reports on social scientists who complain that the new Common Core curriculum standards replicate the problems of No Child Left Behind, by shunting civics and history in favor of English and math. 

I share the concern about the sorry state of civics and history education in American schools, but this is actually one reason why I'm cautiously optimistic about the Common Core. The Common Core English standards call for 50 percent of all reading assignments to be non-fiction and "informational;" currently, according to David Coleman, the lead architect of the new standards, about 80 percent of all reading assignments are fiction or memoir. So if schools and teachers take the Common Core suggested reading lists seriously, students will gain  a lot more exposure to great thinkers and ideas in civics, history, and the social sciences. Here's a sampling of what the Core suggests they read:

K-1: How People Learned to Fly

2-3: Martin Luther King and the March on Washington; A Medieval Feast

4-5:  We Are the Ship: The Story of Negro League Baseball; Let’s Investigate Marvelously Meaningful Maps;  The Kid’s Guide to Money

6-8:  Preamble and First Amendment to the United States Constitution; Freedom Walkers: The Story of the Montgomery Bus Boycott; A Short Walk through the Pyramids and through the World of Art

9-10: "Gettysburg Address;" "Letter from Birmingham Jail;" Bury My Heart at Wounded Knee: An Indian History of the American West;  Before Columbus: The Americas of 1491

11-12:  Common Sense; "Politics and the English Language;" Democracy in America; "Delcaration of Sentiments" by the Seneca Falls Conference

English class reading lists aren't a substitute for high-quality history and civics courses, but they do foster seriousness toward these subjects across the curriculm. And remember: There are pragmatic political reasons why the Common Core isn't explicitly crafting history standards. Every time the country or even an individual state tries to come to any sort of agreement about how to teach history, we become embroiled in culture war nonsense. It happened in the 1990s with the National History Standards, which were written by a staggeringly broad array of scholars and teachers, yet attacked by Lynne Cheney et al as a font of identity politics liberlism. In Texas last year, the state Board of Education approved what can only be called a jingoistic, Christian conservative history and economics curriculum.

The United States remains an outlier among developed world nations in not articulating national standards for history. But with our politics as polarized as they are, and with the Republican Party particularly skeptical right now toward any federal attempt to regulate or even influence local schools, I just don't see the next few years as the time when we'll break through this barrier.

drawing depicts Elizabeth Cady Stanton speaking at the Seneca Falls Conference for women's rights, July 19, 1848

An Explosion in Lobbying around For-Profit K-12 Programs

My friends at The Nation have published a really fascinating investigative piece by Lee Fang about the explosion in lobbying around K-12 for-profit, virtual education, particularly targeting charter schools. This is a sector of the education world that's very difficult to track, because every state has different, often vague laws about how for-profit companies can be involved in public education.

As of 2008, 13 states allowed charter schools to partner with for-profit companies for facilities, 17 states allowed partnerships for services, and three states actually allowed for-profit companies to directly open a charter school. But even in states where for-profits are not allowed to hold a charter themselves, they are sometimes allowed to spin-off an affiliated non-profit to hold the charter, which will then contract with the for-profit for services. (This is prevalent in Ohio). Here in New York, no charter opened after 2010 can be managed by a for-profit, but in Michigan, 80 percent of charters are for-profit.

What do for-profit education services for public K-12 schools look like? Sometimes they are online-learning classes or tutoring services offered to students enrolled in brick-and-mortar schools. Sometimes they are completely virtual schools, like the one described by Katherine Boo in her 2007 New Yorker piece about the collapse of Manual High School in Denver:

Ashley, who had been accepted into a small, competitive program at another public high school, was uneasy, too, and, anyway, there were flyers at Wal-Mart about a publicly funded online charter school a few blocks from home. One of the people involved with the program had been a Denver Nugget, and his daughter was the R. & B. singer India.Arie. Students did their work on the Internet, and it was graded by teachers in an office somewhere else. Plus, they could train to be nurses or doctors, or something; the details weren’t clear. Still, after a stressful year, the chance to stay near home, with Internet access and relational proximity to India.Arie, seemed soothing, so two of Manual’s star students changed their plans.

As Fang notes, online learning can be an important supplement to real-world classrooms; I think this is especially true for advanced high school students, who should be given the chance to jump ahead in the curriculum, perhaps through video lectures from college professors. The problem is that the studies we have of typical online learning outcomes haven't shown very impressive results. Fang writes:

A recent study of virtual schools in Pennsylvania conducted by the Center for Research on Education Outcomes at Stanford University revealed that students in online schools performed significantly worse than their traditional counterparts. Another study, from the University of Colorado in December 2010, found that only 30 percent of virtual schools run by for-profit organizations met the minimum progress standards outlined by No Child Left Behind, compared with 54.9 percent of brick-and-mortar schools. For White Hat Management, the politically connected Ohio for-profit operating both traditional and virtual charter schools, the success rate under NCLB was a mere 2 percent, while for schools run by K12 Inc., it was 25 percent. A major review by the Education Department found that policy reforms embracing online courses “lack scientific evidence” of their effectiveness.

So there's good reason to proceed with caution, which is difficult to do when folks like Jeb Bush are traveling the nation advocating for absurd laws like one recently passed in Florida, which requires every single public high school student to enroll in at least one online course before graduation. This is certainly putting the cart--technological "innovation"--before the horse, which should be a quality education provided by effective teachers.

The 99 Percent, the 1 Percent, and the Bloomberg Education Agenda

 

The folks at The Awl have published my longish essay on the Occupy movement, Mayor Bloomberg, and the public schools. Since I attended the education reform "teach-in" at Zuccotti Park last Friday, I've been thinking through whether the 99 percent/1 percent critique applies to school reform, as Bloomberg critics argue. Here's part of what I concluded:

The trouble with this narrative comes in comparing education reformers with greedy bankers. The dominant ethos of the school choice/Bloomberg/Obama reform movement is one borrowed not from Wall Street, with its desperate lust for profit, but from Silicon Valley, with its commitment to meritocratic innovation that—yes, of course—earns money, but also serves the public.

There's a lot more in the piece, so please check it out. 

Thoughts on My Trip to Mexico City and the Drug War

 

View of Mexico City from the home office of Ricardo Salinas, the retail/media/banking tycoon.

I spent the first half of this week in Mexico City, on a junket sponsored by Grupo Salinas. There's a lot of worry about Mexico's violence--and I have some of my own thoughts on the topic, below--but I didn't feel at all unsafe. Mexico City is an architecturally beautiful town with a lovely culture of late-night, outdoor dining and delicious street food. Swaths of the city reminded me of the immigrant neighborhoods of south Los Angeles, cut by smog and gridlock, but home to vibrant local business strips and cohesive communities. Indeed, Mexico's economy has been less hard-hit by the financial crisis than ours has.

Our group of writers and policy wonks met with a really interesting list of journalists and politicians, including three presidential candidates. My take on Mexico's upcoming election and the country's education challenges are over at The Nation in a new column; in short, I was surprised by just how similar the rhetoric on Mexican school reform sounds--there's a big focus on the excesses of the teachers' union, for example--even though Mexico is confronting an educational inequality crisis far more serious than our own, with the average Mexican dropping out after just eight years of formal schooling. 

So if you want to learn more about education in Mexico, click here.

More broadly, the trip left me all fired up to legalize drugs. About 45,000 Mexicans have been murdered since President Felipe Calderon escalated and militarized the drug war in 2006, with political, financial, and tactical support from the United States. The drug cartels are conducting terrible massacres, and intimidating and sometimes killing both professional and citizen journalists. 

All of this is happening despite the fact that "the war" has successfully led to the deaths and arrests of dozens of top drug lords. In their absence, some of the major trafficking cartels split into smaller, more agile organizations that can better resist detection. These small groups are now fighting among themselves, which has (of course) contributed to the increase in violence.

Why isn't the drug war working? Because it isn't addressing the demand side of the equation: Americans' unquenchable appetite for illegal cocaine and marijuana. The only way to decrease demand is to begin treating addiction primarily as a public health problem, not a criminal one--as Portugal has done--and to legalize the production and sale of less dangerous drugs, like pot, to tamp down on the violent narcotics black market.

Ending the drug war would also revitalize American inner city neighborhoods, where the existence of this underground economy disincentivizes education and work, and sends 17 percent of black men to prison. 

In July the Los Angeles Times' published a riveting four-part series about the regular Americans and Mexicans caught up in the massive Sinaloan drug cartel. I highly recommend it, and think it could win a Pulitzer: Read parts one, two, three, and four.

The "Parenting Problem" is a "Poverty Problem"

"We have a parenting problem, not a poverty problem," Mike Petrilli writes at Flypaper. I agree that parenting matters greatly to a child's academic success or failure, and in fact may be the single largest determining factor. But Petrilli concludes that the best way to solve the "parenting problem" is through cultural messaging promoting marriage and stigmatizing divorce, so that kids benefit from growing up in two-income households. This ignores, I think, the concrete reality of life in many low-income neighborhoods, where many women are making a rational choice when they remain single.

Here in New York, for example, only one of every four young black men has a job. The communities from which these men hail have also been decimated by the drug war; 17 percent of adult black males have been incarcerated, compared to 2.6 percent of white men. In addition, low-income women, regardless of race, are three times more likely to experience violence from an intimate partner.

In other words, as sociologists Kathryn Edin and Maria Kefalas demonstrate in one of my favorite books, Promises I Can Keep, low-income women often prefer to remain unmarried because the men in their lives--men facing chronic unemployment in the legitimate economy, or who may be addicted or engaged in criminal behaivior--simply do not make stable husbands or fathers. 

If we want to get to the root causes of the "family values" issues in poor neighborhoods, we need to think not only about culture, but take a broad approach to social and economic policy-making, one that reforms the drug war, creates jobs, and, yes--educates people of all ages about the benefits of delaying childbearing and forming strong marriages. 

In Which I Cite My Sources in an Attempt to Deflate the Hot Air from the Teacher Quality Debate

There is so much hot air in education reform, and it's extremely frustrating when one's arguments and supporting research are misconstrued. 

RiShawn Biddle writes the following:

A penchant among far too many education writers who embrace the Poverty Myth of Education is to oversimplify the debate over the role of education in stemming the long-term effects of poverty. First, they argue that school reformers proclaim that education is the sole solution for economic development in poor communities — even though no one ever says this. Then they argue that education can’t possibly be either the long-term or short-term solution for poverty — and find some flimsy data or examples to back it up.

Dana Goldstein of the Nation weakly pulled this funny trick earlier this month in her review of Steven Brill’s Class Warfare, trotting out the famed Coleman study to argue that high-quality teachers cannot help students overcome the consequences of economic poverty. Of course, she ignores the Coleman study’s conclusion that teaching accounts for nearly half of in-school effects on student achievement, and that it concludes that if teaching is of high-quality, schooling will be a bigger factor than socioeconomic background — and conveniently bypasses Brill’s ultimate conclusion that good-to-great teachers alone can’t improve education or stem poverty. 

This is a complete and utter misrepresentation of my essay on Class Warfare. First of all, I never mention the 1966 Coleman Report, whose research I consider too old to be reliable or definitive. Because my piece appeared in print, there are no hyperlinks embedded to my sources, so let me elucidate exactly how I came to the conclusion that differentials in teacher quality account for about 15 percent of student achievement outcomes, with much of the rest of the achievement gap attributable to students' socioeconomic status:

• This 1998 paper by Eric Hanushek, John Kain, and Steven Rivkin--economists who support free-market education reforms--concludes that teacher quality accounts for "at least 7.5 percent" of student achievement outcomes.
• This 2004 review of a number of studies on teacher effectiveness estimates that between 7 and 21 percent of student achievement differences can be attributed to teachers. See pg. 240. The authors are education researchers at Northwestern, the University of Chicago, and Tennessee State.
• On pages 3 to 4 of this 2002 University of Pennsylvania study, the researchers conclude that depending on the valued-added method used, between 4 and 28 percent of student achievement gaps can be attributed to differences between classrooms within a school, of which the teacher would be the most significant. 

As you can see, by estimating teacher effects at 15 percent, I've interpreted the research consensus quite generously. Matthew DiCarlo, a sociologist with the Shanker Institute, has looked at this same body of research and concluded that another 20 percent of the causes of student achievement gaps are "unobservable" (ex; differences in innate intelligence, statistical error, other mystery causes); and that the rest, about 60 percent, can likely be explained by all the myriad factors associated with socioeconomic status. These conclusions are based on research far more sophisticated and current than the Coleman Report, much of it conducted by the education reform movement's own favorite economists and teacher quality experts.

Nor do I claim, as Biddle writes, "that high-quality teachers cannot help students overcome the consequences of economic poverty." To the contrary, in the essay I state outright that high-quality teachers can and do have an enormous impact on student achievement. I write:

Brill and the accountability crowd are correct to note that high-performing teachers are consistently able to raise the test scores of even the poorest children. Research shows that an improvement of one standard deviation in teacher quality leads to approximately two to four points of gain for a student on a 100-point test in reading or math. Five years of great teachers in a row, therefore, could raise a student’s test scores by ten to twenty points. Whether this potential growth is incidental or transformative depends on where a student starts out: if he began at the twentieth percentile in reading, he’d still be failing; a jump from the seventieth percentile to the ninetieth could make him a candidate for selective colleges.

My source for this paragraph was the research of Columbia University economist and teacher quality expert Jonah Rockoff.

Lastly, Biddle accuses me of bypassing "Brill’s ultimate conclusion that good-to-great teachers alone can’t improve education or stem poverty." Actually, I made every effort in the piece to sketch out the evolution in Brill's thinking over the course of his book. While Brill ultimately became more critical of Teach for America and the 24/7, manic work culture of some charter schools, he maintains throughout the book that good teaching can overcome poverty. Writing about the research of economists Kane and Staiger, whom I cite above, Brill concludes: “A snowballing network of education reformers across the country…were producing data about how teaching counted more than anything else. ... It wasn’t that poverty or other factors didn’t affect student performance. Rather, it was that teacher effectiveness could overcome those disadvantages." (emphasis added)

Actually, Kane and Staiger's research does not show teacher quality counting "more than anything else;" it shows teacher quality counting, in their own words, for "at least 7.5 percent" of a student's achievement outcomes. 

Apologies for how detailed and wonky this is, but I did feel the need to respond to Biddle. It's ridiculous to claim that I think school reform doesn't matter or doesn't have a role to play in fighting poverty. If I believed that, I wouldn't have chosen to devote my professional life to this topic. If I thought schools didn't matter, I wouldn't spend time visiting schools to report on the awesome things they are doing to give poor kids a leg up. 

Back soon to your regularly-scheduled, non-confrontational programming...

Meet the Teachers' Union Contract of the Future

This post has been updated and corrected with more details on the contract, provided by the UFT

On Friday Geoff Decker of GothamSchools reported on the renegotiated contract between New York's United Federation of Teachers and Green Dot, the California-based charter school operator whose schools are all unionized. Two years after the first Green Dot/UFT agreement was drafted, the union agreed once again to a contract that lacks both traditional tenure and seniority-based layoffs. The new contract also pays veteran Green Dot teachers a $2,000 bonus as a reward for raising test scores, and a base salary 20 percent higher than their traditional school counterparts.

New York state exempts charter schools from tenure law, so one of the union's goals in a contract negotiation like this one is to build back in some job security. In the case of Green Dot, teachers actually have a shorter probationary period than in a traditional public schools--just one year, compared to three--but they also agree to a system in which administrators have more say over layoffs of non-probationary teachers. Instead of seniority-based layoffs, as in the traditional school contract, the union and administration will meet at the bargaining table in the event of layoffs to determine what "equitable criteria" should be used. 

Green Dot teachers will be evaluated according to the principles enshrined in New York State's Race to the Top application: 20 percent of the evaluation score will be based on student standardized test scores for grades and subjects in which they are available, 20 percent on other "local" measures of student academic growth, and 60 percent on classroom observation. 

When a teacher is ineffective, Green Dot administrators must prove "just cause" before firing him or her, the same standard practiced within many non-unionized companies in order to avoid wrongful termination lawsuits--and the same standard called for to remove a tenured teacher in the traditional school contract. The difference is that at Green Dot, there is a streamlined, approximately 90-day grievance and arbitration process in which the union can appeal a dismissal, which negotiators expect to be less cumbersome and less likely to involve attorneys than the traditional Department of Education process. 

After their first-year in the school, teachers grieve a discipline or dismissal decision to an indepdendent arbitrator. First-year teachers have no job security protections whatsoever The grievances of first-year teachers are ruled upon not by the arbitrator, but by the school's board of trustees. 

 The teachers' union contracts of the future may look a lot like this one--if we start making much-needed improvements in teacher preparation and allow teachers to get more actively involved in the administration of their schools. (Big "ifs," I know.) When I had lunch with AFT President Randi Weingarten in August, we discussed the future of tenure, and she told me she could absolutely envision an entire urban district organized around the same principles as the Green Dot contract--in other words, without teacher "seniority" as we know it today. "The question," she said, "is how do you create an environment that is mission-driven and has a culture of fairness to it?" 

Indeed, the Green Dot model calls for teams of teachers to be actively involved in hiring their peers; this is a highly-vetted workforce operating in an environment that emphasizes collegiality and professionalism. Without such healthy school environments, unions and teachers will have a hard time giving up the protections they've won because of a very real history of adminstrative overreach. 

Should/Could All Teachers Make $125,000?

Check out my Slate review of "American Teacher," the new documentary co-produced by Dave Eggers and narrated by Matt Damon. It's moving, engaging, polemical...and totally ineffective at advocating for the policy idea it supports: Paying teachers much, much more--say, $125,000 annually.

Read it!