Today Slate published a piece I've been working on for about a month, about computer assessment of student writing. You may have read about a new University of Akron study that found computer programs and people award student writing samples similar grades. The results are impressive, but their applicability is limited. As I discuss in the article, the study looked exclusively at the current generation of state standardized tests, which require students to write essays that are far less sophisticated than the ones we hope they will write once the Common Core is fully implemented in 2014.
The recent push for automated essay scoring comes just as we’re on the verge of making standardized essay tests much more sophisticated in ways robo-graders will have difficulty dealing with. One of the major goals of the new Common Core curriculum standards, which 45 states have agreed to adopt, is to supplant the soft-focus “personal essay” writing that currently predominates in American classrooms with more evidence-driven, subject-specific writing. The creators of the Common Core hope machines can soon score these essays cheaply and quickly, saving states money in a time of harsh education budget cuts. But since robo-graders can’t broadly distinguish fact from fiction, adopting such software prematurely could be antithetical to testing students in more challenging essay-writing genres.
In the rest of the article, I take a look at how public school writing instruction might evolve over the next five years, and also at how computer programmers are hoping to improve automatic writing assessment. It has been fascinating to dive into the field of Natural Language Processing, and especially interesting to ponder how Web search technology might someday allow computers to better "understand" the facts and arguments presented in human writing. As adult cheating scandals and exposés from testing industry whistle-blowers like Todd Farley have made clear, people, too, can be highly unreliable graders -- so I think education reformers are right to continue experimenting with these ideas.
That said, we should proceed with real caution, fully understanding the limits of the current software and the challenges of making great leaps forward in artificial intelligence. There's a lot to follow-up on with this topic. Today the Hewlett Foundation awarded $60,000 to a group of programmers who developed new, statistics-based software for analyzing student essays. And there are smaller companies, like SAGrader, which take a different approach to the technology than some of the testing giants do. I'll be continuing to follow this story, which I think is a really huge one as we roll out the Common Core and the new state tests affiliated with it.
Head on over to Slate to read the whole piece.
Comments