The study that helped this reporter understand student success

What should teachers do to help their students succeed? That was a central question that journalist Paul Tough was trying to answer when he came across an innovative study in 2013.

Written by Northwestern University labor economist Kirabo Jackson, it drew upon data from North Carolina schools to measure not only students’ cognitive abilities, like math, reading, and writing, but also their noncognitive abilities, such as motivation and engagement. While cognitive skills are regularly measured with standardized testing, the method for evaluating noncognitive abilities — based on administrative data such as attendance, suspensions and overall GPA — was novel.

“Jackson’s new index measured, in a fairly crude form, how engaged the student was in school — whether he showed up, whether he misbehaved, and how hard he worked in his classes,” Tough told the PBS NewsHour.

What’s more, Tough said, the study showed students’ noncognitive scores were a better predictor than test scores not only of academic success, but also success in life. While Kirabo’s noncognitive scale was at odds with the way many school districts evaluate students, Tough said, it came as no surprise to teachers he spoke with that these skills were more telling than test scores: “They know there are things going on in their classrooms that standardized test scores can’t capture.”

However, the teachers who were best able to motivate their students to do better in school overall — not just when it comes to standardized testing — weren’t being recognized by the common evaluation systems, Tough said. His book, “Helping Children Succeed,” sought to examine what teachers were doing to effectively improve students’ motivation and attitude.

“That’s the question that I spent the rest of the book trying to answer,” Tough said. “And I think it’s a crucial one not just for teachers, but for parents as well.”

From “Helping Children Succeed”

A few years ago, a young economist at Northwestern University (1) named Kirabo Jackson decided he wanted to investigate the ways we measure the effectiveness of teachers. He found a detailed database in North Carolina that tracked the performance of every single ninth-grade student in the state between 2005 and 2012 — a total of 537,241 students. The data followed their progress not only in ninth grade but through high school and beyond.

Jackson had access to each student’s scores on the statewide standardized test, and he used that as a rough measure of their cognitive ability. Then he did something new. He created a proxy measure for students’ noncognitive ability (2), using just four pieces of existing administrative data: a student’s attendance, suspensions, on-time grade progression, and overall GPA. Jackson’s new index measured, in a fairly crude form, how engaged the student was in school — whether he showed up, whether he misbehaved, and how hard he worked in his classes.

Remarkably, Jackson found that this simple noncognitive proxy was a better predictor than a student’s test scores (3) of whether the student would attend college, a better predictor of adult wages, and a better predictor of future arrests. Jackson’s proxy measure then allowed him to do some intriguing analysis of teachers’ effectiveness. He subjected every ninth-grade English and algebra teacher in North Carolina to what economists call a value-added assessment. First he calculated whether and how being a student in a particular teacher’s class affected that student’s standardized-test score. This is the basic measure of value-added assessment in use today; teachers in many states across the country are evaluated (and sometimes compensated or fired) based on similar measures. But Jackson went one step further. He calculated the effect that teachers had on their students’ noncognitive proxy measure: on their attendance, suspensions, timely progression from one grade to the next, and overall GPA.

What he found was that some teachers were reliably able to raise their students’ standardized-test scores year after year. These are the teachers, in every teacher-evaluation system that currently exists in this country, who are most valued and most rewarded. But Jackson also found that there was another distinct cohort of teachers (4) who were reliably able to raise their students’ performance on his noncognitive measure. If you were assigned to the class of a teacher in this cohort, you were more likely to show up to school, more likely to avoid suspension, more likely to move on to the next grade. And your overall GPA went up — not just your grades in that particular teacher’s class, but your grades in your other classes, too.

Jackson found that these two groups of successful teachers did not necessarily overlap much; in every school, it seemed, there were certain teachers who were especially good at developing cognitive skills in their students and other teachers who excelled at developing noncognitive skills. But the teachers in the second cohort were not being rewarded for their success with their students — indeed, it seemed likely that no one but Kirabo Jackson even realized that they were successful. And yet those teachers, according to Jackson’s calculations, were doing more to get those students to college (5) and raise their future wages than were the much-celebrated teachers who boosted students’ test scores.

1. I first came across Kirabo Jackson’s North Carolina study in 2013, and I visited him a year later at Princeton, where he was a visiting scholar. His study and our interview helped unlock for me one of the mysteries I was trying to solve in this book: What should teachers do in the classroom to help their students succeed? (If you’re interested, you can read Jackson’s paper online.)
2. To me, this was Jackson’s conceptual breakthrough. Rather than trying to invent a scale to measure students’ grit or perseverance, he came up with a way to measure their motivation and engagement based on data that schools were already collecting.
3. I thought this was Jackson’s most remarkable finding: His new noncognitive score turned out to be a better predictor than test scores not just of academic success, but of life success. This is at odds with the way many districts evaluate teachers, but when I talk to teachers themselves about Jackson’s study, they tell me that his discovery makes perfect sense to them. They know there are things going on in their classrooms that standardized test scores can’t capture.
4. When I visit schools, I often hear from educators that they have a strong hunch which teachers in a school building are best able to motivate their students. Jackson’s study supports this hunch with data. If you’re a student and you’re lucky enough to be assigned to one of these teachers, you’re likely to do better in school overall – not just in that teacher’s class.
5. Jackson’s study answers some important questions. But, for me, working on this book, it raised plenty of others. What exactly are these teachers doing in their classrooms that is so effective at improving students’ motivation and attitude? That’s the question that I spent the rest of the book trying to answer – and I think it’s a crucial one not just for teachers, but for parents as well.