:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/46185854/shutterstock_143876476.0.0.jpg)
College rankings measure prestige, success after graduation, and whatever else the group publishing them has decided matters in higher education. But a new study finds they're not great at measuring something that should matter a lot: how engaged students are in the classroom.
Researchers from Indiana University compared college rankings for 64 colleges from Forbes, US News, and Washington Monthly with results on the National Survey of Student Engagement. The survey measures students' study skills, how often they interact with faculty members, and how often they're asked to use critical thinking skills, as well as other factors thought to contribute to how much they learn.
They found no almost relationship between the rankings and the engagement scores. In some cases, going to a higher-ranked college seemed to put students at a disadvantage. In most cases, students at highly ranked colleges said they interacted with faculty less than their peers at lower-ranked colleges. The exception was freshmen at colleges ranked highly by US News, who spent more time interacting with faculty than students elsewhere. But the difference disappeared by senior year.
Who you are, not where you go, matters more for engagement
:format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/3632562/shutterstock_191822357.0.jpg)
The three ranking systems try to measure different aspects of the college experience. US News, which measures prestige, says the best college in America is Princeton University. Forbes, which measures student satisfaction and earnings after college, says the best is Williams College. Washington Monthly measures how colleges contribute to the public good, and the University of California San Diego comes out on top.
It's particularly striking that student engagement doesn't seem to correlate with prestige, student satisfaction, or future earnings. You might expect top colleges to do better at getting their students to study and discuss. You'd certainly expect it to play some role in student satisfaction.
But the researchers, Indiana's John Zilvinskis and Louis Rocconi, demonstrate that in many cases it's the students, not the place, that matters most. (The paper was presented at the recent annual meeting of the American Educational Research Association.) Zilvinskis and Rocconi found that colleges were responsible for, at most, 4 percent of students' engagement scores, after controlling for students' characteristics. Who you are and what you study seems to matter more than where you go to college.
Why college rankings don't measure learning
College rankings are mostly just entertainment. They decide what they value and then measure it — regardless of whether that's actually good for students. But amid concern that college students are studying less and learning less than they used to, rankings that measure what they're actually getting out of their education could be useful. This is a much harder task than it sounds.
Comparing student learning generally still requires standardized tests. And while there are some tests that try to broadly measure what students have gained from college, such as the Collegiate Learning Assessment, those tests aren't required, the stakes are low, and the results for individual colleges, majors, and programs aren't always disclosed.
Given the dismay over the effect of standardized tests in K-12 education, colleges are understandably concerned about any proposal to use high-stakes, publicly disclosed tests to measure what their students are learning. So instead, people trying to judge quality rely on other things — whether that's faculty qualifications, salaries after graduation, or surveys of student engagement.
But we don't always know if those measures are a good proxy for gaining knowledge and skills. And that's why it's so hard to rank colleges based on the outcome many Americans would say is most important: an education.
WATCH: '10 things they don't talk about at graduation'
