GRE: Physicists Say Grad School Entrance Exam Is a Bad Predictor of Success

For most graduate students, the path to an advanced degree starts in a small lightless room in front of a computer screen, taking yet another round of standardized tests. As they frantically type an essay that’s destined to be graded by a computer, they may wonder: “How is this going to help me get through grad school?” A study published Wednesday in Science Advances is here to validate their internal angst. The GRE is not only a bad way to determine grad school success; it might also do more harm than good.

Led by Casey Miller, Ph.D., an experimental physicist at the Rochester Institute of Technology, this study focuses specifically on physics graduate programs, though Miller believes that his findings would likely apply to a variety of fields. When he analyzed anonymous GRE scores from 27 university graduate programs around the country, he found that they failed to predict whether they actually ended up with their degree at the end of the day.

"Physicists are a wily bunch. They don’t typically change how they do things without data.

This is already pretty well known amongst physicists, he says, but his study adds hard data to show that both the general GRE test and the physics-specific test aren’t a sure-fire way to tell who’d make a great physicist.

“Physicists are a wily bunch. They don’t typically change how they do things without data,” he tells Inverse. “So the fact that we can put data to this specifically in the physics community will help the faculty see the utility of these tools. So they’ll see, hopefully, that the tests don’t predict Ph.D. completion or grades very well.”

Things the GRE Can’t Predict

In the study, Miller analyzed scores from three different GRE test sections: the GRE-V (verbal), the GRE-Q (math) and the GRE-P (the physics-specific subject test). He compared those scores to the completion rates and grades of students in physics Ph.D. programs in four overlapping groups: students from the US, students from abroad, female students from the US, and male students from the US. Across all four groups, the GRE-P *failed to predict whether students would finish their program, as did the GRE-V section.

There was a small glimmer of hope for the GRE’s math section, which showed some correlation with completion rate when the results from US female students and US male students were combined. However, it was not a good predictor when applied to students as a whole and solely students from outside the US.

Nevertheless, things don’t look particularly good for the GRE in this paper — especially the physics-specific GRE. Students who scored below the 50th percentile had similar degree-completion rates to those who score in the top percentiles, Miller points out. That, he adds, is a key observation because a number of physics Ph.D programs have hard cutoffs. Roughly 25 percent of physics programs advertise that a GRE score should be above the 55th percentile for an applicant to have a shot at admission.

Things the GRE Can Predict

In short, GRE scores fail to predict how good a physicist a person may turn out to be. Instead, they reflect [demographic patterns] (https://www.nature.com/naturejobs/science/articles/10.1038/nj7504-303a) that are equally useless in predicting a person’s success in graduate school. There’s no statistically significant correlation between gender and a person’s likelihood of finishing a Ph.D. in physics, says Miller, but there is a significant gender gap on the GRE-P scores: The median score for a woman in the US is 580 and the median score for a man in the US is 680. The implication is that both groups would have a similar chance of excelling at the coursework required to become a physicist, but going by their test scores, one stands a better shot at admission than the other.

Importantly, neither of those scores would likely even be good enough to get either applicant into an estimated 25 percent of programs — though Miller was able to show significant gaps by gender, race, and ethnicity in who reaches that score cutoff and who doesn’t. And that’s not even taking into account the income patterns associated with GRE test scores.

"…the skills required to be successful in courses aren’t the same skills that are important to be a successful research physicist.

The big takeaway, says MIller, is that the GRE doesn’t tell admissions committees anything about what makes a good physicist — and could be partially responsible for physics’ abysmal diversity statistics. Only 5 percent of physics Ph.D. degrees awarded annually go to underrepresented minority groups, and only 20 percent of go to women. Miller’s findings suggest that there are a lot of people in both these groups whose test scores put them solidly in the rejected pile before they even have the chance to try their strength.

“We’re reasonably good at selecting who does well in coursework, but of course the skills required to be successful in courses aren’t the same skills that are important to be a successful research physicist,” says Miller. “Moving forward, we’re trying to focus on the things we know make a good researcher, which the current system doesn’t do very well.”