Elite Chinese Computer Scientists Don't Outperform Average Americans

Three years ago, former President Barack Obama went all-in on the growing digital economy. His $4 billion “Computer Science for All” initiative set out to empower students, from kindergartners to high schoolers, with computer science skills. The program also quietly addressed the growing concern among policymakers that the US isn’t producing enough top-tier computer experts relative to emerging peers like China and India. Fortunately, a new study suggests those fears may be overblown.

“There has been little understanding of how well computer science programs in college – where most of the training of computing professionals seriously starts – have been doing at equipping students with computer science skills,” Stanford researcher Prashant Loyalka tells Inverse.

The results of the new study show that American computer science graduates are still vastly outperforming peers in China, Russia, and India, the three countries who, along with the United States, produce more than half of the computer science graduates worldwide. It’s a finding that should come as some comfort to those who worry that America’s competitive advantage in technology, particularly with regards to China, might be closing. Over the weekend, the New York Times published a new report suggesting as much.

To arrive at these new findings, published Monday in Proceedings of the National Academy of Sciences, Loyalka, along with researchers from Berkeley, the World Bank, and the Education Testing Service, spent nearly three years evaluating the skill levels of international undergraduate seniors majoring in computer science. For data, the researchers looked at scores from more than 8,000 students who sat for a two-hour exam designed by the non-profit Education Testing Service (ETS). Rather than a specific language like Python, ETS lead Lydia Liu explained to Inverse that ETS used “pseudo-code” testing, too, to assess information management skills, software engineering, and deftness with programming algorithms, as well as other complexities. The idea was to develop a framework for how test takers approached “the underlying principles of coding.”

The team, explains Liu, didn’t have any preconceived notions going into the study.

“We knew that the US has been a very advanced country in computer science, and, in general, in technology and STEM,” Liu said. She also pointed to efforts on the part of the governments in China, Russia, and India who have all put an emphasis on besting the global rankings for computer science graduates.

Some of those efforts appear to be paying off: The US is no longer churning out nearly as many computer science graduates as their global competitors. China and India both produce roughly three times as many CS majors as the US. (Russia turns out about a fourth as many as the US.) But despite the institutionalized emphasis on computer science graduates, the study found that US graduates out-performed peers from China, Russia, and India — and not just slightly out-performed.

“‘Slightly’ is an understatement,” said Liu.

American students in average CS programs (as in, non-elite) performed as well as the elite Russian, Indian, and Chinese students. When comparing top students from each country, US students surged ahead of the pack. The findings were dramatic enough to even prove surprising to the study’s researchers, including Tara Beteille, who served as the team leader for the World Bank’s Technical Education Quality Improvement Project initiative with the Indian government.

“We hadn’t expected to see our elite colleges, not just India but China and Russia, so far behind elite colleges in the US,” Beteille tells Inverse.

Though a follow-up study will hopefully illuminate the “why” behind the study’s findings, both Beteille and Liu suspect that American students show up to college on their first day already better prepared than their global peers. All those fourth grade “hack-a-thons”? All the coding classes? They’re working. So even though Indian students may make the greatest strides once they get to a higher education setting, the head start in a long-term investment of developing computer science skills early may help create an unbridgeable gap.

Beteille hopes the findings from this study, the first of its kind to provide direct evidence of learning, will motivate all four governments to consider a more holistic approach to education, to look at not just the sheer number of computer science graduates, or even job placements, but the quality of their education.

“Rankings don’t really measure student learning or skills,” said Beteille. “What are students really learning?”

Correction: A previous version stated that the average US students performed better than their elite peers from China, India and Russia; the two factions in fact perform essentially the same.