Scientists can test the techniques they use to understand the complex, organic systems of the brain by studying the results of the same techniques aimed at man-made, inorganic systems. That’s why researchers Eric Jonas and Konrad Kording attempted to use modern neuroscience analysis techniques on the 6502 microprocessor of an Atari 2600. And that is why the results of their experiment, published Thursday in PLOS: Computational Biology, are so discouraging. The inner working of Donkey Kong proved shockingly difficult to parse.
Jonas and Kording argue that despite our increasing understanding of neuroscience, the field is held back by “the fact that it is hard to evaluate if a conclusion is correct” and because “the complexity of the systems under study and their experimental inaccessibility make the assessment of algorithmic and data analytic techniques challenging at best.” Put differently: Major neuroscience findings might simply be wrong and replicating experiments is hard as hell.
It’s nearly impossible to tell how well data analysis algorithms work in the brain because scientists don’t know exactly how neural systems work. Ataris are different though. We know how Ataris work. And that’s where the rubber burns up on the road: The neuroscience techniques employed by the researchers did provide a result, but did not provide meaningful insight into how the console’s microprocessor actually works.
An Atari 2600.
The researchers analyzed the connections on the chip, whole-device recordings, the individual transistors, the joint statistics across transistors, and the single-unit tuning curves. They used neuroscience techniques to see if they could reveal “known characteristics” — things they already knew would happen. But instead of an illuminated understanding, they write, “We find that many measures are surprisingly similar between the brain and the processor but that our results do not lead to a meaningful understanding of the processor.”
This result suggests to the researchers that the analytic approaches used in neuroscience right now may do a poor job in creating an actual understanding of the brain. Without an adjustment to these systems, a big data approach to neuroscience may fall short in creating accurate research.
