Theoretical Physics, the Scientific Method, and the Damage Done

Science isn’t supposed to be fast, but as progress speeds up it can actually feel like the pace of discovery is slowing — specifically in regards to physics. Because it has become commonplace for physicists to develop theories that no existing technology can test, the practice of physics has lagged behind scholarship — to whatever degree that’s possible — allocating resources for experimentation has become difficult.

The poster child (who’s buying this poster?) for the lag between theory and experiment is string theory, which hasn’t been disproved, but which some experts say may not be testable at all. For scientists, who can claim success proving or disproving hypotheses, this is the worst case scenario. Which is why scientists are talking it out: Last week, a cadre of physicists and philosophers gathered in Munich to discuss this problem at a conference called “Why Trust a Theory?”.

Things were surprisingly heated.

One of the conference organizers, Richard Dawid, argues that physicists can confirm a theory without experiment by appealing, for example, to its explanatory power, its internal consistency, or the absence of viable competitors. You won’t be surprised to learn that others disagree. George Ellis and Joe Silk published a comment in Nature called “Defend the integrity of physics”, in which they argued that Dawid’s approach undermines the physical sciences. The only way to test a theory, they say, is to put it to experiment.

Sabine Hossenfelder, a researcher at the Frankfurt Institute for Advanced Studies and the blogger behind Backreaction, agrees with Ellis and Silk. Hossenfelder works in quantum gravity phenomenology, which means she looks for experimentally testable predictions of quantum gravity theories. Inverse talked to her about theories, experiments, and how to get physics out of the weeds and back on track.

You’ve been very vocal about criticizing one recent experiment, Fermilab’s Holometer, on the grounds that the experiment wasn’t supported by theory, that we knew ahead of time that the experiment wouldn’t find anything. What’s the right relationship between theory and experiment?

The issue is the following: I work in quantum gravity mostly, which is one of the fields of physics that I would call foundational physics. These are the areas concerned with the structure of space time, what is matter made of, and what is the origin of the universe. And what has happened in the last century, not so surprisingly, is that all of the easy things have been done.

But it’s really here, where you want to push the frontiers of your theories, where the trouble is right now. Because what has happened is that you get this big gap between experiment and theory. The theorists have a lot of freedom and the experimentalists don’t really know what to do. So how do you make progress in an area like this?

Well you have to be very clear about how much you can trust your theory, because you need the theory to tell the experimentalist where to look. You need the theory to identify the most promising tests. Because then we have to invest a lot of money and build some experiment which might take decades.

So where does the Holometer come in? Well, you can ask the same thing about the Holometer: Was it promising to do this experiment because we trusted some theory that we might find something with it? And the answer is: No, there was no indication whatsoever.

Now this experiment didn’t cost billions. My understanding is that it’s primarily about increasing the precision possible in such measurements, and that’s an achievement all by itself. I don’t have a problem with that. But the idea that with this you could test quantum gravity was from the beginning nonsense, and everybody working on quantum gravity phenomenology knew this.

It seems like the idealized process of peer review or the idealized process of vetting scientific experiments would prevent this sort of thing from happening in a perfect world.

Yes, in a perfect world. But in reality, shit always happens. Still, peer review is the best that we have. I’m not saying there’s nothing wrong with peer review, I’m saying that the issues with peer review are organizational issues. They could in principle be fixed. Peer review is still a good thing that drives science, because it’s the only criterion we have — other than judgment by nature of course. But when it comes to theory development — before nature has made her judgement — peer review is the only way you can sort the good from the bad. When it comes to developing a new theory, you have to ask experts, “Is this theory compatible with everything we know about nature?”

And when you’re talking about experiments that potentially cost large piles of cash, it becomes even more important.

Right, so with Hogan, it was Fermilab and they probably had some reason. It’s not a huge amount of money. But now that he’s going around and saying we’ve tested the holographic principle, that really pisses me off, because really it has nothing to do with quantum gravity. From my perspective it also sheds a bad light on the whole field of phenomenological quantum gravity, where people actually try to test these things. I worked on this for a long time and now I’m trying to get out of it because I couldn’t get funding. So that’s how it works for the average people like me.

The Holometer problem sounds like an experimentalist without a theory. But you have the reverse problem in some cases as well, not necessarily for lack of trying, but you have theorists with no clear experimental prospects. I think you describe this problem as physicists getting “lost in math.” Am I understanding you properly?

One could interpret it this way. But what I refer to with “lost in math” is more this: Physicists have focused very strongly on a few theoretical possibilities when it comes to the theory of quantum gravity or to unification. It’s the same with dark matter. And along the way they have discarded many other possibilities. Physicists make a lot of assumptions that they use in theory development that they do not explicitly write down. And these then get “lost in math.”

What I mean with this is that theory selection is influenced by a lot of criteria that are not explicitly recognized. The examples on my mind are those of naturalness, simplicity, and beauty. These are all criteria that are in practice being used, but then they get converted into mathematical requirements and people forget that it was a choice, that they were hypotheses that you need to test.

Take this example of naturalness. The argument is that whatever the theory is, it should not have any parameters that are either very large or very small because that doesn’t seem natural. I think that’s complete nonsense.

I mean, nature doesn’t really care about your expectations.

Right. That’s a good point. Nature also doesn’t care about what you find beautiful. Why? Why should nature care? When people tell me, “I like this theory because it’s so elegant,” I’m like, “So what?”

With the Holometer, we get years of coverage of something that, in the end, probably isn’t testing anything at all. Nevertheless, because it has such a catchy premise, journalists will flock to it. What could science journalists do to kind of rein themselves in a bit and treat science more as it actually is than as this kind of fanciful TV show?

In the case of the Holometer it would have been very easy, because I think people just repeated what was in the press release from the place that actually funded the experiment. So what do you expect? Do you expect this to be objective? I mean, at least you should go and ask somebody from the field what they think about this experiment. In this case, it was incredibly sloppy for journalists to repeat this.

More generally, I think that it’s just difficult in some cases to really figure out what the scientists are saying. It is very difficult for someone who is not really working on this to figure out what is true, what is slightly bent and what is total nonsense. But a lot of this is also driven by pressure. There’s time pressure, and then it has to sell, so you have to come up with a big headline.

And that’s how science in the press morphs from a slowly moving, gradually progressing field of knowledge into this thing in which breakthroughs are happening every two days, and our fundamental understanding of the universe is being overturned every couple of months, so here’s this great story about it. Your mind is blown.

My mind has been blown so many times that I don’t know what to say about this anymore.