Music study is a breakthrough in visualizing the neuroscience of imagination

“Amazing Grace.” ​​Johann Pachelbel’s “Canon in D Major.” “Baby Shark.” The songs have one thing in common: They’re all instantly recognizable.

They are so recognizable that you can likely “hear” when you think about them — even when you’re sitting in silence. But what’s happening in your brain when you imagine them? What about the moments of silence between notes of music? What’s happening in your brain then?

These questions have long perplexed scientists. However, a pair of studies published Monday in the Journal of Neuroscience has, in an unprecedented way, finally illuminated the auditory-imagination process.

These findings point to a better understanding of the neural processes involved in “the music of silence,” and give a more precise picture of the neuroscience of imagination. Ultimately, music is more than a sensory experience: Our brain attempts to predict notes even when no music is playing.

Co-author Giovanni Di Liberto, a researcher and Assistant Professor at Trinity College Dublin tells Inverse this study also serves as “a new method to study imagination.”

“The brain tries to predict upcoming music events,” Di Liberto says. “That same predictive process is, in my opinion, related to what we experience as imagination.”

What you need to know first — Central to the study is the concept of “melodic expectations.”

Our brains are very good at learning patterns and using that information to make predictions about what might happen next. For example, if you’re driving and see another car weaving dangerously in and out of traffic, you might instinctively give that car a wide berth, knowing that they could abruptly change lanes or brake unexpectedly.

The same process of recognizing patterns occurs when we listen to music, explains Di Liberto.

“Our brain learns musical patterns throughout our life. It then attempts to predict upcoming musical notes all the time,” he says. That’s melodic expectation.

“When we hear something that is different from what our brain considers ‘expected,’ that generates a brain response. A mismatch that tells us that something is off,” Di Liberto says.

In other words, when that happens, our melodic expectation is challenged.

Di Liberto and his colleagues wanted to understand the neural activity of melodic expectations as they related to the imagination (the foundation of the first study) and the silences between notes (the foundation of the second study).

Previous studies used functional magnetic resonance imaging(fMRI) to study melodic expectations and imagination — this research team used electroencephalography (EEG) instead.

“fMRI is slow,” Di Liberto says. “fMRI usually gets an image every 2 seconds; EEG, instead, has a resolution of 10 ms or even more.”

Additionally, fMRI measures blood oxygen levels — not brain electrical activity. EEGs, in turn, allow for a more detailed picture of neuronal processing, he says.

“Previous studies could not see the imagination responses,” Di Liberto says.

How the discovery was made — In the first study, the researchers used EEG to measure the brain activity of 21 professional musicians when they imagined Bach piano melodies.

In the second study, the researchers had the same professional musicians listen to piano melodies by Johann Bach while monitoring their neural activity via EEG.

What’s new — The researchers found that the brain responds similarly to imagined music and music the participant is actually listening to. When there was a silent moment in the music, the brain was functionally imagining what would come next, as it would if it were imaging the music itself.

There were, however, some important differences in the processes.

During the listening study, the neural signal suggested there was a combination of at least two brain processes:

• The brain was responding to the sound.
• The brain was calculating whether the sound matched its expectations or not.

Further, the imagined responses don’t happen at the exact same time as the listening responses, and heard notes go in the opposite direction as imagined notes. Di Liberto explains that “heard notes produce a positive electrical response, imagined notes a negative electrical response.” Positive and negative refer to the electrical voltage measures.

The researchers believe neural activity observed shows the brain responding to the sound (positive electrical response) and predicting the sound (negative electrical response).

“When the prediction is spot on, the positive and negative cancel each other,” Di Liberto explains, “but your brain responds more strongly to unexpected or unpredictable events.”

There is no sensory input during silence or imagined music, so the brain activity is coming from the imagination process itself.

What it means for the future — This study brings us “one step closer” to understanding why music is “so important and interesting for our brain,” Di Liberto says.

Eventually, he’d like to use the listening part of this paradigm to develop a diagnostic tool that can “tell us whether the brain is operating in a typical way.”

He sees this tool for detection and intervention: Something that can be used for a range of aims, from recognizing early signs of dementia to issues with attention.

Abstract for study 1: Musical imagery is the voluntary internal hearing of music in the mind without the need for physical action or external stimulation. Numerous studies have already revealed brain areas activated during imagery. However, it remains unclear to what extent imagined music responses preserve the detailed temporal dynamics of the acoustic stimulus envelope and, crucially, whether melodic expectations play any role in modulating responses to imagined music, as they prominently do during listening. These modulations are important as they reflect aspects of the human musical experience, such as its acquisition, engagement, and enjoyment. This study explored the nature of these modulations in imagined music based on EEG recordings from 21 professional musicians (6 females and 15 males). Regression analyses were conducted to demonstrate that imagined neural signals can be predicted accurately, similarly to the listening task, and were sufficiently robust to allow for accurate identification of the imagined musical piece from the EEG. In doing so, our results indicate that imagery and listening tasks elicited an overlapping but distinctive topography of neural responses to sound acoustics, which is in line with previous fMRI literature. Melodic expectation, however, evoked very similar frontal spatial activation in both conditions, suggesting that they are supported by the same underlying mechanisms. Finally, neural responses induced by imagery exhibited a specific transfomation from the listening condition, which primarily included a relative delay and a polarity inversion of the response. This transformation demonstrates the top-down predictive nature of the expectation mechanisms arising during both listening and imagery.

Abstract for study 2: During music listening, humans routinely acquire the regularities of the acoustic sequences and use them to anticipate and interpret the ongoing melody. Specifically, in line with this predictive framework, it is thought that brain responses during such listening reflect a comparison between the bottom-up sensory responses and top-down prediction signals generated by an internal model that embodies the music exposure and expectations of the listener. To attain a clear view of these predictive responses, previous work has eliminated the sensory inputs by inserting “artificial” silences (or sound omissions) that leave behind only the corresponding predictions of the thwarted expectations. Here we demonstrate a new alternate approach in which we decode the predictive EEG responses to the silent intervals that are naturally interspersed within the music. We did this as participants (experiment 1: twenty participants, ten female; experiment 2: twenty-one participants, 6 female) both listened or imagined Bach piano melodies. Prediction signals were quantified and assessed via a computational model of the melodic structure of the music, and were shown to exhibit the same response characteristics when measured during listening or imagining. These include an inverted polarity for both silence and imagined responses relative to listening, as well as response magnitude modulations that precisely reflect the expectations of notes and silences in both listening and imagery conditions. These findings therefore provide a unifying view that links results from many previous paradigms, including omission reactions and the expectation modulation of sensory responses, all in the context of naturalistic music listening.