Where Music Meets Neuroscience

“It’s not the first time people have tried to connect brain waves with music, or sound production, but this is new and exciting,” said Khaleghian, a multimedia artist and composer, whose work bridges the gap between music and technology.

In short, he explained, the work Park pursued under his Gibbons Summer Research Program has taken important steps toward the design of a medical device that could help people suffering with neurological conditions like anxiety and dementia. The idea is for patients to wear a specially designed electroencephalogram (EEG) cap, which is linked to a digitally connected self-playing piano (in this case the Steinway Spirio).

They would then interact with the piano using their brainwaves, through the development of a brain-computer interface (BCI). The sounds thus created would in turn stimulate the brain, causing the creation of more music and leading to a healing effect. It’s a process Khaleghian describes as a “neural feedback loop,” comparable, perhaps, to a breathing exercise that uses thoughts instead of breathing. “Let's say somebody has a high heart rate, so if they can actually control the music with their mind, they could use this mechanism in a way that might slow their heart rate,” he explained.

The final product is still some way from completion, said Khaleghian, but the work done by Park over the summer, experimenting with data flow and algorithms to enable different systems to talk to each other, was an important step.

“At first, the plan was simple,” said Park, who is majoring in music and computer science—“map raw brain data directly to notes, which would be sent to a player piano via MIDI (the language instruments and computers use to talk to each other).”

However, the complexities of dealing with different types of brainwaves and developing the tools to convert them into an audio signal required the help of some professional neuroscientists.

This included Professor of Neuroscience and Biology Manuel Díaz-Ríos, who directs Bowdoin’s neuroscience program and who offered advice on the project. Assistant Professor of Computer Science Jeová Farias also helped with some of the coding. Additionally, Park and Khaleghian brought in two outside collaborators: the neurology department at the University of Texas’s MD Anderson Cancer Center and California-based medical device company Zeto, which develops EEG headsets, containing electrodes to measure brain activity.

In late July, Park met with MD Anderson’s Dr. Mei Rui, a neurosurgery professor who is also an award-winning concert pianist and director of the center’s music-in-medicine initiative.

The purpose of the meeting, said Park, was to elicit feedback from Rui on the data flow system he had developed for connecting certain neurological bandwidths (or brainwaves) with the electrodes in the headset. Brain frequency bandwidths are divided into five categories, each reflecting a different level of brain activity. They range from delta waves, which reflect deep sleep, to gamma waves, indicating high level cognitive processing.

“I showed Dr. Rui my system, which at the time was focusing on the alpha bandwidth [reflecting calmness and relaxation] and occipital electrodes and triggering these musical motifs on the piano I composed when certain strength thresholds were met,” said Park. Rui liked the concept and encouraged him, added Park, offering guidance on expanding the project to include more electrodes and bandwidths.

“That led to the current state of my project, which detects which region of the subject's brain is most dominant, along with which frequency bandwidth is strongest (theta, alpha, beta, and gamma). Based on the combination of the results, different musical motifs will be played, but this is still to be finalized.”

Park said the project was a huge learning experience for him and makes him feel comfortable when it comes to troubleshooting technical roadblocks, collaborating with others, and managing time under pressure.

Park is now headed to Denmark, where he will study abroad for a semester, but the fruits of his summer project will help digital music scholar Khaleghian in his ongoing research into the subject, in collaboration with the MD Anderson Center. For him, a fun part of the project has been connecting the dots, taking existing pieces of pioneering research and repurposing them for something new. “For example," said Khaleghian, "when EEG caps were designed, they were meant for use in purely medical settings, monitoring the brain activities of people with severe head injuries in ambulances or hospitals. No one considered using them quite the way this project is proposing. Likewise, when Steinway designed its Spirio player piano, there was no expectation that it might have a medical application.”

Another important aspect, he added, is the multidisciplinary nature of the research. “My hope is that faculty colleagues from across the spectrum might be interested in working with the music department once they learn about this kind of cool stuff that we’re doing.”