Duet: Brainwaves Music BCI Project Wins Cal Hacks 2024

A talented group of students at USC Berkley took home first prize at Cal Hacks 2024 for their innovative BCI music project, "Duet."
Duet uses AI to compose music based on real-time emotional states as collected and translated by an Emotiv EPOC X headset and EmotivPRO software with the Cortex API.
The students hope Duet will offer effective music therapy for children and individuals with disabilities through personalized musical experiences.

Emotiv extends heartfelt congratulations to USC Berkley students Jiahui Jin, Sophia Zhang, Daniel Wang, and Justin Wu for winning first prize at Cal Hacks 2024, the world's largest collegiate hackathon. Jin et al.'s BCI project, Duet, translates real-time brainwave performance metrics into AI-generated music and visual feedback.

The inspiration for Duet was to meet the challenges of teaching music to children with developmental disabilities.

"Unfortunately, many children are left without the tools they need to communicate and express themselves creatively," the team explains. "That's where Duet comes in. By utilizing EEG technology, we aim to transform the way these children interact with music, giving them a voice and a means to share their feelings."

How It Works

Duet uses artificial intelligence to compose music based on a person's real-time emotional states by reading brainwaves. The team used Emotiv EPOC X headsets to stream brainwave data and processed it via Python with the Emotiv Cortex API. To assist them in their endeavors, Emotiv arranged for the students to borrow the EEG hardware and granted temporary access to its PRO subscription software.

Emotiv's proprietary machine learning classified the brain activity into emotional states (performance metrics), fed into Google’s Gemini LLM to generate music. Instead of following fixed rules, Duet acted as a live composer, deciding on the next few seconds of music based on emotions, available instruments, and previous outputs.

Sonic Pi handled live music playback, SingleStore managed low-latency data, and our Next.js frontend visualized both brainwaves and the corresponding music in real-time, the team explained.

"At Duet, we are committed to making music an inclusive experience for all, ensuring that every child—and anyone who struggles to express themselves—has the opportunity to convey their true creative self!"

The Future of Duet BCI

Jin et al. says their biggest challenge was coming up with a way to generate live music with artificial intelligence.

"We originally thought it was impossible and that the tech wasn’t “there” yet — we couldn’t find anything online about it, and even spent hours thinking about how to pivot to another idea that we could use our EEGs with," they said.

"However, we pushed eventually through and came up with a completely new method of doing live AI music generation that, to our knowledge, doesn’t exist anywhere else," Jin et al. explained. "It was most of our first times working with this type of hardware, and we ran into many issues with getting it to connect properly to our computers — but in the end, we got everything to run smoothly, so it was a huge feat for us to make it all work!"

To learn more about Duet and view the code in GitHub to try it for yourself, check out the full story on DevPost.

From all of us at Emotiv - congratulations, team, we're so proud of you!

You might also enjoy:

BCI Innovation for All

Emotiv proudly supports innovation for the betterment of humanity. If you would like to inquire about product lending, donations, or financial assistance to obtain Emotiv products for your EEG or BCI project, we'd love to hear from you.