September 18, 2017

Matthew Bain

Visualizing Music Through Code

Topics: Music Makes You Smarter, Compose Music, Creativity, People of Compose Yourself, STEAM

Screen Shot 2017-09-14 at 2.18.38 PM.png

matt.jpgMatthew received his BFA and MFA from The Ohio State University in music and 3D computer animation. His thesis work, entitled "Real Time Music Visualization: A Study in the Visual Extension of Music", is the foundation for his company, Live Music Visuals. Matthew's visualization work has been exhibited in numerous performances including the Drums Downtown concert series, PASIC, and SIGGRAPH. Matthew recently worked for Dreamworks Animation as a layout artist and contributed to movies like"Penguins of Madagascar," "Madagascar 3," and "Rise of the Guardians." He also plays the piano and composes music in his spare time. 

My name is Matthew Bain and I would call myself an artist who codes.  I started learning to code in college and it really wasn’t until graduate school that I started to understand how coding could influence and improve my artistic designs. The ability to create a “for loop” and iterate over hundreds of items at one time enables me to generate visuals that can respond to music in real time. This coding ability has also served me well in my previous job. I used to work for DreamWorks Animation as a Final Layout Artist.  Some of the movies I worked on included Megamind, Madagascar 3, and The Penguins of Madagascar.

Screen Shot 2017-09-14 at 2.09.33 PM.png       Screen Shot 2017-09-14 at 2.10.21 PM.png

Screen Shot 2017-09-14 at 2.13.05 PM.png

As a final layout artist, I was responsible for many different shots in a movie. A shot would consist of different characters, camera angles, and actions to help tell the story.  Many of the tasks I had to do often required scripting which would help automate those tasks. Final Layout artists are often asked to modify many different shots at a time. Coding skills allowed me to write for loops as an example to iterate and modify numerous shots at one time. Rather than doing the same thing over and over again on 30 different shots, I could write a script or snippet of code that performs the same operation on all 30 shots in significantly less time.  Another benefit to the script is that I can reuse it in the future making my job easier and more efficient.  I could spend more time working on the creative aspects of my job while the scripts automate other functions in the background.

Screen Shot 2017-09-14 at 2.18.07 PM.png

In addition to my time at DreamWorks, I also develop apps for Android/iOS and build real-time visualizations of music. Coding plays an integral role in this work and allows me to create stories that react to music. Historically, music visualizations were hand-drawn and painstakingly synchronized to the music.  Disney’s Fantasia is an excellent example of this. Artists drew all of the characters by hand and animated their movements to the timing of the music. The advent of computers and faster processing speeds enabled us to create algorithms and programs which could respond to music.  Today many of these processes can be completed in real time.  Video games are a great example of this capability. Everything in the game reacts to your input whether it's a touch input on a screen or game controller button being pressed.  As you perform these events, the video game responds in real-time. My music visualization work takes a similar approach but instead of a game controller, music drives the visuals.

Music can be considered a stream of numbers from the computer’s perspective. Coding scripts let you read and manipulate those numbers in unique and artistic ways.  A music’s volume or frequency can be transposed into rays of light that shine through clouds, raindrops that freeze into arcs of ice, or trees that sprout into massive forests. Volume can be used to control the size of an object.  As music gets louder, 3D objects like trees, skyscrapers, or raindrops get bigger in size.  As music gets softer, these objects become smaller in size.  The different sized icicles are a good example of this.


Frequency in music is how high or low pitched something sounds.  A flute or violin has a much higher frequency range compared to a double bass or trombone.  These frequencies can relate to the color of a 3D object.  For example, bright green leaves could represent a flute playing while dark gray boulders could respond to a tuba playing. Coding scripts makes it much easier for me to explore these relationships between sounds and visuals.  The visualization below shows how all the different frequencies can react together within the same 3D space.

Screen Shot 2017-09-14 at 2.18.38 PM.png

Coding has enabled me to extend my artwork and build things I could not have done before. It has also helped make my job easier, more efficient, and often times more fun.  Programming code is a lot like solving puzzles.  It’s a fun challenge and super rewarding when you get to see the results generate right in front of you.  As an artist, I can solve puzzles and create new virtual worlds that respond to music or work on tons of shots in a movie.  I would encourage everyone to learn a little bit of code.  You may discover new ways to solve problems, create beautiful artwork, or find a way to make life easier my automating a task.

Check out my websiteblog, and follow me on Twitter @livemusicvisual for more music visualization examples and upcoming concerts featuring my work.

Color Cube Sudoku