This is a compound final project for an Audio Art course and an Advanced Interactivity course in the Time Arts program at Northern Illinois University. It explores the concept of evolving audio within a 3D interface, which also takes on a biological aesthetic, using the Unity3D development platform coupled with MAX/MSP/Jitter. The intention was to create a space that acts as a semi-controllable environment populated with “beings” that each possess a unique tone and color. The user acts a selector, allowing a sound to propagate and diversify. It is a consideration of beautiful procedure; although the initial tone splits and collapses into new signals mathematically, the finished chords often possess an ethereal and pleasing sound. In this basic simulation of a natural process, I mean to illustrate how a very simple premise can lead to a complex array of sounds and sensations, with each variation containing a distinct beauty.

The visual space is populated by floating spheres, each colored to represent a different frequency. When the user click on one, it splits. This is represented visually by a mitosis-like animation created in 3DS Max. A preset algorithm determines the path of evolution, which is presented as varying frequency, modulation, and color. The new frequencies and modulations are stored into a multidimensional array, which is parsed into slices (representing each stage of the evolution) and passed over TCP/IP to the MAX/MSP patch. The sound algorithm is as follows:

1. The initial tone F0 is randomly selected from a set range. In the current design this range is between 60Hz and 330Hz. An initial modulation frequency is set to 0Hz.
2. The tone is split into two frequencies, F1A and F1B. The first is a random variation on F0, the second is the harmonic overtone of F1A. The modulation frequency is randomly raised or lowered by 1Hz.
3. F1A and F1B are split into four tones, F2A-D. These are random variations of the prior two tones and their harmonics. The modulation frequency is varied again.
4. These four are split again into eight tones comprised of random variations and harmonics, F3A-H. The modulation frequency is varied again.
5. The first variation and harmonic are averaged with the following four, collapsing into six tones F4A-F. The modulation frequency is varied again.
6. These six tones are averaged together again, created three final tones F5A-C. The final modulation frequency is varied once again.

I originally intended to split the frequency into six distinct trails (to mirror the visual animation), but simulations of this arrangement seemed to lose definition at their peak, which would have involved 108 combined signals. Perhaps a future implementation of this project will attempt this level of complexity.

The simulation compiles the frequencies and modulations into a multidimensional array within Unity: six generations of sound, eight channels per generation, with a frequency and modulation frequency for each channel. On every frame, the current generation of sounds is serialized into a string which is sent over TCP/IP to a designated port. MAX/MSP, receiving on this port, routes the string and unpacks it into a tree of frequencies and modulations, which are turned into tones and fed into the DAC. A gain slider allows for performance tweaking, as the signal usually contains heavy distortion at 100% volume.

Unity communicates with MAX/MSP using a custom implementation of the Mu Max-Unity Interoperability Toolkit developed by the DISIS Interactive Sound & Intermedia Studio at the Virginia Tech Department of Music. The system uses a set of C# scripts within Unity to broadcast on a local port. MAX/MSP sends and receives packets using Tom Igoe’s netsend/netreceive plug-in. Unfortunately, the bulk of my project was written in Javascript, causing errors due to the C# scripts compiling ahead of the JS files. I ended up re-writing several of the C# files in order to make the project work. This was preferable to the only existing workaround, which involved placing Javascript files into a Standard Assets folder that Unity compiles ahead of the base directory. While this will allow a JS=>C# channel, it breaks any existing C#=>JS relationships. In my project, a C# plug-in that allows easy conversion to/from HSB Color would be interrupted by this workaround.

The project is available for download below, but you will need Max 5 installed. If you feel particularly adventurous, you can try the software on a trial basis free for 30 days. In order to work this simulation, perform the following steps:

1. Open the Unity binary (Windows or OSX).
2. Open the Max patch (genesound.matpat)
3. Click connect localhost in the patch, adjust the gain bar, and turn on the DAC.
4. In the Unity simulation, click on a sphere to begin!

Download Project

Notes:
- Optimal gain seems to be no more than 55-60%
- The simulation may not work in Windows without downloading the Win version of netsend~

A video of the thing in action coming soon.