matanuki

This is a compound final project for an Audio Art course and an Advanced Interactivity course in the Time Arts program at Northern Illinois University. It explores the concept of evolving audio within a 3D interface, which also takes on a biological aesthetic, using the Unity3D development platform coupled with MAX/MSP/Jitter. The intention was to create a space that acts as a semi-controllable environment populated with “beings” that each possess a unique tone and color. The user acts a selector, allowing a sound to propagate and diversify. It is a consideration of beautiful procedure; although the initial tone splits and collapses into new signals mathematically, the finished chords often possess an ethereal and pleasing sound. In this basic simulation of a natural process, I mean to illustrate how a very simple premise can lead to a complex array of sounds and sensations, with each variation containing a distinct beauty.

The visual space is populated by floating spheres, each colored to represent a different frequency. When the user click on one, it splits. This is represented visually by a mitosis-like animation created in 3DS Max. A preset algorithm determines the path of evolution, which is presented as varying frequency, modulation, and color. The new frequencies and modulations are stored into a multidimensional array, which is parsed into slices (representing each stage of the evolution) and passed over TCP/IP to the MAX/MSP patch. The sound algorithm is as follows:

1. The initial tone F0 is randomly selected from a set range. In the current design this range is between 60Hz and 330Hz. An initial modulation frequency is set to 0Hz.
2. The tone is split into two frequencies, F1A and F1B. The first is a random variation on F0, the second is the harmonic overtone of F1A. The modulation frequency is randomly raised or lowered by 1Hz.
3. F1A and F1B are split into four tones, F2A-D. These are random variations of the prior two tones and their harmonics. The modulation frequency is varied again.
4. These four are split again into eight tones comprised of random variations and harmonics, F3A-H. The modulation frequency is varied again.
5. The first variation and harmonic are averaged with the following four, collapsing into six tones F4A-F. The modulation frequency is varied again.
6. These six tones are averaged together again, created three final tones F5A-C. The final modulation frequency is varied once again.

I originally intended to split the frequency into six distinct trails (to mirror the visual animation), but simulations of this arrangement seemed to lose definition at their peak, which would have involved 108 combined signals. Perhaps a future implementation of this project will attempt this level of complexity.

The simulation compiles the frequencies and modulations into a multidimensional array within Unity: six generations of sound, eight channels per generation, with a frequency and modulation frequency for each channel. On every frame, the current generation of sounds is serialized into a string which is sent over TCP/IP to a designated port. MAX/MSP, receiving on this port, routes the string and unpacks it into a tree of frequencies and modulations, which are turned into tones and fed into the DAC. A gain slider allows for performance tweaking, as the signal usually contains heavy distortion at 100% volume.

Unity communicates with MAX/MSP using a custom implementation of the Mu Max-Unity Interoperability Toolkit developed by the DISIS Interactive Sound & Intermedia Studio at the Virginia Tech Department of Music. The system uses a set of C# scripts within Unity to broadcast on a local port. MAX/MSP sends and receives packets using Tom Igoe’s netsend/netreceive plug-in. Unfortunately, the bulk of my project was written in Javascript, causing errors due to the C# scripts compiling ahead of the JS files. I ended up re-writing several of the C# files in order to make the project work. This was preferable to the only existing workaround, which involved placing Javascript files into a Standard Assets folder that Unity compiles ahead of the base directory. While this will allow a JS=>C# channel, it breaks any existing C#=>JS relationships. In my project, a C# plug-in that allows easy conversion to/from HSB Color would be interrupted by this workaround.

The project is available for download below, but you will need Max 5 installed. If you feel particularly adventurous, you can try the software on a trial basis free for 30 days. In order to work this simulation, perform the following steps:

1. Open the Unity binary (Windows or OSX).
2. Open the Max patch (genesound.matpat)
3. Click connect localhost in the patch, adjust the gain bar, and turn on the DAC.
4. In the Unity simulation, click on a sphere to begin!

Download Project

Notes:
- Optimal gain seems to be no more than 55-60%
- The simulation may not work in Windows without downloading the Win version of netsend~

A video of the thing in action coming soon.

This project is an experiment in 3D interface and behavior. When you click on the planet, it will spawn a little mushroom. As the plant grows, it will drop spores that grow more mushrooms. Moving the mouse to the edges of the screen will spin the planet, causing the spores to land on other parts of the sphere or fly out into space. So you’re sort of in control, but not in control.

Click on the image to view. This piece requires the Unity Web Player, I promise it won’t make spiders and ninjas flow out of your monitor if you install it.

All of the textures used in this project were sourced from original scans of handmade paper. I enrolled in a papermaking course last year and in addition to learning how to turn plants into writable surfaces, I walked away with a huge catalog of organic and exotic textures. I love the depth and variation they can add to a digital work.

My classmates told me this project reminded them of a Super Mario game. Guess I was channeling Miyamoto when I put the visuals together.

A few years ago I worked with a local hookah bar owner to start selling his hookahs and accessories online. The site started out pretty simple, but my buddy Colin got on board and the project evolved into a kind of crazy mix of ZenCart and WordPress. A redesign was long overdue, and here’s the new look.

Big versions of the other banner images:
Mya Diva Version
Shisha Close-Up Version
Mya Nile Version
Mya Vortex Version
Mya Bambino Version
Golden Layalina Version

Unlike the current design, the new setup will downplay the social/blog element and stress that yes, we do in fact sell things. I’m going for a simpler, cleaner interface that highlights the featured products in a more artistic way. The real stumbling block has been the original photograpy; most hookah and shisha suppliers don’t provide decent images of their products, so we had to do a lot of our own image work.

On the plus side, it’s been a lot of fun getting into the photography. I built a photo stage inspired by this tutorial by Davide Guglielmo. That guy is awesome. His stock photography is amazing and he posts a lot of it free-to-use.

My take on his stage uses a 36×48 sheet of Lexan instead of plexiglass, and the frame is made out of 2×4 connected with wood screws and steel joiners instead of free-standing particle board. I decided to use Lexan because it’s much stronger, and flexible enough to bend into a curve while only needing a few anchor points to keep its shape. Unfortunately, a sheet that big was nearly $70, but I think the increase in material cost was worth it. The result was a large, lightweight stage about 36x30x30. Here are a few really bad pictures I snapped with my phone:

Crappy photos of a photo stage? HAHA! Irony.
Here’s a shot of it in action with the SLR:

Eventually, something resembling the design above will appear here:
Hookah.WS [link]

In the mean time, enjoy this nifty article I wrote about how to use a hookah. The photos in that article are courtesy of Colin. He’s got mad skills.

Watch this one with headphones for the full effect.

This is a two-part project for an Audio Art course in the Time Arts program at Northern Illinois University. The soundtrack is a musique concrète piece with a focus on Foley sound — except for the samples of spoken word and explosions, everything was originally recorded and mixed. The second part of the assignment was to create a synesthesia visual element that drew on the timbre and quality of the sound.

This project is a meditation on nuclear energy. Almost a century since the discovery of atomic power, we have not yet succumbed to the threat of man-made apocalypse, nor has the futurist dream of a uranium-powered utopia come true. While it is a nearly infinite source of both energy and destruction, the mechanics of the subatomic world are still not fully understood. To me, it is fascinating to wonder what brief, inexplicable sight occurs at the moment of fission. Beneath the implosion plates, within a perfect sphere of enriched plutonium, the heavy atoms are compressed. They pack tighter and tighter, until the bonds that hold the world together cannot hold — an atom shatters. Within the dense core the particles collide with the nuclei of other atoms, collapsing the sphere unto itself and setting off a chain reaction that releases the power of the sun.

The audio elements used in this piece are meant to invoke an emotional response, as well as carry a vague narrative of experiencing a bomb detonation. The visuals parallel this intent. It could be thought of as a waking dream, starting with visuals and sounds that are ambiguous, even beautiful at times, before abruptly falling into a nightmare. I am not a musician, but decided to use actual musical instruments (viola, guitar, drums, cymbals) in order to convey emotion more effectively than with sound effects alone. My lack of training, in this context, helped me treat the instruments abstractly. I focused on matching the timbre of each sound to a stage of the surreal experience, and in turn use a visual that resonated emotionally.

This is a reflection on fear and awe that I can only imagine. My generation mocks the use of a color-coded scale to indicate terrorist threats; my mother and father were shown a cartoon of a turtle who tells them that everything they know and love might be destroyed at any moment, and their best plan to survive is to hide under their desks with their hands over their heads.

When I searched through the archives of government films about nuclear weapons and radiation, every one contained at least a few sequences of beautifully rendered animation. These cartoons varied in style and message, but all had one element in common: horror. Sometimes it was the intent of the animator to shock the audience, but most of them are clearly meant to be informational or even reassuring. Imagining my parents watching these films, glancing out the window to see if the “Red Menace” was about to strike, I’m not sure how their reactions could have been anything but screaming and ducking under the nearest piece of furniture.

With hindsight, the fear of nuclear annihilation seems surreal, even quaint — like a monster evaporating when the closet light comes on. While that doesn’t mean the threat was non-existent, I believe it can give us perspective about those times of hysterical, paralyzing fear. If nothing else, there will always be an animator willing to visualize your horror in technicolor.

Some fun, trivial information:
- The sound of the building collapsing is a recording of ice being knocked off the side of my freezer.
- The entire soundtrack to the fission sequence was sampled from four arbitrary notes in a recording of my roommate practicing her viola, pitch-shifted up and down perfect fourths to simulate chords and doubled up against a convoluted version to make them sound more orchestral.
- The scurrying noise near the end of the video is a recording of a moth trapped inside a plastic bag with a microphone.

Video sourced from:
‘Medical Aspects of Nuclear Radiation,’ USAF Special Weapons Project (1950)
‘Duck and Cover,’ Federal Civil Defense Administration (1951)
‘Stay Safe, Stay Strong: The Facts About Nuclear Weapons,’ USAF (1960)
‘Radiological Defense,’ U.S. Office of Civil Defense (1961)

Sources courtesy of the Prelinger Archives.

My buddy Colin just posted another demo video of the game engine he’s working on, and it looks amazing! In addition to dynamic buildings and weather, his engine renders depth of field and simulates HDR lighting. Pretty amazing for a just-for-fun side project.

Please excuse the melodramatic music.

Read more about it at Colin’s website.

My first 3D animation since high school! Very exciting.

Created with 3dsmax, sound coming soon.

I’ve been doing some work with Processing, which is a Java-based development platform for artists interested in creating interactive works. This project draws a self-perpetuating helix wherever you click and drag.

Click on the image above to see it.
A Java browser plug-in is required to view this work.

My first 3D project of the semester. I was going for kind of an ambiguous biological theme, like something that could be the size of a sea urchin or only visible under a microscope. We’ll never know.

Click for big version.

The CDs are back from the printer, and the web version is up! The Connections CD is a technical walkthrough for new students at Northern Illinois University. The annual project started a few years before I came to work for ResTech, and in the five versions since I took over, it’s changed a lot (I used to suck at Photoshop).

The screen demos and interface were authored with Adobe Captivate. For the actual CDs, I wrote PC and Mac specific launchers in ActionScript and burned them to a dual-partition format. Hopefully whoever takes over for me next year likes working on this project as much as I did.

My friend Colin is working on a from-scratch game engine for a 3D tower-building simulator. Here is a very early demo of what he’s programmed so far. This expanded-style view is the foundation of the level editing mode.

Read more at Colin’s Website.