I was recently at a workshop / residency at the Mapping festival 2010, Geneva, Switzerland. Many collaborators, artists, musicians, performers, visualists took over various spaces at La Parfumerie to create audio-visual performances and installations. Constructing a large scaffold structure in the main hall, armed with DMX controlled lights, microphones, cameras, sensors and projectors; we converted the space into a giant audio-visual-light instrument for the audience to explore and play with, and be part of and experience a non-linear narrative performance. The project involved live projection mapping, motion tracking, audio-reactive visuals, piezo-reactive audio and visuals, DMX controlled lights, rope gymnasts, acrobats and much more!
I am happy to announce that my Body Paint installation will be exhibited at the Victoria & Albert Musuem from 8th December 2009, to 11 April 2010 along with many other great works, for the Decode: Digital Design Sensations exhibition, co-curated by onedotzero.
Working with Seeper, we created a vision driven interactive stand for Brandwidth and Toyota IQ. The stand is touring shopping centers in the UK and currently at the Westfield Shopping center in London.
I'm delighted to announce that my "Gold" installation has been selected to be shown at the Tent London exhibition as part of the London Design Festival. 24-27th September 2009, at the Truman Brewery, London, UK. Stay tuned for more information.
The video below is an early demo of the installation.
“Gold” is an interactive installation which explores our obsession with super-stardom, and the extravagance that accompanies it. Through a ‘magic mirror’, revel in a world of excess where you are the super-star. Shower in glittery gold, experience almost omnipotent powers as you materialize, morph and dematerialize into pure sparkling gold dust. Immortalize yourself as a shimmering golden statue, before you collapse and fade away.
The installation uses custom software written with openFrameworks and the OpenCV computer vision library. The software analyzes the video feed from infra-red cameras in real-time and generates 1080p HD output using OpenGL.
Working with Arnold Agency and Todd Vanderlin+Ryan Habbyshaw from their R&D team, we created an interactive display for Citizens bank in branches in major cities in the US.
Using motion tracking, the pedestrians can interact with the display, signaling birds to come flying in and drop coins, grow plants, create wind to blow the plants around and spray pollen etc. The display is also time-reactive, automatically theming itself depending on time of day.
My Secret Heart is a music and film installation & performance commissioned by Streetwise Opera with music composed by Mira Calix and sound design by David Sheppard. Working with video artists Flat-e, we created a film to accompany the 48 minute performance, as well as versions for an installation and short film.
Streetwise Opera
Streetwise Opera are a charity who use music as a tool to help people who have experienced homelessness move forward in their lives. They run a weekly music programme, resident in 10 homeless centres around the country - and also stage an annual production which gives their performers the chance to star in quality shows where there are high-expectations, no compromise and no patronising. The voices you hear in the music, and people you see in the film, are from Streetwise workshops around the UK. 100+ Streetwise performers also sang at the My Secret Heart premiere at the Royal Festival Hall in December 2008. My Secret Heart is about their story.
The film
The film has an abstract narrative derived from individual conversations with each of the Streetwise performers. It is a direct emotional response to their stories combined with the haunting beauty of Mira Calix's composition. Instead of focusing on a specific plot, the film embarks on a complex journey through various states of emotion, starting from pre-birth through birth, curiosity, exploration, excitement, playfulness; through to fear, anxiety and isolation. While it maintains a relatively dark and eerie mood overall, intertwined with the feelings of desperation are strong elements of hope.
The visuals were designed and created primarily with custom software written in C++/openFrameworks, with some Quartz Composer elements, rendered AfterEffects sequences and live action footage. The custom C++ app is audio-reactive and user-interactive, allowing the visuals to be 'performed' live with full control over the behaviour of the virtual inhabitants of the cylindrical aquarium-like rig.
Over the course of a few months, and after many conversations with Mira Calix and listening to the soundtrack over and over and over again, we decided roughly what the visuals should do and what kind of behaviours we wanted the visuals to perform at specific points in the song. After a lengthy coding period, I had an application that when you ran, did... nothing, but it had the potential to do everything I wanted. The application was a live performance tool with full control over its environment as well as audio playback and control, and an input recording / playback system.
Once the application was complete, I sat down with Robin from flat-e, and pressed 'play' on the app - this started the music playback and the physics recorder. While the music was playing we could control the inhabitants of the virtual world with many sliders, knobs, touchpads, mouse etc. As the music was playing we would respond in realtime by sending messages to make them move gracefully, erratically, flocking together, swimming apart, getting excited, slowing down, speeding up, telling them to die, slowly start twitching, come alive, swim to the surface, sink to the bottom etc - our actions being recorded gave us the ability to later go back and scrub to certain positions in the song and overdub and mix new behaviours we might have missed in the first round. In the end we found that actually we had to do little to no editing. The best overall performance was the one we recorded in a single 50 minute take.
The sensation of performing and recording the visuals was that of actually directing a film with thousands of virtual actors, commanding an army, digital puppetry - an approach I'm sure I will be revisiting in the very near future.
Early tests of visuals on the 'aquarium' (rig built by Gaianova):
I was fortunate enough to be part of the OF Lab team this year at Ars Electronica in Linz, Austria.
Apart from building a 3 storey lab, tons of random little bits of software and visualizations for our environment, we also had the challenge of creating pieces inspired by 5 words submitted by the public - with only a few hours upto a day per project.
Below is an excerpt from "one second before big bang". Visuals all realtime and purely controlled by motion.
"Roots" is an interactive musical/visual installation for the Brick Table tangible and multi-touch interface, where multiple people can collaborate in making generative music in a dynamic & visually responsive environment. It is a collaborative effort between myself and the Brick Table creators Jordan Hochenbaum & Owen Vallis. It will premiere at the Minitek Music + Innovation Festival September 12-14, 2008 in New York.
The essence of the interaction, is that you control parameters of a chaotic environment - which affect the behaviour of its inhabitants - which create and control music.
To breakdown very briefly without going into much detail:
There are vinelike structures branching and wandering around on the table. They live and move in an environment governed by chaos.
Audio is triggered and controlled entirely by how and where the branches move.
You - the user - control various parameters of the chaotic environment. Parameters which range from introducing varying amounts of order, to simply changing certain properties to let the chaos evolve in different directions.
There are varying levels of interaction, ranging from traditional one-to-one correlations - 'this movement I make creates that sound', but also to more complex relationships along the lines of 'this movement I make affects the environment in this way which sends the music into that direction where it evolves with a life of its own'. The visuals are purely generative, as is the audio, and as user you can play with the parameters of that system and watch and listen to the results...
Demo of drawing with roots:
Demo of using fiducials to create magnetic force fields:
"Pi" is an interactive audio/visual installation commissioned by Trash City of the Glastonbury Festival to be shown at the festival in June 2008.
Working with arts and technology collective Seeper, our concept was to take a 50ft tent, and convert it into a giant audio/visual instrument - all of the music, audio and visuals inside the tent are generated and controlled purely by the movements of the occupants.
The space was divided into 6 zones. Two of the zones were purely visual, this was the waiting area. Here people could dance, chill, run about and do what they pleased. Two cameras tracked their movement and applied it to the fluid/particles visuals - so people could 'throw' plasma balls at each other, or send colorful waves propagating around the space. The other 4 zones had the same visual interactions, but in addition were also connected to an audio system. Each of these four zones was allocated an instrument type (drums/beats/percussion, pads, bass, strings etc.), and movement within these zones would also trigger notes or beats - depending on precisely where in the zone the movement was triggered. A lot of effort went into designing the sounds and notes triggered to make sure the end result would almost always sound pleasant and not be complete cacophony.
The first psychedelic fluid/particles interaction prototype developed in processing.org:
Camera -> osc/midi interaction tests (developed in Quartz Composer):
The two concepts strung together and written in C++ with openFrameworks:
