I've been quite busy lately directing and producing the visuals for the upcoming street dance show Blaze directed by Anthony Van Laast (Mamma Mia!, Joseph And The Amazing Technicolor Dreamcoat, Jesus Christ Superstar). Working with Robin McNicholas and Jane Laurie we designed and created all the visuals, and the system to play and map them onto the set (more on this below). It previews tonight at the Peacock / Sadlers Wells in Holborn, London UK, plays for 2 weeks, then off on a tour of Holland and UK.
More information about the visuals coming soon - but to quickly summarize, it involves camera-tracking breakers, projection mapping onto an extremely intricate set designed by Es Devlin (Kanye West, Pet Shop Boys, Lady Gaga) - and it's a touring show - and I'm not touring with it - and some shows are one night stands - and the set is projected on by two projectors with very very short throws and huge amounts of lens distortion! So you can imagine the complexity in devising a setup which is tourable and easy quick to setup and calibrate, and still provides us the power to map the fine detail. All powered by custom software of course.
Huge thanks to the amazing production crew, super talented breakers, all the other creatives and dancers, and special thanks to Theo Watson and François Wunschel.
For the Grammy awards 2010, musician & artist Imogen Heap wanted a dress that would display the tweets and photos coming from her fans in realtime, so she could take her fans with her onto the red carpet. Moritz Waldemeyer designed a flexible LED ribbon which she could wear, and I developed the controlling software, an iPod Touch application that she could carry in her bag that would collect all the tweets and photos from the net, and send the information to the custom LED ribbon.
I designed and programmed visuals (projections) for the OKGo performance and Fendi installation at Design Miami 2009. OKGo were playing modified Les Paul guitars mounted with laser beams in the headstock, designed by Fendi in collaboration with Moritz Waldemeyer. Using a PC equipped with a high-speed firewire camera, I developed custom software to track the laser beams and generate visuals around the spots they hit the wall. These visuals were also audio-reactive, responding to the live audio feed coming from the sounddesk.
More images & video coverage coming soon.
P.S. For additional laser tracking goodness, checkout the Graffiti Research Lab's Laser Tag.
I am happy to announce that my Body Paint installation will be exhibited at the Victoria & Albert Musuem from 8th December 2009, to 11 April 2010 along with many other great works, for the Decode: Digital Design Sensations exhibition, co-curated by onedotzero.
An interactive, generative, audio-reactive visualization app for the iPhone and iPod Touch (what a mouthful!) (This is the continuation of what I originally called "The Meshulator").
Use your fingers to create and interact with abstract geometry in 2D or 3D. Very simple to use, just slide your fingers over the screen and try different gestures. Tilt the device to look at the shape from different angles. Options for two different kinds of audio-reactive behaviour (microphone input - iPhone only): deforming based on audio, or completely generative driven by audio. If you feel adventurous, you can play with plenty of customizable parameters to adjust coloring, physics, shape etc. Or if that feels too intimidating just click 'randomize' to make the settings random and experiment.
It was written with a custom version of openFrameworks (pre-006) and an early version of the ofxiPhone addon. Information on openFrameworks and ofxiPhone at memo.tv/ofxiphone.
148apps.com - 4/5 stars."You will find yourself showing people this application …and then, not getting your iPhone back because they want to keep painting."
geek.com - "I like the fact that the app is not just a port of the website, but includes enhancements which take advantage of the features of the iPhone."
iphonefreakz.com - "...the most jollificating paint app for iPhone and iPod Touch. It’s like getting a Jackson Pollock soul inside you...Definitely worth having it on your phone for. Well worth $0.99"
iPhone adaptation of Miltos Manetas' website jacksonpollock.org (original flash developed by Stamen Design). Save your pictures and send them to jackson@jacksonpollock.org, the best Pollocks will be published in an upcoming book.
The video below shows the basic features of v1.0. In the current version (v1.1) ability to choose your own colors has been added to allow the creation of more controlled paintings like those seen in the images below.
"Reincarnation" is an off-shoot from a visual performance for the Rambert Dance Company's "Iatrogenesis" at the Queen Elizabeth Hall, South Bank, London UK - visuals for the latter directed by my good friends at flat-e.
The project began when I started working with footage of the Rambert dancers. Inspired by the footage, I wrote custom software to track the motion and generate the visuals you see in the video below... and "Reincarnation" was born.
For the Rambert's "Iatrogenesis" piece, I created similar visuals, abstract visual layers containing subtle hints of human forms and motion, which would tie in with the movement of the dancers on stage. These were used by flat-e and composited with various other layers and projected on a see-through gauze in front of the stage.
The video (and music) you see below is not representative of the visuals (and music) of the Rambert's and flat-e's "Iatrogenesis". This is just a standalone piece born from working with the Rambert choreography.
When the clip starts, you probably won't recognize a human shape at first, but your eyes and mind will be searching, seeking mental connections between abstract shapes and recognizable patterns, like looking for shapes in clouds. You'll be questioning what you see, is that it? is it sitting? is it crouching? is it kneeling? Then all of a sudden, it'll be crystal clear. Then you'll try and keep it in focus, following it as it moves around, tracking each limb, using the motion to construct an image of the parts you can't see. It'll fade in and out of clarity. At times you'll be clinging onto just the tip of it's hand swinging round, trying to identify any other recognizable parts. You might see another arm or leg and grab onto it, fighting not to lose it. Then it'll be crystal clear again, and then all of a sudden vanish, literally in a puff of smoke, and your eyes will start searching again.
Working with Seeper, we created a vision driven interactive stand for Brandwidth and Toyota IQ. The stand is touring shopping centers in the UK and currently at the Westfield Shopping center in London.
I'm delighted to announce that my "Gold" installation has been selected to be shown at the Tent London exhibition as part of the London Design Festival. 24-27th September 2009, at the Truman Brewery, London, UK. Stay tuned for more information.
The video below is an early demo of the installation.
“Gold” is an interactive installation which explores our obsession with super-stardom, and the extravagance that accompanies it. Through a ‘magic mirror’, revel in a world of excess where you are the super-star. Shower in glittery gold, experience almost omnipotent powers as you materialize, morph and dematerialize into pure sparkling gold dust. Immortalize yourself as a shimmering golden statue, before you collapse and fade away.
The installation uses custom software written with openFrameworks and the OpenCV computer vision library. The software analyzes the video feed from infra-red cameras in real-time and generates 1080p HD output using OpenGL.
Working with Arnold Agency and Todd Vanderlin+Ryan Habbyshaw from their R&D team, we created an interactive display for Citizens bank in branches in major cities in the US.
Using motion tracking, the pedestrians can interact with the display, signaling birds to come flying in and drop coins, grow plants, create wind to blow the plants around and spray pollen etc. The display is also time-reactive, automatically theming itself depending on time of day.
My Secret Heart is a music and film installation & performance commissioned by Streetwise Opera with music composed by Mira Calix and sound design by David Sheppard. Working with video artists Flat-e, we created a film to accompany the 48 minute performance, as well as versions for an installation and short film.
Streetwise Opera
Streetwise Opera are a charity who use music as a tool to help people who have experienced homelessness move forward in their lives. They run a weekly music programme, resident in 10 homeless centres around the country - and also stage an annual production which gives their performers the chance to star in quality shows where there are high-expectations, no compromise and no patronising. The voices you hear in the music, and people you see in the film, are from Streetwise workshops around the UK. 100+ Streetwise performers also sang at the My Secret Heart premiere at the Royal Festival Hall in December 2008. My Secret Heart is about their story.
The film
The film has an abstract narrative derived from individual conversations with each of the Streetwise performers. It is a direct emotional response to their stories combined with the haunting beauty of Mira Calix's composition. Instead of focusing on a specific plot, the film embarks on a complex journey through various states of emotion, starting from pre-birth through birth, curiosity, exploration, excitement, playfulness; through to fear, anxiety and isolation. While it maintains a relatively dark and eerie mood overall, intertwined with the feelings of desperation are strong elements of hope.
The visuals were designed and created primarily with custom software written in C++/openFrameworks, with some Quartz Composer elements, rendered AfterEffects sequences and live action footage. The custom C++ app is audio-reactive and user-interactive, allowing the visuals to be 'performed' live with full control over the behaviour of the virtual inhabitants of the cylindrical aquarium-like rig.
Over the course of a few months, and after many conversations with Mira Calix and listening to the soundtrack over and over and over again, we decided roughly what the visuals should do and what kind of behaviours we wanted the visuals to perform at specific points in the song. After a lengthy coding period, I had an application that when you ran, did... nothing, but it had the potential to do everything I wanted. The application was a live performance tool with full control over its environment as well as audio playback and control, and an input recording / playback system.
Once the application was complete, I sat down with Robin from flat-e, and pressed 'play' on the app - this started the music playback and the physics recorder. While the music was playing we could control the inhabitants of the virtual world with many sliders, knobs, touchpads, mouse etc. As the music was playing we would respond in realtime by sending messages to make them move gracefully, erratically, flocking together, swimming apart, getting excited, slowing down, speeding up, telling them to die, slowly start twitching, come alive, swim to the surface, sink to the bottom etc - our actions being recorded gave us the ability to later go back and scrub to certain positions in the song and overdub and mix new behaviours we might have missed in the first round. In the end we found that actually we had to do little to no editing. The best overall performance was the one we recorded in a single 50 minute take.
The sensation of performing and recording the visuals was that of actually directing a film with thousands of virtual actors, commanding an army, digital puppetry - an approach I'm sure I will be revisiting in the very near future.
Early tests of visuals on the 'aquarium' (rig built by Gaianova):
I was fortunate enough to be part of the OF Lab team this year at Ars Electronica in Linz, Austria.
Apart from building a 3 storey lab, tons of random little bits of software and visualizations for our environment, we also had the challenge of creating pieces inspired by 5 words submitted by the public - with only a few hours upto a day per project.
Below is an excerpt from "one second before big bang". Visuals all realtime and purely controlled by motion.
"Roots" is an interactive musical/visual installation for the Brick Table tangible and multi-touch interface, where multiple people can collaborate in making generative music in a dynamic & visually responsive environment. It is a collaborative effort between myself and the Brick Table creators Jordan Hochenbaum & Owen Vallis. It will premiere at the Minitek Music + Innovation Festival September 12-14, 2008 in New York.
The essence of the interaction, is that you control parameters of a chaotic environment - which affect the behaviour of its inhabitants - which create and control music.
To breakdown very briefly without going into much detail:
There are vinelike structures branching and wandering around on the table. They live and move in an environment governed by chaos.
Audio is triggered and controlled entirely by how and where the branches move.
You - the user - control various parameters of the chaotic environment. Parameters which range from introducing varying amounts of order, to simply changing certain properties to let the chaos evolve in different directions.
There are varying levels of interaction, ranging from traditional one-to-one correlations - 'this movement I make creates that sound', but also to more complex relationships along the lines of 'this movement I make affects the environment in this way which sends the music into that direction where it evolves with a life of its own'. The visuals are purely generative, as is the audio, and as user you can play with the parameters of that system and watch and listen to the results...
Demo of drawing with roots:
Demo of using fiducials to create magnetic force fields:
I was sitting minding my own business, exploring accumulation buffers in Quartz Composer, when all of a sudden I just zoned out, and next thing I know, I found myself staring at Him on my screen. His Noodly Appendages came down and touched me, and guided my hands, connecting Quartz Composer's very own noodles in His Image.
His Noodly Screen Saver runs on Mac OSX 10.5 (Leopard) and you can download it below.
I've just finished an interactive Quartz Composer project with Moving Brands for an American multinational corporation with a focus on designing and manufacturing consumer electronics and software products. Unfortunately due to the strict nature of PR, I cannot enclose any more information than this, but stay tuned...
"Pi" is an interactive audio/visual installation commissioned by Trash City of the Glastonbury Festival to be shown at the festival in June 2008.
Working with arts and technology collective Seeper, our concept was to take a 50ft tent, and convert it into a giant audio/visual instrument - all of the music, audio and visuals inside the tent are generated and controlled purely by the movements of the occupants.
The space was divided into 6 zones. Two of the zones were purely visual, this was the waiting area. Here people could dance, chill, run about and do what they pleased. Two cameras tracked their movement and applied it to the fluid/particles visuals - so people could 'throw' plasma balls at each other, or send colorful waves propagating around the space. The other 4 zones had the same visual interactions, but in addition were also connected to an audio system. Each of these four zones was allocated an instrument type (drums/beats/percussion, pads, bass, strings etc.), and movement within these zones would also trigger notes or beats - depending on precisely where in the zone the movement was triggered. A lot of effort went into designing the sounds and notes triggered to make sure the end result would almost always sound pleasant and not be complete cacophony.
The first psychedelic fluid/particles interaction prototype developed in processing.org:
Camera -> osc/midi interaction tests (developed in Quartz Composer):
The two concepts strung together and written in C++ with openFrameworks:
Aldeburgh Music is an organization based in Suffolk, UK working with musicians - both professional and just starting out - to help them reach their full potential by providing them with the time and space to discover, create and explore - as well as providing inspirational scenery and a rich musical heritage.
The New Music New Media / Britten–Pears Programme offers advanced performance experience to young professional musicians in the inspiring surroundings of Snape Maltings, home of the Aldeburgh Festival founded by Benjamin Britten in 1948.
Onset assistant director and visual effects supervisor, also responsible for post production on 60second commercial for clothing brand Riva. 3-day shoot in studio (mainly greenscreen shoot) in south France. Post production work involved keying, tracking, virtual set building, stabilizing, compositing, particles, cloth simulations and motion graphics.
Responsible for post production, motion graphics and visual effects on a 45 second video promoting ‘Texas
Instruments Making Wireless’ Technology. Shot on HD in greenscreen
studio. Video will be shown at the Making Wireless World Congress in Barcelona.
Created full flash portfolio website for illustrator and designer Jane Laurie - www.janelaurie.com. Site is entirely data driven via a PHP/MySQL backend. Front end is fully object oriented and written in AS3. Client can login and add/edit/remove images and text.
A volleyball derivative flash game posing as a christmas card. Fully themeable and currently includes 2 skins, main skin by Dirk van Dijk, 2nd skin by Jane Laurie.
Post work (keying, roto, grade, ramping etc.) on footage of top Adidas sportsmen (tennis player Novak Djokovic, footballer Kaka, track athlete Jeremy Wariner, sprinter Tyson Gay, and basketball player Dwight Howard) for adidas techfit mini-site. See it at www.adidas.com/techfit.
