Granular Neural Music & Audio with Magnitude Networks.
Deep neural network audio synthesis. More information and paper coming soon.
Note, the goal here is *not* to create sounds that sound just like the training data (i.e. capable of passing an ‘audio Turing test’), but to create sounds that are somehow reminiscent of the training data, but are totally novel, interesting, and – most importantly – allow a human user to expressively, and meaningfully manipulate the output in realtime. (NB currently none of the sounds on this page were created interactively, this is just the foundation. Interaction is the next step).
Created during my PhD at Goldsmiths, funded by the EPSRC UK.
Long sequence morphs aka #DeepMix. These are not crossfades, but morphing trajectories in z. At points, hopefully creating interesting 'crossover' sounds which contain characteristics from both source and target.
Short sequence morphs. Same as above but on short loops. Hopefully showing how the spectra morphs and grows from sound to sound as opposed to crossfading. NB this is the model trained on Chopin, but feeding speech through it, so the output already has characteristics of both piano and voice.
Time-stretching aka z-trajectory stretching (aka #DeepSmear). A bit like 'Paulstretch'. But instead of repeating blocks (or 'grains') of sound as Paulstretch does, this morphs them (as above). Videos below at 600% and 1500%, with increasing amounts of grain overlap (to 'smooth' the sound out and minimise the tremolo effect). NB. This is the model from 'Long Sequence Morphs' trained on music, and then fed my voice through it, so it has characteristics of both my voice, and various instruments.
Off-by-one error turns what was supposed to be speech, into a 90s IDM-style loop.
Miscellaneous cross-model hacks (putting very unfamiliar sounds through models trained on very different data (e.g. Brenda Lee or Vivaldi through a model trained on Chopin Piano Nocturnes etc)