Projection mapping / quad warping with Quartz Composer & VDMX

This is a demo of projection mapping with VDMX & Quartz Composer inspired by deepvisual's tutorial of doing it in modul8 (uk.youtube.com/watch?v=2bRfdn9lNO8).

VDMX unfortunately doesn't have this feature built-in, but fortunately has beautiful integration with Quartz Composer - allowing me to build a quad warper in QC using a GLSL vertex shader, which should be super fast.

Also, around the 4:30 mark you'll see me masking the video on the box in the back. This is also using a custom Quartz Composition which allows 4 point mask creation. Usage is almost identical to the QuadWarper, but instead of warping the image it just applies a mask, or you can invert the mask and it cuts a chunk out. You could do the same by creating new layers, grouping, using as a layer mask etc. but its a a bit more hassle I think. Using the QuadMask is a lot quicker and you can put multiple QuadMasks on the same layer to draw more complex masks.

Check out this brilliant quartz composition by Daniele Ciabattoni using kineme's opencv plugin and my quad warp glsl patch. Ace!

dynamic surface projection... from Daniele Ciabattoni on Vimeo.

The theory on 4 point warping is explained quite well on Paul Heckbert's Master's Thesis "Fundamentals of Texture Mapping and Image Warping" (www.cs.cmu.edu/~ph/texfund/texfund.pdf). It's a bit dated (1989!) but still relevant.

To cut a long story short: for any point in a normalized quad (0,0 - 1,1), first interpolate along the bottom edge of the new corners to find the bottom position. Then interpolate along the top edge of the new corners to find the top position. Then interpolate between those two points to find the transformed position of the point. If the quad isn't initally a normalized quad (i.e. like the QC coordinate system), it needs to be normalized (a linear map takes care of that).

I implemented this method using a vertex shader by moving a grids vertices depending on the input 4 coordinates for the new corner positions.

uniform vec2 BL, BR, TL, TR;
uniform vec2 renderSize;
 
void main() {
	// transform from QC object coords to 0...1
	vec2 p = (vec2(gl_Vertex.x, gl_Vertex.y) + 1.) * 0.5;
 
	// interpolate bottom edge x coordinate
	vec2 x1 = mix(BL, BR, p.x);
 
	// interpolate top edge x coordinate
	vec2 x2 = mix(TL, TR, p.x);
 
	// interpolate y position
	p = mix(x1, x2, p.y);
 
	// transform from 0...1 to QC screen coords
	p = (p  - 0.5) * renderSize;
 
 
	gl_Position 	= gl_ModelViewProjectionMatrix * vec4(p, 0, 1);
	gl_FrontColor	= gl_Color;
	gl_TexCoord[0]	= gl_TextureMatrix[0] * gl_MultiTexCoord0;
}

I use almost the same shader code for the QuadMask, except that one doesn't need a texture and is just used to create a white quad to be used as a mask.
The 2 Quartz Compositions can be downloaded from the vdmx archive at vdmx.memo.tv. (they are setup to be used as qcFX in VDMX, but can be used standalone in Quartz Composer with no modiications, just connect image providers to the published image inputs). QC3 (Leopard) only I'm afraid.

P.S. In this vid you'll see I have a background layer with the scene in it. Obviously normally you don't need this layer, in VDMX, you can adjust the warping directly observing the projection area. I have it in here now because it makes demonstrating and recording this video easier.

This technique of projection mapping works quite well if projecting onto basic flat surfaces, but if you have a more complex setup with curved surfaces then you will need a different approach (e.g. local.wasp.uwa.edu.au/~pbourke/miscellaneous/warppatch/ or vvvv.org/tiki-index.php?page=How+To+Project+On+Complex+Geometry).