Software
Roy Block is written in Processing and uses the excellent reacTIVision software for tracking.
A screenshot of an early version. At this stage it is only joints and bones.
The core of the software is the hero's physics model. Although the body looks “solid” it actually only consists of a set of joints which are held together by bones. For no particular reason there are no arms… The resulting structure is equipped with a sense of balance.
Hardware
The setup mainly consists of the following items:
A perspex board in conjunction with tracing paper on which everything is projected. When a block, which has a marker for the tracking on one side, is pressed on the tracing paper then it is becoming completely transparent.
A camera (Sony Mini-DV with Nightshot mode) behind the board tracks the markers attached to the blocks. The camera is equipped with an IR pass filter (LEE #87) so it does not see the projection.
From the rear the board is illuminated with (invisible) IR light coming from two simple and quite old healing lamps. It is crucial that the light is scattered. So I put the lamps into boxes that are wallpapered with aluminum foil which I had crumpled before. Each box has a window (~22x22cm) that is covered with 3 x primary blue and 1 x primary red LEE filter.