I’m working on a little world, or at least a window into one. One thing I want to try is digitally merging a live video stream with other elements, such as computer-generated imagery or prerecorded video. This way I can create something that looks like a video, but that reacts in real-time to movement or sound. I like the idea of making something that appears at first to be a video loop, but is actually a computer simulation with endless variation. This is sort of how procedural generation works in game design.
For A House By The Water I want to build a small model of an island or coastline, but I’m still figuring out what materials to use, and how big it needs to be. For now, a rock will do. Today I walked to the studio along a slightly different route, past a field and a construction site, where I found a large white stone.
I spent most of the afternoon putting together some code to knock out the background from a webcam stream. A blue recycling bin made a pretty good “green screen”, and I was able to superimpose live video of the rock on top of another video (a couple of seagulls). I made a little clip of this process while I moved a light around, so you can see what’s happening.
I like having the rock here. My studio desk is full of books and equipment, and the rock brings a different sort of presence and purpose. I often work with things that only exist on a screen, and it’s a reminder to stay grounded in the real world. I’ll keep it around so it can keep an eye on me.
I arrived at The Rooms to find a team of employees evicting my roommates, the tall sculptures left behind by the previous artist in residence. So I have the place to myself now. I feel like capturing the space in some way. I’ve brought my camera, but I set up the Kinect sensor instead. It captures video and also depth information, which I can visualize using Processing. I start with Daniel Shiffman’s Point Cloud demo and spend some time modifying the code. I take it for a spin.
Thirty times a second, the Kinect takes two pictures, compares them, and spits back a bunch of numbers representing the distance of each pixel from the sensor. The Point Cloud script reinterprets these numbers and projects each pixel in a simulated three-dimensional space. Watching the demo in real-time, it feels strange to see yourself whirl around while the camera stays still. The script rotates the scene around an imaginary point in the virtual space.
I want to try something else. I place the sensor on top of a ladder and aim it at the ceiling, which is full of lights, pipes, ventilation and a dangling extension cord. I increase the density of the depth image, and adjust the colours used for the projection. When I end up with something I like, I export screenshots of each frame and stitch them together into a gif animation.
I like this curious architectural snapshot. The virtual rotation creates a strange illusion where it feels as if the camera arcs through the ceiling and looks down through it, into the room. The depth projection is interrupted by the shadows of objects closer to the camera (the dark holes in the orange surface). This is due a limitation of the stereo vision system, but I enjoy the theatrical quality it adds to the scene, like a stage set only meant to be seen from one angle. Or as if the world has unfolded from flatness, like a pop-up book.
One of the books I brought to the studio is In Praise of Shadows by Jun’ichirō Tanizaki. Written in 1933, it’s a wonderfully ruminative essay on Japanese aesthetics, the beauty of impermanence, and shadows in architecture:
An empty space is marked off with plain wood and plain walls, so that the light drawn into it forms dim shadows within emptiness. There is nothing more. And yet, when we gaze into the darkness that gathers behind the crossbeam, around the flower vase, beneath the shelves, though we know perfectly well it is mere shadow, we are overcome with the feeling that in this small corner of the atmosphere there reigns complete and utter silence; that here in the darkness immutable tranquility holds sway.
Content © 2017 Matthew Hollett. RSS