I'm a writer and visual artist in St. John's, Newfoundland, Canada.

For more frequent updates, follow me on Instagram or subscribe to my email newsletter.

Posts tagged “Digital Art” (page 1 of 3)

Moss and misunderstanding

I have not been unhappy for ten thousand years.
During the day I laugh and during the night I sleep.
My favourite cooks prepare my meals,
my body cleans and repairs itself,
and all my work goes well.

—Leonard Cohen, from I Have Not Lingered In European Monasteries

A House By The Water

Some days I stare at a screen and some days I stare at the sea. I’m spending most afternoons programming A House By The Water, a digital projection of houses falling into the ocean, as part of an artist residency at The Rooms. I’m programming in Processing, which is pretty old-fashioned as far as computer graphics go – no fancy lighting engines or physics libraries. The computer talks in rectangles. I’ve written a program that draws houses out of boxes. Each box has a roof, which is eight rectangles and two triangles and a texture. Each box has a chimney. Some of the boxes have windows and doors. The boxes don’t know each other exist, so each house is full of vestigial chimneys that don’t reach the roof, and falling houses pass through each other like ghosts. I spend an entire evening failing to calculate shadows.

The computer talks in rectangles, but it breathes in loops. Loops are its essence. A rectangle is a loop if you untangle it enough (one, two, three, four sides). I write loops for stacking boxes into houses, loops for the choppy vector mesh that simulates the water’s surface, loops for the effect of gravity on a falling house, loops to control the translucency of shadows. Each loop is a set of instructions that repeats itself, slightly differently each time. Some iterations are faster than others. I spend hours rewriting code and the scene looks almost exactly the same as before, but the movement is smoother, or I can generate more houses at once without affecting the framerate.

The work so far, although it does not contain language, is very textual: I’ve made a video of little house-shaped points of light washing up on a shore, and the lights read almost as letters, the house shapes suggesting something different when singular or clustered. Similarly, the projection of falling houses is composed out of code, a kind of language. Making the art is mostly an act of writing.

So the computer and I communicate, often misunderstanding each other. I accidentally instruct it to draw the 2D background in the same three-dimensional space as the houses, and the scene collapses like a theatrical backdrop, houses tumbling into an empty grey aether. Or I forget to erase between frames and each house is drawn again and again as it rotates, spinning into itself like a ball of yarn.

A House By The Water (glitch)

On the weekends I get my eyes away from the machine. It’s been a soft summer, the city muzzled by clouds, more fog than I remember from last year. Frost warnings in July. I have been photographing moss, and I want to know more about moss, perhaps even grow some in the studio. A couple of friends have a moss garden, and invite me to visit. They’ve taken a worn-out corner of the backyard and reupholstered it in different mosses collected around St. John’s. The garden is a dappled tapestry of colour and texture, with dozens of different varieties: stubbled pool-table baize, spiky tufts tinged in orange, sturdy toothbrush-like bristles. A few rusty patches. The mosses they’ve found in the city seem to thrive, while others, collected from rocks higher up on the Southside Hills, don’t seem to like living downtown.

After the garden, we go moss collecting. In the part of the park furthest from the city, landscaped paths narrow into threadbare trails between trees. The ground is soft and damp, and the grass gives way to shrubs and bog. On either side of the trail, rocks and roots are lush with moss. We find a good-sized patch of the species that seems to do best in the backyard, a riot of tiny leaves in enthusiastic chartreuse. It’s like tearing up carpet. You just kind of lift the edge and slide your hand under, and it all comes up in a clump. We fill a garbage bag with miniature islands.

Moss collecting

What I like about moss is that it is unnoticeable. It keeps a low profile, doesn’t draw attention to itself. Moss is a squatter, colonizing any overlooked surface. It thrives in the in-between places, the damp pockets, the shadows. It lurks behind trees. It’s given up frivolities like roots, flowers, seeds. At the same time, it has a humble hospitality. It offers a seat, a place to think. It obliges visitors but isn’t much of a conversationalist, doesn’t know what to say besides hello. It mumbles a bit, repeats itself softly, likes to mull things over. It’s a welcome mat gone feral.

We walk home through the drizzle. My mind is amorphous, making connections that don’t make any sense. I think: maybe moss is a kind of software. A series of instructions on how to generate more moss. A fuzzy green code, writing itself into the world. Maybe I’m just an interface between the moss and the screen. I breathe and walk in loops: one, two, three, four. My body follows unknowable instructions, cleans and repairs itself. I’m not sure how, but I think I’ve learned something about how to calculate shadows. On Monday, back in the studio, I will have so much to tell the computer.

Improbable geographies

Strange neighbours

Improbable geographies

Still building houses out of numbers and loops. Have stumbled across some strange neighbours and improbable geographies along the way.

Fog camera

Okay, now we’re getting somewhere.

Fog camera chair

I took the code that visualizes the Kinect depth data, and the code that knocks out the background, and combined them. So now, as things get further from the camera, they fade into transparency. I can insert any background here, and it doesn’t have to be the same colour, but for now I’ve put the blue recycling bin to work again.

There was a lazy susan in the studio, luckily enough, and it came in handy for spinning the chair (you can see the edge of it showing up in the clip above). Here’s the setup with the Kinect sensor in the background.

Fog camera setup

There’s something eerie about seeing the chair slip out of view, as if the scene is lit by candlelight. Fading the object into the background introduces a kind of fog, and the chair starts to seem larger, a building looming out of mist. Its four legs begin to read as the corners of a house. I want to use this effect to make a small landscape model feel like an island.

I’m also wondering if I could connect this to a higher-quality camera. The Kinect depth data wouldn’t sync perfectly, but it might work well enough. I’d love to try some photography or video outdoors using this effect. Funny to think about introducing virtual fog to the streets of St. John’s.

Here’s a part of a wonderful poem about embracing fog and flaws.

I tell you it has taken me all my life
to arrive at the vision of gas lamps as angels,
to soften and blur and finally banish
the edges you regret I don’t see,
to learn that the line I called the horizon
does not exist and sky and water,
so long apart, are the same state of being.

—Lisel Mueller, from Monet Refuses the Operation

Working on a little world

I’m working on a little world, or at least a window into one. One thing I want to try is digitally merging a live video stream with other elements, such as computer-generated imagery or prerecorded video. This way I can create something that looks like a video, but that reacts in real-time to movement or sound. I like the idea of making something that appears at first to be a video loop, but is actually a computer simulation with endless variation. This is sort of how procedural generation works in game design.

For A House By The Water I want to build a small model of an island or coastline, but I’m still figuring out what materials to use, and how big it needs to be. For now, a rock will do. Today I walked to the studio along a slightly different route, past a field and a construction site, where I found a large white stone.

I spent most of the afternoon putting together some code to knock out the background from a webcam stream. A blue recycling bin made a pretty good “green screen”, and I was able to superimpose live video of the rock on top of another video (a couple of seagulls). I made a little clip of this process while I moved a light around, so you can see what’s happening.

Green screen rock

I like having the rock here. My studio desk is full of books and equipment, and the rock brings a different sort of presence and purpose. I often work with things that only exist on a screen, and it’s a reminder to stay grounded in the real world. I’ll keep it around so it can keep an eye on me.

Artist in Residence, Day 3

I arrived at The Rooms to find a team of employees evicting my roommates, the tall sculptures left behind by the previous artist in residence. So I have the place to myself now. I feel like capturing the space in some way. I’ve brought my camera, but I set up the Kinect sensor instead. It captures video and also depth information, which I can visualize using Processing. I start with Daniel Shiffman’s Point Cloud demo and spend some time modifying the code. I take it for a spin.

Point Cloud in purple

Thirty times a second, the Kinect takes two pictures, compares them, and spits back a bunch of numbers representing the distance of each pixel from the sensor. The Point Cloud script reinterprets these numbers and projects each pixel in a simulated three-dimensional space. Watching the demo in real-time, it feels strange to see yourself whirl around while the camera stays still. The script rotates the scene around an imaginary point in the virtual space.

I want to try something else. I place the sensor on top of a ladder and aim it at the ceiling, which is full of lights, pipes, ventilation and a dangling extension cord. I increase the density of the depth image, and adjust the colours used for the projection. When I end up with something I like, I export screenshots of each frame and stitch them together into a gif animation.

Point Cloud ceiling

I like this curious architectural snapshot. The virtual rotation creates a strange illusion where it feels as if the camera arcs through the ceiling and looks down through it, into the room. The depth projection is interrupted by the shadows of objects closer to the camera (the dark holes in the orange surface). This is due a limitation of the stereo vision system, but I enjoy the theatrical quality it adds to the scene, like a stage set only meant to be seen from one angle. Or as if the world has unfolded from flatness, like a pop-up book.

One of the books I brought to the studio is In Praise of Shadows by Jun’ichirō Tanizaki. Written in 1933, it’s a wonderfully ruminative essay on Japanese aesthetics, the beauty of impermanence, and shadows in architecture:

An empty space is marked off with plain wood and plain walls, so that the light drawn into it forms dim shadows within emptiness. There is nothing more. And yet, when we gaze into the darkness that gathers behind the crossbeam, around the flower vase, beneath the shelves, though we know perfectly well it is mere shadow, we are overcome with the feeling that in this small corner of the atmosphere there reigns complete and utter silence; that here in the darkness immutable tranquility holds sway.



Trying to draw more lately – here’s a recent iPad attempt.

Content ©  2019 Matthew Hollett. RSS