Projection mapping, immersive spaces, and a little interaction. What is today’s media art to people? How does it appear to them? This is the result of looking into those questions. An easy and reasonably convincing way to confirm such perceptions and responses is through search.
Because one can see how many people are making, experiencing, and recording it, most of the texts, links, images, and videos that appear when searching the keyword “media art” on Google and Naver could be summarized by the keywords mentioned at the beginning. This tendency is likely related to screens that approach us as environments, points of contact and connection between virtual and real spaces, and the development of and growing interest in technologies such as virtual, augmented, and extended reality.
The present is a time when the development of technology and people’s interest and preference have come together, resulting in a corresponding increase in diverse cultural and artistic creations related to them.
What is disappointing is whether this flood of content, events, and creations is connected to diversity. Twelve years ago, in 2009, after d’strict and the Department of Information and Culture at Seoul National University presented a media facade using projection mapping on the exterior wall of the Seoul National University Cultural Center, this form and content have shown little change in Korea.
It is either a media facade that projects onto the exterior wall of a building or installs a large LED screen, or an immersive space or exhibition that projects onto multiple surfaces inside a building. Although their scales differ, they provide a kind of illusionary space and an experience of it through images and videos projected onto three-dimensional space. And in this spatial experience, the viewpoint is fixed.
In other words, there is an optimal spot point where this illusion can be seen best. I discovered a new attempt within this projection mapping technique. It proposes a method and experience in which the light source, the starting point of projection, changes fluidly, so that wherever I stand becomes the optimal spot point. This is Joon Moon’s Augmented Shadow – Chasing Stars in Shadow.
What one encounters upon entering the exhibition space is a white space composed of U-shaped temporary walls, a pedestal at the entrance of the space, and a small stair structure placed on one side to the left. To prevent the white floor surface from becoming dirty, I took off my shoes, grasped the handle of an object on the pedestal that looked like a controller, and moved toward the center of the space.
In an instant, the vague white space soon transformed into a rich stage. The controller was the light source that played the leading role in this staging. It seemed like a lantern, or perhaps a candlestick, and around it, a world of shadows unfolded in every direction. Shadows in the form of children appeared on the wall, and one of them gestured as if calling me over.
When I approached, the child pestered me as if asking for something. When I handed the lantern to the child, the child received the light from it and moved through the space with the other children. The children transferred the light to the tips of the sticks they were holding, lowering fishing lines like bait from fishing rods, or using them like magic wands to grow trees or create new entrances and architectural structures on the walls and floor.
Before long, luminous fish gathered toward the light hanging from the fishing rod and roamed throughout the entire space. The tree let down more glowing leaves, another space was created inside the new entrance, and the stage became even richer. The stair structure amplified the spatial illusion by changing the shadow created by the stairs according to the position of the light source held by the children and the light source held by the viewer.
The children moved back and forth between nature and artificial objects, and still seemed conscious of the light source held in the viewer’s hand, dancing in a group around it. Eventually, the space created by the children reached beyond the shadows and the wall, to a dock where the night sky could be seen.
The children boarded a boat moored at the dock, and the boat set off on a journey into the starry night sky. As I saw them off, the stage returned once again to the white space.
Joon Moon’s Augmented Shadow – Chasing Stars in Shadow is an interactive projection mapping work based on a tracker and base station system, a position-tracking sensor system used in VR devices.
The illusionary space unfolded by the projector’s images projected onto three walls and one floor surface belongs to projection mapping, but the tracker carried by the viewer upon entering creates another flow of experience. It is anamorphic content reflected in real time.
Here, anamorphic refers to an optical illusion in which a situation appears differently depending on the angle from which an object is viewed. It is another kind of perspective-based illusion content, familiar to us through trick art. Such anamorphic images are fragments of images that have no meaning from other angles, but from a specific point, they combine and approach us as an impressive image, three-dimensional form, or situation.
When this is realized as a virtual space by a computer and projected onto three-dimensional surfaces through technologies such as projection mapping, it is presented as an even more augmented situation.
The core of the construction of the illusionary space in this work is the shadow, and the tracker, a position-tracking sensor that allows this shadow to change in real time according to the viewpoint. As mentioned earlier, the tracker held in the viewer’s hand operates as a light source radiating in all directions. Centered on this object, the space constantly changes.
The tracker installed here and the base station installed at the upper part of the wall recognize eye level based on the viewer’s coordinate data and transmit it to the computer. It is interesting that this recognition and tracking technology is called Lighthouse.
The computer unfolds in real time a virtual space in which shadows are cast, and projects it into the space through the projector. When the viewer moves, the shadows change in real time according to that position. The viewer feels as if the object in her hand is a light source, and the shadows cast by this light source make the space feel three-dimensional.
This world, composed entirely of shadows, responds when the viewer moves the tracker or moves around, casting the shadows even longer around her, enlarging them, or forming them shallowly or deeply.
If these shadows unfold in dependence on the viewer and the light source, making the depth of the space more concrete, the group dance of yellow fish that appears in the middle fills the space between the shadows and myself, a space that inevitably carries a sense of distance, bringing the viewer and the work even closer together.
The viewer is not so much an observer as a visitor and explorer of this place. Her movement is the starting point of all change. Only when she moves does the surrounding environment change, and only when she hands over the light do the children’s movements begin. The children here receive the light and use it to call fish, grow trees, create new doors, and add the inner spaces beyond them to the viewer’s new path.
The exchange with the children transforms the space into something wider and deeper, ultimately reaching a dock on a sea of stars, which is both the endpoint of the closed space through which one has wandered and an open space from which the children will depart for a new adventure. As the children board a boat for a new journey and move toward the sea of stars, the viewer’s story, the experience of the work, comes to an end.
The story and experience, in which the viewer takes the first step holding a glowing light source and hands that light over in response to the gestures of the various shadows calling her, are simple and concise.
Yet the situations and environments that change diversely as the light carried by the shadows gives depth to the flat space, and the lingering impression of a new journey and farewell into the night sky at the end, are no longer simple. The experience beyond imagination shared with the children is joyful, and the imagination of the children’s future journey is joyful as well.
The events and worlds projected as images have moved beyond the flat rectangular screen through changes in the way devices such as projection mapping are used. The screen has become the surrounding environment, and also a world. We have moved beyond the situation of “viewing” a screen as if looking at scenery outside a window, and now “experience” the image environment as if walking through it.
Alongside the physical world, we perceive the new world we have created, the virtual world, on the same level and in overlap, stimulating new perceptions and sensations. However, as with perspective, a classical technique of representing three dimensions in two dimensions and producing illusion, the representational and viewing points in many contemporary projection mapping and immersive space creations remain fixed at a single optimized point.
In this situation of recognized yet limited, ambiguous imbalance, the artist’s attempt in this work was a fresh step. This work was welcome because it is a different approach to and connection with projection mapping, immersive space, and augmented reality, as well as a variation on existing grammar.
Reflecting once again on the fundamental point of emergence in media art — an approach to, use of, and kind of hacking of a certain technology or technological environment from a different perspective — I look forward to his next step.
