On Holy Saturday 3 April 2021, Compassie was released.
Compassie delights me more than anything I’ve made before. Because it gives me an opportunity to be sad. Hiding behind the VR headset. Where no-one can see me cry. It feels so luxurious to be able to be sad! Just sad, just to be filled with this enormous feeling of failure and desperation, to let it expand and take over my world, with nobody telling me to cheer up or look at the bright side or to relax or fix myself. Just sitting there in the dark, in the absolute, endless darkness of empty cyberspace.
I’m extremely pleased with the result. Please have a look the Compassie webpage to learn more about the project or to download it and try it yourself, if you have access to VR equipment.
Returning to the Virtual Reality Pietà after four months, while a bit daunted by the amount of production work remaining to be done, I took courage from the idea that the design of the piece was finished. All I needed to do now was to make an epic landscape that transitions over twelve thousand years. Daunting in terms of work, but simple in terms of concept.
But then I tried the prototype.
While aesthetically appealing, I didn’t understand why this scene of a mother holding her dead son required such a spectacular context. A simple transition from day to night in a mundane scene would suffice. I also felt weary about the lesson the piece seemed to be preaching with its manipulative albeit minimal interactivity: lift the dead body in a gently embrace to make the light shine. I felt that the “now is bad, then was good” mantra, or the “we ruined the Garden of Eden” rhetoric was a bit pedantic. And all those cheap glowy lights at night looked too much like cheesy science fiction. Maybe this “edgy contrast” between a traditional religious scene and high tech graphics would increase the appeal of the piece. But how would that interest me?
Two characters in a small garden area would be sufficiently poetic and dramatic. This is how the pietà scene was always depicted in the renaissance and baroque art that I admire. That is how it is done. The strength of a pietà is its simple familiarity. It’s a modest tragic scene. The dramatic consequences should happen in the mind of the viewer, not be expressed by the art.
This is not a deposition
I had always considered this piece to include aspects of both the traditional Deposition scene (taking the dead body down from the cross, a scene that often involves many characters) and the Pietà proper (just a mother and her dead son). In the months away from the project I had developed an idea for a Deposition piece. And so i decided to separate the two. This one needs to focus on the Pietà itself.
It’s all about her sadness, it’s all about her tears. She’s cradling her dead son like a baby.
Any modern invention I might add to the pietà (and it’s easy, and seductive, to come up with ideas) does not improve the scene. We think we’re being clever as contemporary artists, but anything we would add would only reduce the impact of the work. Of course, before modernity, many artists have added new elements to the pietà scene. But, as far as I can tell, this was always done with a sincerity that contemporary artists, including myself, seem virtually incapable of. The old masters always created in service of the scene, of the meaning of the scene, even when they were showing off their skills. As opposed to today’s desire for personal original ideas that “criticize” or “subvert” or in whatever manner add something to the scene that doesn’t belong to it. Or is it just that this is the easy thing to do? The safe thing to do? To make a crude joke about a mother crying over her murdered son is safer now than expressing compassion and grief and allowing that pain to silently exist and grow in meaning.
To maximize the impact of the work, I need to not only trust my own sincerity, but also rely on the tradition of depicting this scene. My own judgement does not suffice. When I imitate, I speak with the voice of thousands. This work requires modesty and respect.
The eye does not see itself
I was also again troubled by the viewpoint in Virtual Reality. Since you do not see yourself, the environment becomes what you look at, when you are cast as the protagonist in the scene. Hence my attention to the landscape that surrounds the scene. But the fact that you don’t see yourself doesn’t mean that you don’t know which role you play! It’s not only about the environment, it’s also about who you are and who you are with.
Once you realize that the viewpoint is reversed in VR other art becomes a lot less useful as inspiration. What does a scene look like from the viewpoint of its subject? Despite of the sculptural nature of realtime 3D I’m more inspired by paintings than by statues. Because they represent worlds. But when I browse through pictures of the pietàs that have been made over the centuries I’m confused about what I am supposed to think about what I am making. I’m not making a painting, I’m making the scene that is represented in the painting. But I’m obviously not creating reality. I could consider this scene a sculpture if the spectator would be positioned outside of it. A virtual sculpture. Okay. But in this case, the spectator is the subject of the sculpture, or they are positioned in the exact place of the subject, playing its role. So is this a form of theater? Only if the actor is their own audience. And while the scene is fictional, the spectator is not. Maybe it’s like a novel written in the first person? Virtual Reality may be too real, insufficiently artificial for me to think of it in artistic terms. And yet the experience, the emotional effect, is very similar to the experience of art.
What changes when a Pietà is no longer a display to be witnessed but a scene to be experienced? There’s no need for Mary to express her grief visibly anymore. You are Mary. Your grief is that of the mother, not for the mother. You compassion is for the son, not the mother. And you think of the son as your own son, but also as the son of God, and how his death, his sacrifice means the salvation of mankind. His death is the foundation of Christianity, the philosophy that would impact Western culture more than anything. Your grief is minor in this context, and it adds an eighth sword of pain that pierces your heart.
There’s an additional dimension to a contemporary image of the Pietà. Because God, famously, has been declared dead in our era. Not just the Son, but also the Father and the Holy Ghost. And according to some, they died for the same reason: as a sacrifice for the salvation of humanity. We have sacrificed God again, this time in order to be saved by science and technology, by what we now consider truth.
The best answer is often nothing
When instead of looking at the virgin, we inhabit her body and look through her eyes, what do we see? Fortunately this is not just an aesthetic or logistical problem. What does a person holding a dead child look at? Nothing much, I presume, it’s not important, the world out there does not matter in this moment.
The way in which Caravaggio submerges his scenes in darkness came to mind.
So there would be nothing to see but the corpse in your arms? The infinite void of an empty scene in VR is impressive. There is nothing there, as far as the eye can see. I was drawn to the radical character of this idea. Although I do love being immersed in an elaborate 3D world. There could be visible objects in the immediate vicinity of the scene: the throne, the floor, plants, some objects. This would satisfy my desire to see real things in VR.
I briefly considered filling the black void with abstract decorations, perhaps expressing, supporting the feelings. But can any decoration express these better than darkness? I tried adding contemporary visuals, to express the mood, to demonstrate the vastness of the endless emptiness in which the mourning mother finds herself. But it all feels corny and out of place. I thought the contrast would be interesting but it just reduces the gravity of the piece.
I was still thinking about the simple garden scene. But after some experimentation I realized that anything out there would capture the gaze of the user. They will look at it and that will become the work of art. So I need to avoid that. Because I don’t want to “express” the emotions in the scene. Art should offer context and stimulus for the spectator’s own emotions and is not an opportunity for the artist to manipulate or impose.
What they see out there should guide them inwards. Not just towards looking down at Jesus on their lap. But towards introspection, towards being not seeing. Perhaps my goal/hope should be for the user to close their eyes. After all, a VR headset feels a bit like a blindfold. Instead of entering another world, the VR headset could enable you to enter yourself.
What would a baroque artist do with VR?
Maybe I have been seduced by power of the Northern Renaissance again, into a problem that cannot be solved in the current age. On the one hand because I obviously lack the artistic skill and on the other because we live in a time in which religious faith is not only sparse but also heavily criticized, and by no means supported universally by society. This reminds more of the Baroque times of Counter-Reformation than of the pious context in which the Flemish Primitives were active.
Maybe I should try to imagine what a baroque artist would do with this technology. How would they deal with the endlessness of simulated space? I’m attracted to baroque art because it contains a certain playfulness and spectacle that seems to fit the digital realm with its abundance, ambiguity and focus on the spectator’s experience. As opposed to the grave and solemn nature of the Northern Renaissance that was the starting point of Cathedral-in-the-Clouds and remains an important reference for the Pietà as well. How would a Baroque artist present a 15th century Pietà in 21st century VR?
To do or not to do
I still had the interaction to consider. In the previous design lifting up the body would transition the world from dark present day to bright paradise. Now I was thinking of a simple transition between day and night. Or a sort of focus: when you lift up the body, only its immediate surroundings would be lit. But the dynamics of cause and effect trouble me. I want to create endless environments, not linear stories. I want to create a context in which the spectator can explore their own thoughts and sentiments. I do not want to guide this process towards what I think is interesting. That would be a waste of opportunity and an unnecessary limitation. But I worry that if there is very little to do that causes a change the experience will feel shallow and short. If, on the other hand, there’s is nothing to do that causes any changes, it can feel endless.
I don’t remember exactly when it happened. I was prototyping all sorts of ideas and at some point I ended up in a scene completely empty and dark in front of me but with a bright landscape behind me. I had recreated the situation of the paintings: the mother with her son on her lap sitting in front of a landscape. We do not know what is in front of the protagonists. It is not depicted. But in the physical context of the museum or a church we are it, the spectators. It feels a bit like being on a theater stage with the actor peering into the darkness where the audience is. You can still lift up the body but nothing happens in the scene when you do. It should happen inside of you. You can look behind you, at the landscape, but it’s very uncomfortable, when sitting down. But it feels good to know that there is a whole world behind you while you are staring into the void of your sorrow.
I’m trying to avoid the undignified effect of errors in both human use as machine function. But errors and clumsiness were in part what attracted me to this theme. Holding a dead body is awkward. It is difficult physically. Especially for a (older) woman to hold the body of a grown man. And it is odd mentally, because we feel we owe this corpse an enormous amount of respect while it cannot respond and we are in control of its motions.
But while I am quite sure anybody would handle this situation gloriously in real life holding a real dead body, the same is not true in VR. In a simulation we know things are fake. We’re in a magic circle where we can experiment with irreverence. The opportunity to interact with a human body beyond the repercussions of every day society is alluring. Moreover we love interaction as such, we are fascinated by machines responding to what we do. We want to see what happens when we do something or other. There is no way that a work of art can demand the same reverence as a dead body. Art is play. Even when it deals with serious themes. Art functions only when we play. We enter the art through play. We have to participate.
While sculptures, paintings, films and poems completely ignore whether or not you’re playing them right, an interactive work of art can actually know if you are. And we can make it respond to this data. So there is a temptation to attempt to force a proper experience. But is that wise? Other art forms leave it up to the player. And the only judging that happens is social. If you burst out laughing in front of a crucifix, a Rothko or and photograph of a starving child, you can expect a reaction from people around you. But the work of art does not change.
In a simulation, however, the art can be changed. The player’s interaction could create a grotesque situation in the virtual space, one that may in fact be humorous. Ideally, in my general philosophy, this should be accepted. Real-time art is for exploration and the creator should not prescribe too much or expect anything specific. If the player decides to fool around, it’s their loss. The problem is that this seems to apply to most players. Maybe interactive art is really only suitable for lighthearted entertainment. And even if the player would devote themselves earnestly to the exploration, there still remains the computer that makes mistakes all the time. Accidents happen. The simulation starts freaking out, often causing a horrific effect that we can only protect ourselves against through laughter.
I decided to abandon my initial inspiration of clumsiness. I figured I should remain flexible and respond to what happens during creation. After all, this is still a very new art form, and definitely very new to me. Just because a simulation can be interactive doesn’t mean it needs to be. There’s a lot of unique value to realtime art outside of interactivity. The only thing that matters is giving the player an experience of beauty. There is nothing to prove. No statement to make. No debt owed to technology. No pride or arrogance or purity.
The player plays the role of Virgin Mary. The corpse of her adult son Jesus is on her lap. She can move her arms to lift up the body by the shoulders with the right hand and by the knees with the left. When I simplified the interaction radically to lifting the right arm very slowly up and down over a distance of about 40 centimeters I felt exhilarated. It feels pleasantly naughty to design such minimal interaction in a medium of which interactivity is often considered its pivotal component. But the thing that made it really work for me was slowing down the response. The body is not lifted up with your moving hand, instead it follows the hand very slowly. This encourages one to slow down one’s own movement. And this slowness of motion gives a feeling of weight, solving that other problem. I’m calling this ambient controls.
Part of the initial design of this project is that the environment changes along with your behavior. So the world would become bright when you lift up the head towards your own and dark when you allow the body to fall back into your lap passively.
But with the radically simple interaction came an in hindsight obvious realization. When you cast the viewer in the role of the subject of a scene, the environment around them becomes more important than the main character. The latter in fact becomes as invisible as the viewer’s own body is to themselves. Suddenly my attention shifted to the environment that surrounds this scene.
Now I want this simple gesture of moving your arms 40 centimeters up or down to take you through the 12000 years of human civilization. From the electric darkness of the Anthropocene to the sunlit harmony of the Garden of Eden. I was inspired by how Timothy Morton in Dark Ecology points out that Genesis can be read as a justification for (or lamentation of) agricultural civilization. And the side effects of the Corona virus crisis gave us all a glimpse of what the planet could be like without the impact of industry. It was like traveling through centuries in just a few weeks.
As I was dealing with such dramatic content, I felt drawn once again towards the Flemish Primitives, the artists whose works initially inspired the Cathedral-in-the-Clouds project. Especially Hieronymus Bosch’s depictions of the whole of human history from creation to apocalypse. So I made this desktop wallpaper based on the top of the left panel of the Garden of Earthly Delights, which represents the Garden of Eden. Since the panel is vertical and my monitor horizontal I decided to mirror the tiles to make a continuous picture.
Two things came out of this.
First I noticed Bosch’s use of perspective. As in many Northern Renaissance paintings, the landscape seems to be depicted from above while the objects and characters are seen from the side or the front. It only feels weird when you start noticing it. The picture still immerses despite of this lack of realism.
I had been bothered a bit by how the naturalism of a horizon on eye height in VR caused half of the scene to consist of sky. And there’s nothing happening in the sky. Everything happens on the ground. Bosch’s solution is genius: put the horizon at the top of the image so you can show all the things happening on the land. I can achieve a similar effect if my ground in VR is in fact a hollow hemisphere in which the viewer is positioned quite low. In combination with some scaling of the elements drawn on this hollow ground a rather pleasant fake sort of perspective appears.
I never thought of the environment in this piece as being a naturalistic. Instead I wanted something ornamental, as one can often see in early paintings and tapestries.
My desktop wallpaper suggested a method of creating decorative patterns out of figurative elements: through repeating and mirroring. As a bonus I would only need to produce one segment of a world and then simply repeat it.
Of course, in the sphere of computer creation there is no such thing as simple. All this technology barely works if you try to do anything other than what its unimaginative creators want you to make. Which in this case would be static environments. But my world needed to be highly dynamic: I want to browse through 12000 years in the lift of an arm. From my previous experiments with dynamic objects I had learned that in 2020 computers still can’t handle a few thousand of them simultaneously on screen. Except, I realized, for particle systems! Originally presumably invented for simulating explosions, smoke and fire, particle systems could do all sorts of things these days.
As it turned out a new particle system technology, called Niagara, was being added to Unreal Engine. Since I did not want my work to disappear when they phase out the current technology, I started learning how to use it. It’s quite a powerful system with a reasonably adequate interface. But I did run into some problems, even a bug, that hopefully will be resolved soon.
To aid in my experiments I needed some assets. I figured cubes and spheres would not help me make aesthetic decisions. So I imported some of the models and textures created by Mary Lazar based on concept drawings by Vicki Wong for our sadly cancelled project An Empty World. An Empty World is also structured along a transition from natural to cultural and industrial landscapes.
To end this little report, please enjoy some screenshots of the current state of the project taken in the Unreal editor, because the aforementioned bug causes none of this to show up in an executable build.
Providing the technical issues are resolved, I consider the design of Compassie done. Can’t wait to start production and bring it all together!
Most of the past month was spent in Coronavirus quarantaine. The eternal city was silent as its citizens were forbidden to leave their neighborhoods. Not being able to visit churches and museums as I’m used to was tough at times. But I did not let this stop me from beginning the work on our new VR project. After all, isn’t this what we do: create virtual projects from the comfort of our homes?
Compassie is the working title for a new Virtual Reality diorama in the context of Cathedral-in-the-Clouds. The theme of the diorama is the pietà, that is the Holy Virgin Maria holding her deceased son in her lap. It’s one of the great classic themes of Western art history, and as such often parodied in contemporary art.
And even though as a 21st century person I cannot avoid being a contemporary artist, parody could not be further from my artistic goals. To be honest I find it hard to understand how one can be so cynical as to make fun of a scene in which a mother mourns her murdered child, not to mention the divine nature of this child.
I was inspired by our own piece Cricoterie in which you move life-size ball-jointed mannequins around on a theater stage. Playing Cricoterie, I started to imagine what it would be like to hold a body that represents the dead Christ in Virtual Reality. I imagined it could be a very powerful experience. And I adore the idea of working with traditional artistic and mystical themes in computer technology. In this technology I find encouragement to return to the sincerity and the beauty of the art from before the modern era.
Since Virtual Reality hardware and the fast computers it requires are not widespread commodities, from the start I thought of Compassie as an installation. I started prototyping by sketching the setup in 3D.
The player would sit on a large throne-like structure while wearing the VR headset and holding the two controllers. In the real world this structure would look very bare bones. But in VR very detailed and ornate. This throne in fact references many depictions of the parallel traditional scene of the holy mother with her infant son on her lap.
When experiencing this scene in VR I was immediately confronted with an unwelcome problem: it’s very difficult to manipulate bodies in a physics simulation in VR without things getting out of hand. In this case quite literally: it was very easy to have Jesus slip from your lap and fall on the floor in a must undignified manner. This is where the static arts of painting and sculpture have an advantage.
There is a traditional scene in Western art that is very closely related to the pietà: the deposition, or the scene in which Jesus’ corpse is taken off the cross before it is given to Mary.
A form of deposition had always been how I imagined the start of the VR experience: a host puts the body into the hands of the player.
One thing that fascinates about many depictions of the Deposition of Christ is the number of hands that support the body of Christ. It’s always a group activity that involves several people. This allows for the maneuvering of the body to happen in a serene, dignified fashion. This got me thinking: what if the player in VR has more than two hands? What if there’s a hand for each limb of the body but they are all controlled by the two real hands simultaneously. And what if a few of these hands are cherubs?
I did some research into such controls and was relatively pleased with the result. But even though a severely limited the freedom of movement, it was still far too easy to put the body in awkward positions.
I radically simplified the controls by turning the players hands into large cylinders that could not rotate.
I liked this restriction, it felt good in VR, even if the real hands and virtual hands did not correspond completely, since rotation of the hands was ignored. But it suddenly struck me that this dead body feels very light in VR. To solve both problems I imagine I could mount each controller in a relatively heavy sphere that the player needs to balance on the palm of each hand. Such “input devices” would allow the player to feel the weight of the body, ensure a certain slowness and dignity in motion (lest they drop the spheres in reality) while creating better correspondence between real and VR motion.
The simplicity of this prototype also confronted me with a problem that affects many interactive art pieces. When an art work allows you to impact it by doing something, people definitely will do that thing, all the time. Even when the work is more beautiful when you don’t touch it or only once in a while or only in a certain way. No, people will jump up and down, dance like clowns, wave their hands, and so on, in order to see the reaction of the machine. So much so that they forget to even contemplate the art, they’re just playing, moving their body. Not exactly the goal I have in mind.
The original concept included a sort of reward system: if you hold the body gently and still, beautiful things would start happening. Lately I have been a bit annoyed by these types of structures of cause and effect, so I neglected this idea. But now it seemed like such a mechanic could prevention the problem with interactivity described above: Jesus would only appear when you hold your hands still in the right position.
Initial experiments with fading the body in and out were disappointing. It made Jesus seem like a ghost and the mechanic of hold-still-for-display felt a bit too prescriptive, too simple.
Instead of making the body appear from thin air, I want to compose it from elements that are already floating around in the environment. Most of these elements would be undefined fragments but some could be cherubs, or flowers, or ornaments with certain animations. This idea matches up with my original vision of a garden that surrounds the player and starts blooming when they are properly contemplating.
So that’s where I am at now. The first experiements have been a bit disappointing in terms of performance. A few thousand individual pieces with their own behavior still bring a very powerful workstation to its knees. Story of my life: in the twenty years that I have been using computers for creating art, they have never been fast enough to run my imaginations.
But luckily for my mood I have developed this theory that art happens where artists fail to achieve what they really wanted. When they need artifice to compensate for shortcomings in technology or technique, magic appears. So I hope I find some tricks to replace the original idea that ultimately make the piece better. Wish me luck.