Virtual Reality Theatre
Demystifying VR: A Practical guide for Artists
A dissertation by Tomis Fras
I shall be discussing:
​
-
Why study VR?
​
-
A practical overview to working in VR
​
-
How does VR function?​​
​
-
My thoughts and notes from my own practice.
​
-
What artists can do now to prepare for a rapidly changing digital landscape.
​
-
The future and current state of the VR industry.
This is a political and creative investigation into the current workflow of VR production from the perspective of an independent technical performance artist with no coding skills.
What are some of the lessons from VR production that we may add and adapt to the fundamental existing artistic methodologies?
In my word and image document, I shall translate my studies from the field of Virtual Reality into a guide though the new techniques required for working ethically and creatively in a 4 dimensional medium.
My aim with this dissertation is to demystify VR and to show what it actually is. Creating art for Virtual Reality is more accessible now than ever before and what the medium truly needs is independent artists to lead the avant garde.
The future of VR is daunting, even for digital artists.
"Talking about virtual reality is like dancing about architecture."
'How virtual reality can create the ultimate empathy machine' Ted Talk by Chris Milk
History has taught us through the abuse of the animation industry that a creative medium should not be owned by a handful of wealthy conglomerates.
This restricts the creative development of an entire medium as well as creating unhealthy working conditions.
Already, Facebook's hold on the industry is alarming.
With Facebook’s acquisition of Oculus, the social media company seeks to monopolise VR as a whole. They have made a controversial move to create a ‘meta-verse’ centered around social media, crypto finance and gaming whilst disregarding the massive controversy over user data, privacy and security.
“Facebook is going to probably have a near-monopoly in VR software before it even matters,”
A tweet by The Verge journalist Alex Heath on the subject of Meta.
It could seem that VR is contributing towards a bleak dystopian future.
However, it is still very early days for the medium, and whilst companies like Facebook profit from the public misunderstanding of the medium, they don’t actually own the medium… yet.
VR can be anything. Hence, artists have a golden opportunity to democratically redefine the medium using existing free software.
Therefore, for my word and image document, I will share what I have learnt through working independently in VR in hope that others may find that their skills can be transferred into the digital medium, just as mine were.
This is a study into VR practice in an ever-growing digital landscape.
My Proposal
I shall document my process and observations throughout my practice in VR Production and give my thoughts on where this medium may head.
Why study VR?
“Using a VR headset and especially playing room scale experiences is magical. It messes with your mind in ways you can’t really imagine until you tried it yourself.”
VR Artist Tobias van Schneider states on his personal blog
Perhaps the best argument for working in Virtual Reality is simply trying it. With my background in set-design and theatre, VR felt like the next logical step. However, I had no skills in coding or 3D animation at the time. At the time I considered this to be the barrier to working in VR. Despite that, with free programs like Unreal Engine and Blender, I found myself experimenting with 3D animation in no time - without the need of coding skills.
Virtual Reality is far from accessible. In fact from my own practice, I have found that the medium can be flat-out hostlile to newcomers. From the limitations of movement, to the convoluted set-up required and the clunky experience of wearing a headset for the first time, it can be a hard sell.
​
However, a phenomenon I've come across at the time of writing has been a general apathy among artists to the medium. An "I don't get it and I don't want to", mentality has held firm.
​
And with good reason.
​
Virtual Reality is expensive to use, difficult to learn and once everything works, you must then wear a headset and separate yourself from your surroundings.
​
Even in its element, VR can be an inherently lonely experience. However, this doesn't have to be viewed as a negative. Rather, I believe that this is what makes the medium rise above others.
Think for a moment of being in a theatre. The lights go down, the audience takes their place and phones turn off. And in that moment, the audience places their trust onto the performers.
​
This social norm creates a space where everyone in an entire room can collectively choose to put reality aside for a moment and suspend their disbelief.
This the true magic of theatre, not the technical wizardry or flashy posters: but rather the relationship of trust and gratitude between viewer and performer.
​
When viewing VR, it is important to remember that it is a new medium. It's expensive, heavy, lonely and frankly silly at times, but then again, so is theatre. Sets are expensive, people sit in silence in the dark and at times it seems ridiculous. And yet people still do it.
​
This type of gratitude and respect for the audience is something that it is vital for all artists, especially those working in VR.
To illustrate this point, I created the 'worst' VR scene possible. It is a Photogram of my bedroom turned 90 degrees on its side with the camera flying around the scene very quickly (this creates the illusion of flight for the viewer).
​
Above is the footage of the experience.
​
Whilst VR literacy is still in development, there are still a few fundamental rules for viewer comfort. These include correct eye height, relative scale and attempts to avoid motion sickness.
​
A designer must work carefully not to make the viewer feel sick whilst they are in a vulnerable position. However, I wanted to prove how important this was by creating an environment which specifically broke all the rules of viewer comfort.
​
I, very apprehensively, asked my roomates to try this out and was incredibly surprised by the outcome. Rather than hating the experience, they wanted to try it again and again. After a while, it became a competition to see who could endure the 'nightmare' room the longest. This was all whilst they were nudging shoulders and giggling amongst themselves.
The outcome of my experiment ended up having the exact opposite reaction to what I had predicted. I would argue that this is the missing ingredient to VR: honesty.
​
The audience trusted the experience more when they knew it would be uncomfortable and thus they prepared for it. They would ironically go on to describe the experience as meditative, cathartic and most importantly: unlike anything they'd ever tried before.
​
VR is a medium that, unfortunately, you must experience alone. But this is also its greatest strength.
​
When working in VR, one must always be aware that this is unnatural and virtual - but rather than hiding from that: you should embrace the uncomfortable, the strange and the silly.
Now is the time to make mistakes in VR. It is through peer studies that we may develop the artistic language of the medium.
What is VR Theatre?
For the purposes of my Word and Image document; I will be reffering to VR in relation to my own work - A Room-Scale 3D Animated Experience Rendered in Real Time.
​
Or for simplicity's sake: Virtual Reality Theatre (VRT - meaning 'garden' in Croatian).
​
In order to understand the key differences between Virtual Reality and Real-Reality we must first address how the digital medium functions.
Relative Composition
The art of balance on a 4D Canvas
In film, you have a director who pushes the medium forward. In plays, you have a playwright; in video games, a game designer. Each is essentially the same - the creative lead - a role people can spend a lifetime developing. However, they are separated by the requirements of their specific mediums.
​
So in VR, where film, theatre and video games are at your disposal, how do you make a cohesive narrative?
​
Since VR Theatre (Room-Scaled Animated Experience Rendered in Real-Time), requires multiple very different skills and practices, it becomes a difficult subject to explain - after all each discipline required has a vastly different vocabulary and terminology.
​
However, I have found through my practice that whilst each different discipline is vastly different, they all work together to create a cohesive whole.
Rather than breaking down the roles individually, you must consider how each element works in Relative Composition with everything else.
Relative Composition is a term I've coined to describe what all mediums in VR have in common from the perspective of a VR theatre maker.
​
VR is simply a multi-dimensional canvas. Whatever you choose to bring to the canvas; it is the artist's choice.
​
However, whilst every thing is physically possible in VR; some things are significantly more difficult to recreate than others.
​
Relative Composition describes the balance between completely different mediums and how they enhance one another through time.
"VR has elements of painting, literature, theatre, motion pictures and Video games. Understanding how some of the key elements that makes these specific mediums shine, we can pick them apart to understand how to use those in VR productions."
Virtual Reality Cinema by Eric R. Williams, Matt Love, Carrie Love
My initial research brought me to this book. This is the most comprehensive guide to approaching the new subject from an artistic philosophical stance.
​
Using this guide along with my work in the field of VR and theatre, I hope to show that most of the new cutting edge theories emmerging from VR aren't new at all.
​
In fact, most of the issues with VR have already been solved: by theatre.
​
For example: how do you block a VR Story without an Upstage or Down Stage
For
Overview to working in VR
Making a story for VR can seem daunting at first; after all, the possibilities of what you can do are endless.
​
The most important part, and the first hurdle to get over, is not to get trapped by choice paralysis.
​
From my experience in Central St Martin's Performance Design and Practice course, I was taught to break down the intentions and ideas of a piece, before finding a specific medium/s to translate the idea through.
​
In the centre of the page is my breakdown of the entire process from begining to end of making a VR story.
​
VR is Virtual-Reality; therefore any medium that exists in the real world may also exist in VR.
​
Much like making a movie, play or video game, there is a working pipeline of stages you must follow.
​
Since VR is a 4D medium, this means that you must also break down the creative roles as a whole by their dimensions, as shown in the table below.
Concept:
-
Each medium should relate to the main concept or theme.
-
From here, we start thinking about set design, concept design and music to inform the ideas on writing.
-
This stage requires jumping between all four mediums where musical ideas can enhance concept designs and set design can add dimension to the writing.
-
This part is the most important: with a good story, music and art - the hardest job is done.
​
Development:
-
Once we are confident with the concept art, we begin to mix and match the mediums to add dimension.
-
Set Design and Concept Design are required before starting to 3D sculpt.
-
Music and Concept design is required before 2D animation.
​
Production:
-
With all the elements in place, we may begin directing 3D objects, writing, 2D animation and sound through time, the fourth dimension.
-
The final stage is Relative Composition, editting on a 4D canvas using the tools of all the other mediums.
​
2D
3D
4D
A 2D Medium -X axis -Y axis
A 2D Medium X - axis Z - axis
A 2D Medium - Time - Audio
A 3D Medium - Time - X axis - Y axis
A 3D Medium - Time - Words - Blocking Instructions
A 3D Medium -X axis -Y axis -Z axis
A 4D Medium - Time - X axis - Y axis - Z axis
A 4D Medium - Time - X axis - Y axis - Z axis
A 4D Medium - Time - X axis - Y axis - Z axis
My methodology for VR
​
​When working in VR, I have found it is always best start with your strengths, this could be musical, visuals, scenography or simply a really a good story. During the conceptual stage, I would suggest jumping between all four - translating and complementing the other mediums through relative composition.
​
Therefore, whatever you find most interesting: start with that.
​
Everything else can be as simple or complex as you wish, so long as there is a clear balanced narrative throughout.
During my second year at Central St Martins, I worked alongside my colleagues, Sophie Goodall and Olivia Kelly, to create the short film 'Mary and Bowley'.
​
This 10-minute Puppet/Live-action film, for which I wrote the script and music, required an entirely new approach of writing.
​
Since we were working around the sensitive subject of relationships and where they fail, we had to find a new way of developing characters and their psychological motives.
​
For this, we studied Attachment Styles and other works by behavioural psychologists: John Bowlby and Mary Ainsworth (we named the protaganists after them).
This academic approach to character pyschology is one that I use to this day.
​
Since 'Mary and Bolwey' included so many stylised elements, we were able to tell their personal story through their puppet design, set desgin, musical motifs and cinematography.
​
With their story and psychology at the heart of our foundations; all other elements would be made to enhance the ideas and themes through relative composition.
Much like a film or play, VR also requires a lot of forward thinking. Each idea starts as a drawing or note and through the creative process, we add dimensions to those drawings and ideas.
Concept Design:
​
I like to play with shape language, silhouettes, and appeal to have a rough idea of how my characters look and how their physicality will affect their performance.
​
​
Note: This is not the final design but rather a way to have a rough visualisation of the characters before you start – to help visualise some of the ideas from the attachment styles and initial concept.
Moving to 3D
​
Using the concept design, we may begin sculpting a high detailed character. Since the main concept of the piece have now been developed through the writing, music and concept design, the rest of the work flow is simply a case of following your own instructions onto completion. From here, the rest of the choices come from finding creative solutions to the technical limitations.
​
To understand the limitations, first we must understand how VR Theatre functions.
How Does VR Theatre function?
Unlike VR cinema (360 video), VR Theatre is room-scale.
​
This means that the viewer is virtually in the space with the characters, rather than being shown a flat video in 360. Walking around the play-space; you see the characters from different angles - much like immersive theatre.
​
In order to do this however, all characters must be rendered in real-time. This means that all actors/props and sets are made in Digital 3D.
So, to get over the biggest hurdle of Virtual-Reality Stories, it's virtual.
​
Nothing is physical in the most literal sence. Object information is stored as detail in the form of vertices, edges and faces along the x, y and z axis (as well as time, as I will describe later on).
​
Hence, all objects in Virtual Reality Stories are simply outlines with no volume.
Vertices:
​
A vertice is a 4D position along the xyz axis and time (frames).
​
For example:
x= 2, y= 1.5, z= -0.5, frame= 0
x= 1.8, y= 1.5, z= -0.6, frame= 0
x= 2, y= 1.6, z= -0.5, frame= 0
​
Every dot's position is stored as number notations. The notation is then stored as computer memory (RAM). If this looks like maths, it's because it is. Luckily computers are very good at doing the boring calculations.
From the vector positions (3D postion along the x,y,z axis stored as numbers) of the vertices, the compter calculates a straight line between the two dots: this is called an edge.
​
Once four vertices are connected by four edges, we create a Face: a four-sided shape with a calculated normal (the direction it's facing stored as angles ).
Animation:
3D animation is 3D sculpting with time as an additional dimension.
​
For example:
x= 2, y= 1.5, z= -0.5, frame= 0
x= 2, y= 1.5, z= 4, frame= 24
​
Here, the vertice would move from z-position -0.5 to 4 over the course of 24 Frames (one second).
​
The notation for change in location is called a key frame.
Key frame interpolation:
​
Key frames are represented as yellow diamonds on the timeline.
​
How you wish the computer to animate the movement from one keyframe to another is called key frame interpolation.
​
Sin wave: This means that the object accelerates at the begining and slows down at the end.
​
Linear: This means that the speed is the same throughout (no acceleration or deceleration)
​
Saw tooth: This means that the object teleports to the position.
​
The diagram shows how this timegraph is portrayed in the timeline. By changing the shape of the timegraph (Sin, linear or sawtooth) we can manipulate the object to move in specific and unique ways.
​
​
All this information can be used to tell the computer how to use this data in the form of shadows, surface textures, bevelled edges/sub-divisions.
Along with digital light sources - creatively placed throughout the digital scene - the computer can calculate precise contact shadows and advanced properties such as ambient occlusion, subsurface shading and motion blur.
VR isn't magic, it's simply computer maths applied to creative expression.
Each vertice is recorded as a written position and this takes up RAM; but what does this mean for artists?
DETAIL (RAM) IS SIZE
Here are two blue blocks; In the real world, one would be larger and heavier than the other. This is not the case with VR.
In VR, each object has six vertices, 12 edges and six faces. So while one is larger than the other in terms of size, they both take up the same amount of RAM (Computing memory).
Once we understand that in the digital world, size doesn't exist, we can begin playing around with scale in relativity to everything else in the scene. Therefore, you can do things in VR that you cannot do in any other medium. Props can be as large and weightless as you want. This is one of the key principles of Relative Composition.
Here are two blue blocks; In the real world, one would be larger and heavier than the other. This is not the case with VR.
We need to rethink the way we view digital art production.​
I have learnt, through my practice, that each piece of work starts with a figurative, or sometime literal, blank canvas. What the artist brings to the canvas is the service that we provide. The same applies, if not more so, to digital art work.
​
After all, in Vitual Reailty, nothing exists until you consciously make it.
i.e: gravity, environment, sunlight...
Since computer memory (RAM) is the limiting factor when working with digital art, it requires the artist to be very intentional with their creative choices; in particular where to place detail.
The human face continues to be the hardest challenge to recreate both technically and creatively. Humans are very good at recognising faces; we see faces everyday. Therefore a face has to be perfectly realistic... ...or it has to go down a stylised route. When creating characters for VR, it is always best to start with clear concept drawings. The drawings should act as the key reference for the rest of the 3D process.
As I stated earlier, each 3D model in VR is made up of vertices, edges and faces. In order to create the illusion of a digital curve, the computer needs to have a shape with millions of faces. This is incredibly taxing on the computer and requires too much RAM to function. When 3D sculpting, creating a shape with so many faces is fine when it is stationary, but the computer is at very high risk of crashing if you try to animate this.
The model doesn't have to be evenly detailed throughout. Re-topology is where artistic intention and technical requirements must work hand-in-hand in relative composition. A flat surface such as a shoulder plate will require less detail than a curved cheekbone. I would reccomend that the most complex thing in any scene should be a human face - this is the emotional core of the story. But this doesn't mean that the rest of the scene should be ignored.
The human face continues to be the hardest challenge to recreate both technically and creatively. Humans are very good at recognising faces; we see faces everyday. Therefore a face has to be perfectly realistic... ...or it has to go down a stylised route. When creating characters for VR, it is always best to start with clear concept drawings. The drawings should act as the key reference for the rest of the 3D process.
Time as the Dimension that connects them all
Any program with a timeline function is, by its own nature, depicting time as a separate dimension. The fact is that, be it film editing, animating or composing, each has its own form of displaying and manipulating media through time.
​
This becomes something very important to have in mind when working with multiple different mediums across multiple different dimensions.
​
For example 3D animation includes working across 4 different physical axes:​
​
-
X axis - horizontal
-
Y axis - vertical
-
Z axis - depth
-
Time - Frames per second
​
Why is this important? As I stated earlier, detail is size - and each dimension being recorded requires additional RAM. Therefore some objects don't require time. Let me explain: A prop can be stationary and therefore we do not need to record it through time.
Or rather, through Relative Composition, we can place a stationary image onto a 3D moving object. That way we can save on RAM by placing one object within another.
​
So considering that time is an dimension that multiple medias use; we can break down each indidual practice by what they have in common with one another - much in the same way as Stage Management.
Theatre blocking in VR
Proximity and gravity In VR, instead of having Upstage and Downstage, characters and blocking are dictated by proximity to the viewer. INFO: Inner Orbit Near Orbit Far Orbit Outer Orbit In order to represent the sense of depth, I have colour coded the set by proximity to the viewer.
To add to the work of Virtual Reality Cinema, I have also incorporated my own philosophies from Art and Theatre. Rule of three: Use the rule of three to have orientation points in the scene. They should vary in location, size, height and proximity to the viewer and their surroundings.
For example: The Floor light (an orientation point) is present in both frames. This way, the audience may look at either a character on the sofa or one on an arm chair. Both however should be aesthetically pleasing. This is the most practical use of Relative Composition - arranging instances by proximity and gravity relative to other instances.
Proximity and gravity In VR, instead of having Upstage and Downstage, characters and blocking are dictated by proximity to the viewer. INFO: Inner Orbit Near Orbit Far Orbit Outer Orbit In order to represent the sense of depth, I have colour coded the set by proximity to the viewer.
Current state of VR
VR currently stands at a crossroads in time.
Within the industry, a massive digital land grab is underway, initiated by major companies such as Google and Microsoft, with Facebook being the current clear winner.
However, Facebook doesn’t own VR. In fact, despite renaming their company ‘Meta’, they don’t even own the ‘Metaverse’ – a name first coined by science fiction writer Neal Stephenson in his 1992 book ‘Snow Crash’.
As of now, the ‘Metaverse’ is still just science fiction. However, this hasn’t deterred the company from trying to own all aspects of VR.
Considering that Facebook’s ‘Meta Quest 2’ is currently the cheapest standalone headset selling at £300-£400 (their closest competition being the HP Reverb selling at £500-£600, a price that doesn’t include the cost of a PC needed to run PC VR, which comes in at an additional £1,000), the company has little competition in terms of affordable headsets, and they intend to carry on in this vein.
The financial advice company ‘The Motley Fool’ suggests that Facebook's plans can be summarised as follows:
​
· Sell the headset
· Own the store developers sell content on
· Own the biggest content studios and sell their content (potentially exclusively) on Oculus
· Build a Facebook social/meeting platform
· Require users to log into Facebook accounts
· Serve ads to VR users
This becomes a larger issue considering Facebook’s previous history as regards user privacy and security.
​
​Verity McIntosh, a VR expert at the University of the West of England, stated the following in a BBC interview:
"Now it's not just about where I click and what I choose to share, it's about where I choose to go, how I stand, what I look at for longest, the subtle ways that I physically move my body and react to certain stimuli. It's a direct route to my subconscious and that is gold to a data capitalist.”
Verity McIntosh, VR expert at University of the West England
​“THE PEOPLE THAT ARE GOOD EITHER ALREADY WORK AT FACEBOOK OR THEY’RE BUYING THEM”
Alex Heath, VR journalist at The Verge
​In addition, Facebook has been buying out smaller studios currently working with VR and setting the condition that viewers must have a Facebook account to view their work.
“I deactivated my Facebook account earlier this year, and don’t plan on devving for Quest / Oculus ever again,”
Blarp! creator Isaac Cohen
However, I don't subscribe to this defeatist mentality. Some people will want to work with Facebook, and they may do good work there, but it’s not the only player in town. There are already many different ways to work ethically in VR.
From a production standpoint, both Unreal Engine and Unity offer cross compatibility with OpenXR, a format which is accessible for all VR devices, not just Facebook’s. In addition, some artists are beginning to explore VR as a medium.
Joshua Skirtich, left, has created art in VR and sold his work as NFTs outside of Facebook’s marketplace.
Additionally, there is good advice from Timoni West, the principal designer at Unity Labs, on how to find sustainable work in VR.
“If you want to find VR gigs, I’d do a few things. I’d start tweeting about looking for gigs, put my information up on Unity Connect, join some AR/VR slack channels, go to meetups and do my research on what studios are getting the gigs — The Mill, MPC, Framestore, Tool, FakeLove, etc. Reach out to your favourite indie VR devs and ask them for advice. Don't be afraid to cold call; the community is still small and people are really nice.”
Timoni West, the principal designer at Unity Labs
Museum of Other Realities
​For those curious about the possibilities, I can recommend the Museum of Other Realities, which is a digitally-curated museum showcasing the works of Avant Garde VR artists such as: