Augmented Reality Theater Takes a Bow. In Your Kitchen.

LONDON — Standing in front of a golden bandstand, dressed in a white satin gown and pearls, the vocalist Nubiya Brandon sashayed to a gentle beat. Stepping toward the spotlight, she took a lazy turn around the stage, singing a playful calypso number and smiling occasionally at the band behind.

The weird thing about this show, called “All Kinds of Limbo,” was that Brandon appeared to be in this reporter’s kitchen. The singer was in fact an eerily realistic holographic avatar on a mobile phone screen; her performance had been recorded and was now being broadcast in augmented reality from the National Theater in London.

Via the technology’s strange alchemy, which overlays digital imagery onto whatever a camera phone is pointing at, Brandon seemed to be singing and sashaying on the countertop. After she took a bow, her image evaporated and the bandstand faded into nothingness, leaving only a sink full of washing up behind.

The success of digital-only theater productions has been one of the pandemic’s surprise silver linings: Audiences have been willing to try them and theater companies have found fans thousands of miles away. But could immersive technologies provide a more intriguing path forward for drama, one that will endure once Covid-19 (hopefully) subsides? Augmented reality (A.R.) and virtual reality (V.R.) are already changing gaming, music and art; might theater be next?

“All Kinds of Limbo’s” director, Toby Coffey, said he hopes so. In 2016, he set up the National Theater’s Immersive Storytelling Studio, which operates as a kind of “start-up” within the company, he said in a recent interview at the studio’s modest space, which was crowded with a jumble of technical equipment. The team’s brief is to see how live theater and new technologies can interact and intersect.

“Theater makers are naturally fascinated: They’re used to working in 3-D,” Coffey said. “As soon as you bring a director or stage designer or choreographer into V.R., you see their brains whirring.”

The studio’s first production, “Fabulous Wonder.land,” was a V.R. music video featuring a track by the musician Damon Albarn with words by the playwright Moira Buffini. The team has since made 360-degree films of live shows, developed a one-on-one piece in which an audience member interacts with a live actor while wearing a V.R. headset and created a mixed-reality “exhibition” about government welfare cuts.

“All Kinds of Limbo” came into being in 2019 after the National Theater had a hit with “Small Island,” a play about postwar Jamaican immigration to Britain. Coffey and his team commissioned Brandon, the vocalist, and the composer Raffy Bushman to create a 10-minute song sequence responding freely to the play’s themes. It was written, performed and motion-captured that year, and was initially presented as a V.R. experience in one of the theater’s event spaces.

When the pandemic shut down British performing venues in March 2020, Coffey accelerated plans to turn “All Kinds of Limbo” into an at-home experience. The retooled version can be watched via A.R. on a mobile device, via a V.R. headset, or on a regular computer. Brandon’s performance stays the same, but, depending on the device used, the experience feels subtly different.

To summon some of theater’s shared intimacy, it’s being ticketed and broadcast as live, although the show is recorded. Other people attending virtually are represented by blades of moving white light and, by playing with the settings, you can move around the space and see the action from different angles.

It’s a short piece, but “All Kinds of Limbo” does feel like the glimmering of a new art form: somewhere between music video, video game and live cabaret show.

Over the last few years, Britain’s theater scene has become a test bed for similar experiments. Last spring, the Royal Shakespeare Company co-produced an immersive digital piece called “Dream” that featured actors performing using motion-capture technology and was watchable via smartphone or computer. Other projects, such as shows by the Almeida theater in London and the company Dreamthinkspeak in Brighton, England, require participants to turn up in person and get equipped with VR headsets.

Francesca Panetta, a V.R. producer and artist who was recently appointed as the alternate realities curator at the Sheffield DocFest film festival, said in a video interview that practitioners from audio, gaming, theater, TV and other art forms were collaborating as never before. “Many different people are trying to explore this space and work out what it really is,” she said. “No one is quite sure.”

One of the most keenly awaited partnerships is between the immersive theater troupe Punchdrunk, which pioneered live site-specific shows such as “Sleep No More” and “The Masque of the Red Death” in the mid-2000s, and the tech firm Niantic, best-known for the wildly successful A.R. game Pokémon Go.

Speaking by phone, Punchdrunk’s co-founder Felix Barrett seemed invigorated by the creative possibilities. “We’re on the cusp of a new form of entertainment,” he said. “It’s a new genre; it just hasn’t been named yet.”

Later this year, Niantic and Punchdrunk plan to unveil the first results of their collaboration. Barrett was reluctant to reveal too much, but said that it would offer participants “a citywide adventure” that will feel like an immersive video game happening in the real world. “Our goal is to try and make you the hero of your own living movie,” he said.

Ambitious as such projects are, they are also — at least by theater standards — time-consuming and forbiddingly expensive. The Royal Shakespeare Company’s “Dream” wouldn’t have been possible without corporate sponsorship and a hefty grant from a roughly $55-million British government fund promoting digital arts innovation. The latest iteration of “All Kinds of Limbo” relies on a partnership with Microsoft and the livestreaming platform Dice.

There’s also the question of audience. Though theater-led projects such as “Dream” and “All Kinds of Limbo” have gained positive reviews, they have attracted only a tiny fraction of the 12 million viewers who watched a 2020 virtual performance by the rapper Travis Scott in the online game Fortnite. The chances of monetization at scale look slim, at least for now.

And the irony is that, while the pandemic may have whetted audience appetites for digital drama, it has had devastating consequences for theater companies themselves. The National Theater’s Immersive Storytelling Studio originally had four staff members; after belt-tightening layoffs in the company, it’s now just Coffey and one full-time co-worker. “Even before the pandemic, we could have been doing 10 times more than we had resource to be able to do,” Coffey said. “We need to work within those restrictions.”

What happens next is up for debate. The National Theater is working on redeploying the app and distribution platform used for “All Kinds of Limbo” into something that works for other projects. Panetta said that the metaverse, if it genuinely takes off, offers its own possibilities for live performance. “It’s difficult to see what the pathway is; I suspect it’ll be a mix of many different things,” she added.

So how long until we’re watching Ibsen or Shakespeare in augmented reality at our kitchen tables? Coffey laughed, then cautioned that designing successful A.R. performances was still an emerging skill. “But some day it’ll happen, I have no doubt,” he said.

All Kinds of Limbo
Streaming through Jan. 30; allkindsoflimbo.com.