Augmented Reality (Computers)

Subterms

    Latest story

    113 Shares99 Views

    Wagner’s ‘Parsifal’ at the Bayreuth Festival Experiments With AR

    by

    Cutting-edge technology has again come to the Bayreuth Festival, where Wagner premiered his final opera with the latest stagecraft in 1882.For Richard Wagner, the latest technology was crucial to staging his operas.In Bayreuth, Germany, where he opened a hilltop theater in 1876 to realize his vision for his works, he promised that “the most up-to-date artistic resources will be used to offer you scenic and theatrical perfection.”That year, the Rhinemaidens at the start of his “Ring” were supported behind the scenes by wheeled machines that made them seem to swim. A projector with prisms tried to create the effect of gods walking across a rainbow. The auditorium was dimmed — unusual at the time — to focus the audience’s attention and enhance the illusions.Nearly 150 years later, cutting-edge technology has come again to Bayreuth: augmented reality, which adds a dense, often impenetrable layer of surreal imagery to Jay Scheib’s new production of “Parsifal,” which opened on Tuesday.Among the many AR images visible through special glasses are motion-capture outlines of figures walking, embracing and suddenly ablaze.Joshua HiggasonThis medium could hardly be further from the creaky machinery and gas lighting of the 19th century. But the goal is the same as Wagner’s: to create “scenes such as you might imagine had come from an ideal world of dreams.”But there’s a catch.After a squabble within the notoriously squabbling Bayreuth Festival about funding the expensive augmented reality, or AR, glasses, money was allotted for 330 sets in a theater of 1,925 seats.So 83 percent of the audience just experiences the old-fashioned article: Wagner’s operatic mystery play about a young man who ends up redeeming the ailing rituals of a corps of Holy Grail knights, straightforwardly staged and superbly sung, and conducted with muscular solidity by Pablo Heras-Casado. A much smaller group, including critics, gets the glasses, which superimpose on that live staging a crowded AR environment that is constantly in motion.Are the 83 percent missing much?They miss the space between them and the stage seeming to fill with twinkling stars as the soft prelude begins. The bare trees rotating in the ether. The motion-capture outlines of figures walking, embracing and suddenly ablaze. The asteroids. The fly that seems to land on the outside of the AR lenses.Later, the flocks of birds, blood-red globules and spiky strawberries. The slithering snakes and spinning, silently cackling skulls. The blossoming flowers. The arrows, spears, machetes, axes, grenades and severed arms. The forlornly quivering plastic bags and the bounding fox. The rocky ledge that appears to fill the area beneath the seats in the third act.In AR style, the 3-D images don’t move with you as you move your head. Rather, you seem to be able to pan across an environment that surrounds you: not a realistic landscape but a galaxy of disembodied elements floating in the darkness, a free-association, stream-of-consciousness panoply linked, to varying degrees, to the plot.Some of the images’ textures are photorealistic, but most emphasize their computer-generated unreality, their unnatural angles and fake finishes, their eerie weightlessness. The aesthetic — with its collagelike excess of uncanny juxtapositions and its flat affect — evokes the digital art that has sometimes been winkingly called post-internet.Georg Zeppenfeld on the spare, slightly ominous, vaguely sci-fi set for Act I, designed by Mimi Lien.Enrico NawrathBut for those wearing the glasses, the union of the production’s AR and live aspects isn’t generally happy. The lenses are tinted, so the live performance looks considerably dimmed, and the staging’s frequent video projections are almost invisibly faint.The AR elements (designed, along with the video, by Joshua Higgason) often block the onstage action, even as those elements are fragmented enough to suggest they are offering a complement to that action, rather than a self-sufficient alternative.However dreamlike, the resulting visual confusion doesn’t convey the hypermaximalist, proudly absurdist overload of Bayreuth productions like Christoph Schlingensief’s 2004 “Parsifal” or Frank Castorf’s 2013 “Ring.” This is because Scheib’s sensibility — in both the virtual and live spheres — is basically plain and direct.When I peeked below the glasses to watch bits of the performance without AR, there was nothing particularly imaginative or illuminating about this “Parsifal.” The first act takes place in a spare, slightly ominous, vaguely sci-fi landscape — the sets were designed by Mimi Lien — with a halo of flashing lights that brings to mind the spaceships of “Close Encounters of the Third Kind” or “E.T. the Extra-Terrestrial.”These Grail knights wear stylish, contemporary clothes — long tunics, yellow skirts, boldly patterned hoodies — designed by Meentje Nielsen. The sorcerer Klingsor’s enchanted garden in Act II is a psychedelic pool party in “Barbie” colors. After Parsifal destroys the garden, the third act is set in a lonely desert encampment, alongside a machine on the blurry line between war and industry: maybe an earthmover, maybe a tank.The tenor Andreas Schager is tirelessly passionate and convincingly boyish as the guileless Parsifal, and the bass-baritone Derek Welton is mournful yet reserved as Amfortas, the wounded king of the Grail. The bass Georg Zeppenfeld is an elegiac Gurnemanz, who oversees the knights; the baritone Jordan Shanahan, a brooding Klingsor.Klingsor’s enchanted garden in Act II is depicted as a psychedelic pool party in “Barbie” colors.Enrico NawrathThe mezzo-soprano Elina Garanca sounds luxurious — lean yet velvety — as the ambiguous, ambivalent Kundry, cursed to shuttle forever between the realms of Klingsor and the Grail and a role too often screamed. Bayreuth’s chorus, directed by Eberhard Friedrich, is, as ever, poised and powerful. On Tuesday, the orchestra didn’t quite bring out the exquisite transparency and delicacy of some important passages, but Heras-Casado’s conducting was vibrant, even-keeled and well-paced.There were a few memorable AR moments. At the end of Act I, a boy in jeans seems to walk through the space, slowly flapping wings attached to his arms — perhaps a melancholy nod to the winged children in Stefan Herheim’s celebrated 2008 “Parsifal” here, just as the dam we seem to be at the bottom of at the start of Act II may be a reference to the hydroelectric plant that opened Patrice Chéreau’s centennial “Ring” at Bayreuth in 1976.Yet there is something bland and empty at the production’s core. It’s not clear what Scheib thinks the nature of the sickness is at the root of this Grail cult, so it’s not clear what Parsifal’s climactic redemption offers. If the final AR image of plastic bags, echoed by one onstage, gestures toward a critique of environmental despoliation, it’s a wan gesture.This means the augmented reality has little profound substance to support, just a jittery desire to stimulate — to ornament and impress — which is just what Wagner didn’t want from stage technology. Scheib’s AR decorations rarely inspire emotion or a sustained sense of wonder: the impression, as Gurnemanz says to Parsifal, of time becoming space.The inadvertent result of all the lavish resources is to prove the superiority of the live over the digital — to keep us sneaking back under our glasses from the augmented real to the really real. The closest parallel in the opera to contemporary technical wizardry is Klingsor’s false garden; it feels rather perverse to extend those artificial seductions to the rest of a piece that’s condemning them.We have come a long way from this opera’s premiere at Bayreuth in 1882, when Gurnemanz and Parsifal stepped in place as a painted backdrop scrolled by, turned by hand on rollers, to create the illusion they were walking. “The simplest of means,” one observer wrote, “had brought about an overwhelming effect.”For all its ambitions and expense, Scheib’s “Parsifal” never overwhelms.ParsifalThrough Aug. 27 at the Bayreuth Festival in Bayreuth, Germany; bayreuther-festspiele.de. More

    More stories

    • 113 Shares149 Views

      in Music

      At Wagner’s Festival, New Technology Reveals a Leadership Rift

      by

      The Bayreuth Festival’s production of “Parsifal” will feature augmented reality. Securing the equipment set off a financial and philosophical dispute.The American director Jay Scheib was looking at a bank of monitors inside the Bayreuth Festival Theater on a recent afternoon.He was rehearsing his new production of Wagner’s “Parsifal,” which opens the storied Bayreuth Festival on Tuesday, and as performers circled a large metallic monolith onstage, the screens showed three-dimensional flowers floating through blank space — psychedelic animations that will come to life for audience members who see them with augmented-reality glasses.Through those glasses, Scheib said, the flowers, and other items during the performance, will appear to float through the auditorium. In keeping with the opera’s themes, he added, these moments are meant to provide the audience with “sacred visions” of “a world where wonder still exists.”Scheib’s production is one of the most ambitious, and high-profile, attempts to incorporate augmented reality into opera performance. But it also caps months of tumult at Bayreuth, after plans to outfit nearly 2,000 audience members with the glasses for each performance were downscaled because of an apparent money dispute between the festival’s artistic and financial leadership. The compromise, in which only 330 attendees will be provided with the glasses to experience the production’s signature flourishes, has left many fuming, and concerned that internal conflicts at one of the most important events in opera were undermining its relevance.Founded by Wagner in 1876 as a showcase for his work, the Bayreuth Festival draws opera fans from around the world for one month every summer to hear a handful of the composer’s works in repertory — including a new production at the start of each edition. A major event on the German cultural calendar, the opening is usually attended by prominent political figures including Angela Merkel, the country’s former chancellor.Angela Merkel, the former chancellor of Germany, with her husband, Joachim Sauer, at the opening of last year’s Bayreuth festival, which remains a major event on the German cultural calendar.Christof Stache/Agence France-Presse — Getty ImagesThe festival remains treasured worldwide for the pristine acoustics of its theater, a hilltop opera house that Wagner had a hand in designing, and for its connection to the composer: It has been led by a family member since his death in 1883. His great-granddaughter Katharina Wagner took over creative leadership with her half sister, Eva Wagner-Pasquier, in 2008, before becoming the sole artistic director in 2015.In recent years, though, a new leadership structure has added a layer to the festival’s decision-making. In 2008, the budget came under the control of four members of an independent board representing outside shareholders that collectively provide about 40 percent of the budget: the city of Bayreuth, the state of Bavaria, the German federal government and a group of private donors called the Society of Friends of Bayreuth, who currently chair the board.Although the funders are meant to refrain from interfering with choices made by Bayreuth’s artistic leadership, some in the media have argued that the decision to withhold the money for the purchase of 2,000 glasses represented an attempt by the shareholders to rein in Wagner’s approach to the festival and her great-grandfather’s work.Since World War II, Bayreuth directors — including Richard Wagner’s descendants — have brought a modern or experimental sensibility to the composer’s works. In 2013, Katharina Wagner invited Frank Castorf to reimagine the “Ring” as an anticapitalist epic about oil; the next “Ring,” Valentin Schwarz’s production, which opened last year, recast the cycle as, in part, an allegory about the anxieties of aging.Toni Schmid, a former high-ranking Bavarian civil servant who led the festival’s board of shareholders until 2020, said the decision not to fund the glasses was emblematic of the Society of Friends of Bayreuth’s “more conservative idea of how a Wagner opera should look today,” which is at odds with Katharina Wagner’s vision.A scene from Frank Castorf’s reimagined “Ring,” in 2013. Since World War II, Bayreuth directors have brought a modern and sometimes experimental sensibility to their Wagner stagings.AlamyThe largely older members of the donor group, Schmid said, “would like to have the productions they saw 50 years ago back, when they were young — but that’s not art, it’s a museum.” He added that he wished the shareholder’s board was occupied by representatives “who know what they’re talking about” and described the decision to not finance the full number of glasses as “a joke.”Manuel Brug, a German journalist and critic for Die Welt, said in a phone interview that the current festival structure allowed too much power to Friends of Bayreuth. “The group is too old, with many people who joined because it makes it easier to get tickets,” he said, arguing that the donors should be excluded from the governing body in the future. The Bavarian arts minister Markus Blume said in article in the Nordbayerischer Kurier on Thursday that the state of Bavaria might take over some of the donor group’s shares in the future.Georg von Waldenfels, the chairman of the shareholders board and head of Friends of Bayreuth, disputed that he had interfered in Wagner’s decision-making and said in a phone interview that the decision to downscale the number of glasses was “purely a decision of the artistic leadership.” He added that the shareholders had merely “stuck to the business plan.” Wagner, however, said that the original plan failed “because of the financing and divergent views about the glasses” and that the outcome was “unfortunate.”This disagreement reflects a broader debate about Wagner’s legacy, and adds another chapter to the festival’s history of public arguments and reckonings. Winifred Wagner — the English-born wife of Richard’s son, Siegfried — who oversaw the festival from 1930 to 1944, was an avowed fan of Adolf Hitler until her death in 1980. Following World War II, the composer’s grandsons, Wieland and Wolfgang, opened the festival anew as something more apolitical.More recently, the festival has been a subject of chatter, including longstanding rumors of a feud between Katharina Wagner and her former musical director, Christian Thielemann, who left his post in 2020. Last year, he publicly criticized her decision to replace the word “Führer” (“leader”) with the word “Schützer” (“protector”) in a production of “Lohengrin,” a change that had been made out of sensitivity to Bayreuth’s past associations with Nazism.Katharina Wagner, a great-granddaughter of the composer, took over creative leadership of the festival with her half sister in 2008, before becoming the sole artistic director in 2015.Enrico NawrathJay Scheib, the American director who is staging “Parsifal” for the festival this year.Helen DurasIn a phone interview, Thielemann denied any feud with Wagner, and said that Bayreuth has long been plagued by gossip. “There is something about Wagner that poisons people,” he added. “He is both an intoxicant and a perfume.”Wagner’s contract will be up for renewal this fall, pending a vote by the festival’s board of directors. She said that if the offer were made, her acceptance would be contingent on changes to the festival’s organization. “You need to make this place ready for the future, and if some structural things don’t change, then it’s impossible to do the work,” she said, though declined to provide specifics.If she were to depart the festival, it would likely mean the end of the Wagner family’s creative leadership: No other relative has publicly expressed an interest in taking over.Wagner said that her push to find innovative ways of staging her great-grandfather’s work was necessary, given the “limited repertoire” of the festival — Richard Wagner’s 10 mature works — and global competition among high-profile theaters staging his operas. If Bayreuth just continued to mount old-fashioned productions, she added, “people can just watch a DVD.”The idea of incorporating augmented reality into “Parsifal” emerged in early 2019. Among the challenges was adapting the technology, which is conceived for looking at nearby objects in brightly lit spaces, for a large, darkened theater. Ultimately, Scheib’s team solved the problem by creating a laser scan of the entire auditorium, down to the millimeter.Scheib said that augmented reality would emerge during crucial scenes, and would include a gigantic floating tree and a flaming horse. When Parsifal naïvely kills a swan, a pair of enormous ones will appear to fly near the auditorium’s ceiling, spouting blood.An example of the augmented-reality that viewers with glasses will see in “Parsifal.” Scheib said the uncertainty about the glasses had been a “distraction.”Bayreuth FestivalThis “Parsifal,” however, can still be experienced without the glasses, with sets, lighting and costume design depicting what Scheib described as a “post-human landscape in which the last group of people are hanging on, trying to make sense of faith, forgiveness and belonging.” But, he noted, the uncertainty about the glasses has been a “distraction.”The use of the technology, Scheib said, was in keeping with Wagner’s own way of approaching opera. “He carried out so many innovations, with lighting and architecture,” he added. “Ultimately, he wanted the theater to completely disappear.” More

    • 138 Shares99 Views

      in Music

      A ‘Virtual Rapper’ Was Fired. Questions About Art and Tech Remain.

      by

      Young people are increasingly at ease consuming culture via digital avatars or made with artificial intelligence. Should the same moral guidelines and laws apply to those works?The story of FN Meka — a fictitious character billed as the first musical artist partly powered by artificial intelligence to be signed by a major record label — might seem like a bizarre one-off. In August, Capitol Records dropped FN Meka, whose look, outlaw persona and suggestive lyrics were inspired by real-life music stars like Travis Scott, 6ix9ine and Lil Pump, amid criticism that the project trafficked in stereotypes.But to seasoned observers of technology in pop music and the debate over cultural appropriation, the rise and fall of this so-called robot rapper, whose songs were actually written and voiced by humans, has raised important questions that are not going away anytime soon.Last month alone, an A.I. artwork won a prize in Colorado and a computer program improvised a classical music solo in real time in New York City. From DALL-E 2, the technology that creates visual art on command, to Hatsune Miku, a Japanese software that does something similar for music, the arts world may be on the precipice of a sea change in how its products are created.And young people feel increasingly at ease consuming culture via digital avatars like FN Meka. It has already been happening in hip-hop: A hologram of the rapper Tupac Shakur, who died in 1996, performed at a music festival in 2012; Travis Scott gave a concert through his avatar in the video game Fortnite in 2020; and Snoop Dogg and Eminem rapped as their digital selves and their Bored Ape avatars in a metaverse performance at the MTV Video Music Awards last month.In this brave new world, do fake characters based on real people amount to unseemly borrowing, even theft, or just the kind of homage that has always defined pop music? Even when artificial intelligence does help write music, should the humans behind it be accountable for the machine-created lyrics? And as far as race is concerned, how do the rules of cultural appropriation work when the person doing the appropriating is not a human being with a unique cultural background but a fictitious identity backed by an anonymous, multiracial collective?“A lot of our moral intuitions and codes as humans may have evolved for a context where we have discrete human actors,” said Ziv Epstein, a Ph.D. student at the M.I.T. Media Lab who studies the intersection of humans and technology. “These emerging technologies require new legal frameworks and research to understand how we reason about them.”From left: The Tupac Shakur hologram, Travis Scott in Fortnite and Snoop Dogg at the Video Music Awards.From left: Christopher Polk/Getty Images; via YouTube; MTVFor FN Meka’s critics, the presence of more Black people or people of color in the rooms where the character was conceived, designed and promoted may have helped prevent the negative stereotypes that they say it furthered. Industry Blackout, a nonprofit advocacy group, said FN Meka “insulted” Black culture and leeched off the sounds, looks and life experiences of real Black artists. Capitol seemed to agree when it apologized for its “insensitivity” in a statement.To the critics, FN Meka’s (exaggerated) debt to A.I. and its exclusively digital existence had the effect of absolving the people who were really calling the shots. “There are humans behind technology,” said Sinead Bovell, a futurist and the founder of WAYE, an organization that educates young people about technology. “When we disconnect the two, that’s where we could potentially risk harm for different marginalized groups.“What concerns me about the world of avatars,” she added, “is we have a situation where people can create and profit off the ethnic group an avatar represents without being a part of that ethnic group.”In pop music generally and especially in hip-hop, the culture most likely to be exploited is Black culture, said Imani Mosley, a professor of musicology at the University of Florida.“There’s so much overlap between digital culture and Gen Z culture and Black culture, to the point where a lot of people don’t necessarily recognize that a lot of things Gen Z says are pulled from African American vernacular,” she said. “To interact with that culture, to be a part of that discourse, is to use certain digital and cultural markers, and if you don’t have access to that discourse because you’re not Black, one way to do that is to hide one’s own ethnicity behind the curtain of the internet.”For some, though, vilifying FN Meka’s creators raised the specter of artistic censorship.James O. Young, a professor of philosophy at the University of Victoria who studies cultural appropriation in art, acknowledged there is a long tradition in music of placing a premium on the artist’s lived experience. Young quoted the famous line attributed to the jazz legend Charlie Parker: “If you didn’t live it, it won’t come out of your horn.”But recently the consensus has moved toward sanctioning only art that arises out of lived experience, to the detriment of both art and political solidarity, Young argued. He pointed to an episode five years ago in which a white artist was pilloried for painting the Black civil rights martyr Emmett Till’s corpse.“One of the claims is, ‘This is digital blackface,’” Young said of FN Meka. “Maybe it is.” But he advocated for balanced examination, rather than swift reaction. “You’ve got to be very careful: I don’t think you want to claim that all representations of Black people are somehow morally offensive.”The broader impoverishment highlighted by both sides of this debate is a lack of language and concepts for discussing art that is not, or not entirely, made by people.Epstein, of the M.I.T. Media Lab, cited the thinking of Aaron Hertzmann, a scientist at Adobe Research. In a paper called “Can Computers Make Art?,” Hertzmann argued that at the moment art can be made only by humans, who are the only ones capable of interacting socially with other humans. In this understanding, machine learning is a tool; the artist behind a drawing made by DALL-E or the similar program Midjourney is not the software, but the person who gave it instructions.However, Hertzmann allowed, “Someday, better A.I. could come to be viewed as true social agents.”Meanwhile, as culture is increasingly mediated through the digital realm, questions of how to account for all of the other people who directly or indirectly touched that art will multiply, undermining the conventional notion of the artist as expressing her indivisible perspective.Some art is now the result of “a complex and diffuse system of many human actors and computational processes interacting,” Epstein said. “If you generate a DALL-E 2 image, is that your artwork?” he added. “Can you be the social agent of that? Or are they scaffolded by other humans?”A final question is deceptively profound: Does it even matter who, or what, composes the song, paints the painting, writes the book? Metaverse avatars and A.I. programs are intrinsically derivative: They are all but guaranteed to be riffs on already existing artists and their works.Anthony Martini, a co-founder of Factory New, the virtual music company that created FN Meka, stands firmly on one side of that debate: “If you’re mad about the lyrical content because it supposedly was A.I.,” he said, “why not be mad about the lyrical content in general?” More

    • 88 Shares169 Views

      in Music

      What Is a ‘Fake’ Artist in 2022?

      by

      Subscribe to Popcast!Apple Podcasts | Spotify | StitcherHere’s one sort of simulated artist: This month, a virtual rapper called FN Meka became the center of a critical storm involving digital blackface and the ethics of using artificial intelligence to (re)create cultural production. As a result of the backlash, FN Meka was dropped from Capitol, the major label that had signed the project, though it was debatable exactly how much of the rapper’s music was algorithmically derived at all.And here’s another sort: Spotify continues to populate some of its main playlists with so-called “fake” artists, which is to say, music made by artists under pseudonyms who create tracks purely to populate these playlists at a lower cost to Spotify than artists who are signed to major record labels. They have, in some cases, millions of listens, but outside of the walls of the streaming platform, they fundamentally don’t exist.Are either of these cases acceptable? And more pressingly, are they avoidable?On this week’s Popcast, a conversation about the various ways music is being alienated from the humans who created it and the listeners who hear it, and the philosophical implications for creative agency.Guests:Joe Coscarelli, The New York Times’s pop music reporterRyan Broderick, author of the Garbage Day newsletterTim Ingham, founder and publisher of Music Business WorldwideConnect With Popcast. Become a part of the Popcast community: Join the show’s Facebook group and Discord channel. We want to hear from you! Tune in, and tell us what you think at popcast@nytimes.com. Follow our host, Jon Caramanica, on Twitter: @joncaramanica. More

    • 113 Shares109 Views

      in Theater

      Augmented Reality Theater Takes a Bow. In Your Kitchen.

      by

      The Immersive Storytelling Studio at the National Theater in London is using technology to bring a miniature musical to viewers’ homes. It’s one of several high-tech British projects pushing dramatic boundaries.LONDON — Standing in front of a golden bandstand, dressed in a white satin gown and pearls, the vocalist Nubiya Brandon sashayed to a gentle beat. Stepping toward the spotlight, she took a lazy turn around the stage, singing a playful calypso number and smiling occasionally at the band behind.The weird thing about this show, called “All Kinds of Limbo,” was that Brandon appeared to be in this reporter’s kitchen. The singer was in fact an eerily realistic holographic avatar on a mobile phone screen; her performance had been recorded and was now being broadcast in augmented reality from the National Theater in London.Via the technology’s strange alchemy, which overlays digital imagery onto whatever a camera phone is pointing at, Brandon seemed to be singing and sashaying on the countertop. After she took a bow, her image evaporated and the bandstand faded into nothingness, leaving only a sink full of washing up behind.The success of digital-only theater productions has been one of the pandemic’s surprise silver linings: Audiences have been willing to try them and theater companies have found fans thousands of miles away. But could immersive technologies provide a more intriguing path forward for drama, one that will endure once Covid-19 (hopefully) subsides? Augmented reality (A.R.) and virtual reality (V.R.) are already changing gaming, music and art; might theater be next?“All Kinds of Limbo’s” director, Toby Coffey, said he hopes so. In 2016, he set up the National Theater’s Immersive Storytelling Studio, which operates as a kind of “start-up” within the company, he said in a recent interview at the studio’s modest space, which was crowded with a jumble of technical equipment. The team’s brief is to see how live theater and new technologies can interact and intersect.Toby Coffey, who founded the National Theater’s Immersive Storytelling Studio in 2016.Suzie Howell for The New York Times“Theater makers are naturally fascinated: They’re used to working in 3-D,” Coffey said. “As soon as you bring a director or stage designer or choreographer into V.R., you see their brains whirring.”The studio’s first production, “Fabulous Wonder.land,” was a V.R. music video featuring a track by the musician Damon Albarn with words by the playwright Moira Buffini. The team has since made 360-degree films of live shows, developed a one-on-one piece in which an audience member interacts with a live actor while wearing a V.R. headset and created a mixed-reality “exhibition” about government welfare cuts.“All Kinds of Limbo” came into being in 2019 after the National Theater had a hit with “Small Island,” a play about postwar Jamaican immigration to Britain. Coffey and his team commissioned Brandon, the vocalist, and the composer Raffy Bushman to create a 10-minute song sequence responding freely to the play’s themes. It was written, performed and motion-captured that year, and was initially presented as a V.R. experience in one of the theater’s event spaces.Brandon performing in a motion capture studio to record “All Kinds of Limbo.”The National TheaterWhen the pandemic shut down British performing venues in March 2020, Coffey accelerated plans to turn “All Kinds of Limbo” into an at-home experience. The retooled version can be watched via A.R. on a mobile device, via a V.R. headset, or on a regular computer. Brandon’s performance stays the same, but, depending on the device used, the experience feels subtly different.To summon some of theater’s shared intimacy, it’s being ticketed and broadcast as live, although the show is recorded. Other people attending virtually are represented by blades of moving white light and, by playing with the settings, you can move around the space and see the action from different angles.It’s a short piece, but “All Kinds of Limbo” does feel like the glimmering of a new art form: somewhere between music video, video game and live cabaret show.Over the last few years, Britain’s theater scene has become a test bed for similar experiments. Last spring, the Royal Shakespeare Company co-produced an immersive digital piece called “Dream” that featured actors performing using motion-capture technology and was watchable via smartphone or computer. Other projects, such as shows by the Almeida theater in London and the company Dreamthinkspeak in Brighton, England, require participants to turn up in person and get equipped with VR headsets.Francesca Panetta, a V.R. producer and artist who was recently appointed as the alternate realities curator at the Sheffield DocFest film festival, said in a video interview that practitioners from audio, gaming, theater, TV and other art forms were collaborating as never before. “Many different people are trying to explore this space and work out what it really is,” she said. “No one is quite sure.”One of the most keenly awaited partnerships is between the immersive theater troupe Punchdrunk, which pioneered live site-specific shows such as “Sleep No More” and “The Masque of the Red Death” in the mid-2000s, and the tech firm Niantic, best-known for the wildly successful A.R. game Pokémon Go.Speaking by phone, Punchdrunk’s co-founder Felix Barrett seemed invigorated by the creative possibilities. “We’re on the cusp of a new form of entertainment,” he said. “It’s a new genre; it just hasn’t been named yet.”Later this year, Niantic and Punchdrunk plan to unveil the first results of their collaboration. Barrett was reluctant to reveal too much, but said that it would offer participants “a citywide adventure” that will feel like an immersive video game happening in the real world. “Our goal is to try and make you the hero of your own living movie,” he said.Ambitious as such projects are, they are also — at least by theater standards — time-consuming and forbiddingly expensive. The Royal Shakespeare Company’s “Dream” wouldn’t have been possible without corporate sponsorship and a hefty grant from a roughly $55-million British government fund promoting digital arts innovation. The latest iteration of “All Kinds of Limbo” relies on a partnership with Microsoft and the livestreaming platform Dice.Production work on “All Kinds of Limbo.” The show can be watched via A.R. on a mobile device, via a V.R. headset, or on a regular computer, through Jan. 30.The National TheaterThere’s also the question of audience. Though theater-led projects such as “Dream” and “All Kinds of Limbo” have gained positive reviews, they have attracted only a tiny fraction of the 12 million viewers who watched a 2020 virtual performance by the rapper Travis Scott in the online game Fortnite. The chances of monetization at scale look slim, at least for now.And the irony is that, while the pandemic may have whetted audience appetites for digital drama, it has had devastating consequences for theater companies themselves. The National Theater’s Immersive Storytelling Studio originally had four staff members; after belt-tightening layoffs in the company, it’s now just Coffey and one full-time co-worker. “Even before the pandemic, we could have been doing 10 times more than we had resource to be able to do,” Coffey said. “We need to work within those restrictions.”What happens next is up for debate. The National Theater is working on redeploying the app and distribution platform used for “All Kinds of Limbo” into something that works for other projects. Panetta said that the metaverse, if it genuinely takes off, offers its own possibilities for live performance. “It’s difficult to see what the pathway is; I suspect it’ll be a mix of many different things,” she added.So how long until we’re watching Ibsen or Shakespeare in augmented reality at our kitchen tables? Coffey laughed, then cautioned that designing successful A.R. performances was still an emerging skill. “But some day it’ll happen, I have no doubt,” he said.All Kinds of LimboStreaming through Jan. 30; allkindsoflimbo.com. More

    • 125 Shares189 Views

      in Music

      Don Letts, Mad Professor Team With Times on Carnival Story

      by

      #masthead-section-label, #masthead-bar-one { display: none }At HomeBake: Maximalist BrowniesListen: To Pink SweatsGrow: RosesUnwind: With Ambience VideosAdvertisementContinue reading the main storySupported byContinue reading the main storyTimes InsiderBreaking Down the Sounds of Carnival, on Your PhoneThe pandemic has dampened the celebrations worldwide. But a Times special project, which includes an interactive music mixing feature, lets readers get into a party mood.The Notting Hill Carnival in 2019. Although parties over the past year have been canceled, a Times project seeks to keep the Carnival spirit alive this winter. Credit…Peter Summers/Getty ImagesFeb. 19, 2021Updated 10:48 a.m. ETTimes Insider explains who we are and what we do, and delivers behind-the-scenes insights into how our journalism comes together.The world could definitely use a party. Unfortunately, because of the coronavirus pandemic, cities around the globe have canceled or curtailed what is annually their biggest bash, Carnival.These celebrations, many of them pre-Lenten, trace back hundreds of years to the Caribbean islands, and the tradition has continued with the Caribbean diaspora in cities like New York (the West Indian American Day Parade), Toronto (Caribana) and London (Notting Hill Carnival).In the chill of February, readers of The New York Times can get a flavor of the sensory richness of these blowouts with Carnival in Winter, a special online package of multimedia experiences about the festivals produced by the Narrative Projects department.Carnival is all about history, community, costumes, food — and music. To immerse readers into the sonic experience, Narrative Projects teamed with the Graphics department and the R.& D. department to create a special effect on Instagram that puts you inside a classic Carnival song. The effect, through the Instagram Spark feature that can be downloaded on phones, uses augmented reality, which lays a computer-generated image or animation over a user’s view of the real world. (Click here from your phone.)In this case, you can hear a Carnival anthem broken down into four musical tracks — and you can “see” the tracks through Carnival-inspired 3-D animations and manipulate the music by moving in your physical space. The effect turns your phone — and living room —  into a virtual mixing board.Picking just one song to represent the music of Carnival — which incorporates soca, calypso, reggae, dub, house and more — is a near impossible task. So, as one of the editors on the project, I reached out to an expert.Don Letts at the Roxy in London in 1977. Recently, he suggested the song that was used in The Times’s interactive feature on Carnival music. Credit…Erica Echenberg/Redferns, via Getty ImagseDon Letts, a 65-year-old filmmaker, broadcaster and musical matchmaker, is an icon of the British music scene. In the 1970s as the D.J. at the Roxy in London, he introduced the club’s punk clientele to reggae, the rising sound from Jamaica, his parents’ homeland. Between his friendship with Bob Marley and his close ties to the fledgling punk scene (he later formed the band Big Audio Dynamite with Mick Jones of the Clash), Letts earned the nickname “The Rebel Dread” for bucking convention and orchestrating cultural collisions that changed the course of popular music.He also has been a regular at the Notting Hill Carnival for over 40 years. In 2009, he directed a documentary, “Carnival!,” on the history and politics of the festival.Asked to name a “typical” Carnival anthem, Mr. Letts at first dismissed the task as impossible. Upon reflection, though, he directed us toward an old friend, the producer Mad Professor, and his 2005 track “Elaine the Osaka Dancer” — “A strange title, I know,” said Mr. Letts — which was written for a performer, Panafricanist, on the Mad Professor’s label. Mad Professor, whose name is Neil Fraser, is himself a well-known name in British music history. He pioneered the emergence of the British dub sound and collaborated with performers like Sade and Massive Attack.Mr. Letts chose “Elaine” because, as he put it: “At Carnival you can stand on a street corner and hear a float going past with steel pans, along with the sound of a Jamaican sound system right around the corner. This song perfectly captures that sound: the collision of calypso and soca with the bass-heavy rhythms of reggae.”If you’re reading this article on a desktop computer or tablet, you can view the AR experience on Instagram by opening your camera on your device and point to this QR code. For those reading on their phones, the link in the story above will call up the effect. Credit… Mad Professor agreed to license the song, so we asked him to break it down into individual instrumental tracks or “stems,” each of which would then be manipulated by the user of the Instagram effect.This process proved to be slightly more analog — and painstaking — than anticipated. At one point, when asked for a progress report, Mad Professor relayed that he was “baking the tapes” — which might sound (or did to me, anyway) like a bit of music producer slang. In fact, it’s a literal description of the process in which analog master tapes are restored by exposing them to a high temperature for hours, reducing humidity that can affect the quality of the tapes.Once the tapes were baked and the stems were procured, our graphics and R. & D. team built the Instagram effect. With the effect, the user can play with the drums, bass, horns and steel pan tracks while seeing commentary from Letts on why each element is crucial to a Carnival song.It’s not the same thing as dancing to steel pans on a simmering street in London’s Notting Hill in the heat of summer. But in a year when Carnival has been canceled nearly everywhere, we hope it gets you as close to that feeling as possible.AdvertisementContinue reading the main story More