Can a Synthetic Voice Be Taught to Sing Opera?

The opera opened with amplified breathing: gasps, hisses, labored inhalations. A string quartet introduced spidery harmonics that consolidated into brighter chords and were joined, over time, by radiant voices. Exuberantly lyrical, their lines unfolded in stark contrast with those of the protagonist, who, strapped into a wheelchair center stage, had thus far contributed only some short comments in the machine tones of a text-to-speech synthesizer.

But then the synthetic voice began to sing, cutting through acoustic textures with a sound profile all of its own. In the upper register it seemed to combine the timbre of a boy soprano with a brushed metal finish, while the lower range had some of the compressed warmth of a countertenor. The voice, an uncanny combination of expressive directness and can’t-quite-place-it strangeness, moved from one note to the next with the slick flexibility of rubber.

The voice belonged to Mark Steidl, the star and co-librettist, with Katherine Skovira, of “The Other Side of Silence.” The first act of this opera, composed by Robert Whalen, was presented last week in a public workshop at the Experimental Media and Performing Arts Center at Rensselaer Polytechnic Institute in Troy, N.Y.

Steidl has cerebral palsy and speaks through an augmentative and alternative communication (or A.A.C.) device, which can make ordinary interactions painfully slow. Making space for underrepresented voices has become a stated priority for much of the opera world. To tell the story of a nonspeaking disabled character in “The Other Side of Silence,” a team of creators, researchers and software developers had to first learn how to engineer the voice itself.

The opera’s creators believe that Steidl’s singing voice is the first case of a generative synthetic voice taught to sing opera. While “The Other Side of Silence” depicts a disabled person’s struggle for creative flourishing and agency, the underlying theme of opportunity and fear in the age of artificial intelligence has a Faustian resonance that fits comfortably into this art form’s canon.

In the work, which is being developed with Opera Saratoga, Zari, a nonspeaking, nonbinary character based on Steidl, is heavily dependent on a team of caregivers, including a mother who chafes at her child’s gender identity. Zari decides to move into an experimental smart home in a remote desert, run by an A.I. entity called the Chimera that promises unprecedented independence. But with access to Zari’s thoughts, the Chimera begins to take over their decision-making and, in a medical crisis, intervenes in ways that leave Zari’s mind altered forever.