Listen to an AI-powered neural network 'sing' a cover of Dolly Parton's "Jolene"
The track features a computer-generated Voice Model trained on the voice of electronic musician Holly Herndon
Plenty of artists have given us their spin on Dolly Parton's "Jolene" over the years, but Holly Herndon's take is without doubt the most intriguing rendition we've come across yet.
On first listen, it might sound like a decent, if somewhat uninspired cover of the 1973 country classic. But the voice you're hearing in the track isn't the sound of electronic musician Holly Herndon's vocal cords - it's the sound of a neural network, imitating her singing voice through the power of machine learning.
The neural network, nicknamed Holly+, was developed by Herndon in collaboration with Never Before Heard Sounds. Described as Herndon's "digital twin", Holly+ is a Voice Model, a "deep neural network" capable of generating raw audio that convincingly mimics an individual voice.
The network is trained on recordings of speech and singing from a target voice, and can be used to synthesize sung phrases using text input, or to accomplish "audio style transfer", where audio from an existing recording can be converted to resemble the target voice. Those curious about the technology's potential are able to submit their own audio to be reproduced in Holly+'s voice through a website set up by Herndon.
Herndon demonstrated Holly+ live in a recent TED talk, which you can watch below. In the video, she outlines the radical possibilities opened up by the technology and explores the ethical and legal questions they present, suggesting that soon, any producer or musician could be able to sing in the voice of their favourite vocalist.
In the live demonstration, beginning at 5:25, vocalist Pher alternates between two microphones, performing a song both in his own voice and that of Holly+, before duetting with both at once.
Writing in a blog post published following the announcement of Holly+ last year, Herndon stated that she's "confident that generating convincing spoken and sung voices will soon become standard practice for artists and other creatives, as presaged by the popularity of celebrity vocal deep fakes already found all over YouTube."
Get the MusicRadar Newsletter
Want all the hottest music and gear news, reviews, deals, features and more, direct to your inbox? Sign up here.
"As our ability to produce more detailed and convincing voice generation evolves, so too will the need for comprehensive, high fidelity vocal training data, as well as the urgency of discerning provenance," she continues. "For this reason I believe that there will be demand for official, high fidelity, vocal models of public figures, so why not experiment with my own?"
A prominent question raised by Herndon's work is how music produced by both authorized digital likenesses such as Holly+, and unauthorized 'deepfake' voice models, would be dealt with by the music industry's existing framework of intellectual property rights.
Herndon proposes that the public be given "open source access" to her voice model, governing the access, decision-making and profits made from her digital twin through a DAO (Decentralized Autonomous Organization), a blockchain-based organization that would collectively act as stewards of the voice model and any works produced using the technology.
"The Holly+ model creates a virtuous cycle," Herndon writes. "I release tools to allow for the creative usage of my likeness, the best artworks and license opportunities are approved by DAO members, and profit from those works will be shared amongst artists using the tools, DAO members, and a treasury to fund further development of the tools."
I'm MusicRadar's Tech Editor, working across everything from product news and gear-focused features to artist interviews and tech tutorials. I love electronic music and I'm perpetually fascinated by the tools we use to make it. When I'm not behind my laptop keyboard, you'll probably find me behind a MIDI keyboard, carefully crafting the beginnings of another project that I'll ultimately abandon to the creative graveyard that is my overstuffed hard drive.
“An amazing sounding product, and very capable thanks to its augmentation in operation, geared towards ultimate creativity”: GForce Oberheim OB-1 review
“There’s constant hype about the instruments, synths, speakers and computers that producers use - but will your music sound better because of that?”: Gábor Lázár on sound design, generative techniques and designing his own instruments in Max