10 tracks that harness the power of artificial intelligence

Despite the numerous AI platforms which serve up routes to auto-generate functional music, many artists who have overtly worked with AI have approached the concept via more individual means. 

Take Holly Herndon, the Berlin-based composer and musicologist who recently created her own intelligent musical accomplice. Dubbed ‘Spawn’, this vocal-sample generator was taught by Herndon and partner Mat Dryhurst to reproduce a bank of vocal-types (including her own) via months of training its complex neural network. Spawn was able to organically add vocals to tracks presented to it. 

Though, as Herndon told Art in America, the process is still finding its feet: “AI is not that smart, it’s very low fidelity, it’s not real time, it’s very slow and unwieldy. Spawn can take more than 24 hours to process someone’s vocal input. On the other hand, it has some unique capabilities that are pretty exciting-slash-scary. The AI can extract the logic of something outside its operator’s own logic and re-create it. This is entirely new for computer music.”

A virtual colleague

Herndon’s approach – to upturn the often predictable creative choices of the human musician, and hack out new musical avenues of exploration – is a commonality shared across numerous artists who have worked with AI. Alex Da Kid’s ‘collaboration’ with IBM’s Watson was triggered by the Grammy-winning producer’s interest in whether it was genuinely possible to make a song with a virtual colleague. 

Watson uses an accumulation of data gathered by a web of smart APIs. These include Watson Alchemy Language, which studies five years of media to determine current pop cultural themes, Watson Tone Analyser, which similarly analysed around 26,000 lyrics, and, crucially, Watson Beat, which determines the best chords, keys and frameworks to correspond with a certain theme. With the track Not Easy, the pair explored the theme of heartbreak – and produced a stunning statement, that also saw interjections from Wiz Khalifa, Elle King and the X Ambassadors. The brilliant end result was popular enough to top both the iTunes and Spotify charts. 

While Alex and Holly’s involvement with AI was driven by a desire to research the potential of AI, YouTube star Taryn Southern’s stunning I Am AI LP was conceived when the singer/songwriter was finding it difficult to realise the musical ideas she had in her head. Using a combination of AI-music generators Amper Music and Aiva as well as Google’s Magenta and IBM Watson, Taryn created a new musical toolkit with which to work. As Southern told Digital Trends, her approach was to use the fine-tuning tools of software like Amper to deviate from her original ideas, download the stems and then rearrange the end results in her DAW.

These artists, each venturing further down the AI rabbit hole, albeit differently, are at the vanguard of a new paradigm for creators. It’s almost a question of ‘when’ as opposed to ‘if’ AI will cease to be regarded as an industry buzzword and become an everyday facet of music creation. Via their work, these artists are laying the foundations for the fruitful absorption of AI into the creative process.  

Learning to fly

While these examples only scratch the surface of how AI has been applied by artists in various genres and contexts (see our list of ten AI-built records), in other areas, AI has already fully started informing our daily lives. 

It’s entirely suffused itself into how we listen to our music. Pervading streaming platforms and music-listening services are algorithms which smartly serve up similar tracks to the ones we regularly play, building cleverly curated playlists based on the data imparted by our listening habits. 

Spotify’s ‘Discover Weekly’ smart playlists were designed in collaboration with French AI startup Niland. This smart neural network determines how to best populate these lists by scanning other users’ playlists that feature these tracks, as well as analysing the waveforms of the tracks to determine musicological similarities. “We’re working on a number of ways to elevate the experience even further.” 

Spotify’s research lead, Rishabh Mehrotra explained to AI News, “Reinforcement learning will be an important focus point as we look into ways to optimise for a lifetime of fulfilling content, rather than optimise for the next stream. In a sense this isn’t about giving users what they want right now as opposed to evolving their tastes and looking at their long term trajectories.”

Fear not

We’ll take the liberty of skirting around the numerous prophecies of doom foretold by those fearing AI’s encroachment into their financial territory. Let’s instead consider the alternative, that while online AI platforms can easily create the type of utilitarian music which accompanies a simple YouTube video or a podcast, for genuine soundtracking requirements, real human composers will certainly be needed. 

Feeling, heart, soul and also the ‘story’ of the soundtracking process are all essential to modern film, television and video game soundtracks. In production terms too, there’s so much that AI can bring, as opposed to take away, from the modern producer’s workflow. “Will AI take away all mixing jobs? Definitely not,” says Sonible’s CMO Alexander Wankhammer. 

“Will it change the way the industry is working in some fields? Definitely yes. I think people should not be afraid of the new possibilities AI is creating – they should remain curious and see AI as a new, really exciting tool that will (up to a certain extent) shape the future of music production.”

10 recordings that harness AI

1. Outside - David Bowie (1995)

While not strictly speaking ‘AI’ in the modern sense, Bowie’s use of the Verbasizer software to concoct new lyrical ideas was something of a digital continuation of the randomising ‘cut-up’ technique he’d relied on for earlier works. Many of the tracks on Bowie’s hidden gem Outside were mapped out of these computer-suggested ideas.

2. Not Easy - Alex Da Kid (2016) 

A collaboration between Dr Dre and Eminem producer Alex Da Kid and IBM’s cultural learning and idea-forming digital intelligence, Watson. Told to investigate the theme of ‘heartbreak’, Watson provided Alex with a suitable rhythmic, lyrical and chordal foundation. The stunning results also featured Wiz Khalifa, Elle King and X Ambassadors.

3. Return 0 - Ash Koosha (2018)

Created by Iranian composer Ash Koosha in conjunction with artificial intelligence Yona, Return 0 is a sometimes beautiful, sometimes challenging work that emphasises its innovative conceit in its futuristic arrangements. Capable of generating melodic sequences and even its own lyrics and singing, we’re very keen to see what Yona does next. 

4. Smartbeats - Toro Y Moi (2019)

Built in tandem with commercially available AI soundscape designer Endel, American producer and songwriter Toro Y Moi built up a series of four tracks for the purposes of promoting Glaceau Smartwater. The Smartbeats EP contains the tranquil and emotive fruits of his – and his virtual assistant’s –  labours. 

5. Holly Herndon - Proto (2019)

Harnessing ‘Spawn’, Herndon’s self-trained AI vocal generator, Proto’s melding of the characterful artistic sensibilities of Herndon, and the twisting, rule-bending Spawn result in a captivating listen. Its glitchy, computer-ised textures battles against Holly’s creative persona in fascinating ways, ultimately leading to the aural equivalent of the merging of the two. 

6. Shadow Planet - The Cotton Modules (2021)

Examining the space between human composition and AI-driven arrangements, Oakland-based duo The Cotton Modules’ debut is a remarkable listen. Working with machine learning decision-maker Jukebox, the record swims in ambient, evolving textures and loops. In making the record, the duo came to regard this other sentience as an instrument that they needed to learn how to work with. 

7. Hello World - SKYGGE (2018)

Benoit Carré presented Sony CSL’s Flow Machines with ideas, which the AI then built out. He took that and made a solid pop record. Rather than a neural network, Flow Machines used ‘Markov chains’, probability equation generators that need less data to work.

8. I Am AI - Taryn Southern (2018)

Perhaps the most high-profile AI-built record on the list. YouTube superstar Taryn Southern’s eight-track LP was constructed with music composition fine-tuner Amper. Southern’s work received widespread press.

9. The Lost Tapes - The 27 Club (2021)

A novel idea, to encourage musicians to speak openly about mental health. The 27 Club used AI to make new tracks, based on the music of those famous, epoch-making musicians who oddly all left us at the age of 27. With real vocalists replacing the late singers, this is AI used for the human act of remembrance. 

10. Music Samples In Style of RATM - AI Kittens (2021)

Similarly to The 27 Club project, this album was created by an AI which studied the typical structure of the rap metal quartet’s canon and generated short samples, which artists AI Kittens carved into robust guitar workouts.