“We strongly believe that in the coming years, writing code manually won't be a thing. What will be left is the fun stuff”: Inside the new wave of AI tools turning prompts into plugins

The first that most of us heard of vibe coding was when Rick Rubin called it the “punk rock of software.”

For those of you not up on your punk software, vibe coding is a cutting-edge technique that utilizes an AI assistant like Claude or ChatGPT to develop software applications through prompts. Or, according to IBM, “a fresh take in coding where users express their intention using plain speech and the AI transforms that thinking into executable code.”

If you’re an electronic musician, you’re probably already thinking it: can vibe coding create plugins? The answer is yes, with a few caveats. Until recently, you still needed to have some knowledge of coding languages like C++ and DSP – digital signal processing – to get things to work. But thanks to three new platforms that simplify the process, you can now vibe-code bespoke plugins to use in your productions with no expertise in coding or DSP required.

One of the first plugins to make plugins to appear on the scene was actually not a plugin, but a Max for Live device. Called ChatDSP, the software from respected Max for Live developer Dillon Bastan slots into Ableton Live as an instrument, audio effect or MIDI effect.

Enter a prompt in natural language about what you want - a reverb for example, or an FM synth - and ChatDSP will interface with an AI agent of your choosing to create the custom effect or instrument. It will also generate up to 48 mappable parameters for controlling the device that you can automate, with additional inbuilt standard parameters like pitch bend and mod wheel for instruments, and dry/wet mix for effects.

Interfacing with an AI agent like Anthropic’s Claude isn’t free, so you’ll first need to add credits to the AI and get an API key, something like a password that you enter into ChatDSP so it can communicate with your agent. Prompts cost money, and the more you converse with the AI, the more it costs. While this may seem complicated, the manual walks you through the process, and the good news is this is the trickiest part of the entire process. Once you’re set up, things work as advertised: enter prompts, get results.

Reactions to ChatDSP have been, as you might expect, mixed, with some excited about the possibilities inherent in the new technology, and others upset about the use of AI. “There are many many valid concerns about AI, especially big AI, data centers, and the effect on the environment, industry, and warfare, so I'm not surprised at all and respect people hating it,” says Dillon. “There is so much harm in all our industries that are a part of our daily life, from using a smartphone, eating meat, using your car, and of course using AI. Everyone draws different lines to reduce harm, so I think it’s very understandable that many people's lines are at AI.”

While early iterations of ChatDSP only allowed the use of large-scale AI agents like Claude or ChatGPT, Dillon has since updated it to include local AI as well, giving users more of a choice. “I also don't like a ton of things about AI,” says Dillon, “so I was happy to add local model support in it. The one aspect I don't see eye to eye with is the technical gatekeeping and elitist aspect. While deskilling is a major concern for AI, especially in education, some of the criticisms remind me of classical musicians hating the keyboard, drum machine or DAWs.”

Dillon admits that he's not an enthusiastic proponent of AI, choosing to create ChatDSP because the technology had reached a point where it had become feasible. “Once I have an idea that seems interesting, I generally make it and move on to the next one. [ChatDSP] was one passing idea I had. Of course, I love bringing whatever my imagination conceives to life and this device can do that itself for smaller projects.”

ChatDSP - Device Walkthrough - YouTube

Watch On

Unless you’re an Ableton Live Suite user, you won’t have access to Max for Live. That doesn’t mean you can’t create your own plugins from within a DAW though, as the developer Artists in DSP has created its own vibe-coding software generator, Amorph.

Like ChatDSP, Amorph includes a suite of options to create different kinds of plugins: instrument, effects and MIDI. These are available in VST3 and AU formats for both Mac and PC, and so should work in most major DAWs. Amorph is also free.

To create your own plugin, begin by writing a prompt. Amorph will turn your text into code, what Artists in DSP calls a “pre-optimized payload,” that you’ll then have to paste into an AI agent. Take what the AI spits out at you, drop it back into Amorph, and it will be compiled into a plugin that you can then play with.

“We were classic plugin developers,” explains Artists in DSP CEO Leo Biasca-Caroni on why the team went the vibe coding route. “We’ve been developing VST and AU plugins for Windows and Mac for three years [but] we never thought that agents would become smart enough to create a whole audio plugin.”

The seed of the idea came from a user, who pointed the developers in the new direction. “He mailed us and said that Polyend Endless is a promptable pedal,” says Leo. “If that existed for VSTs or AU, that would be a huge [opportunity] for us.”

As with ChatDSP, the reaction from the music production community has been polarized. “We were expecting to get skeptical responses to it,” admits Leo, “because, of course, you are afraid when you see that big of a change of a paradigm. It changes the way you think, you work. On the other hand, we also received a lot of messages saying that this way of using AI is finally as a tool. We believe it’s a tool that gives you the power to use AI as a creative partner and not as a substitution for you.”

Being able to create custom plugins as the need arises opens up new areas of creativity for the musician and producer. “You know the [plugin] you want to have. You know how you want to play it. It's just too hard to find [something] on the internet that exactly meets your expectations. That's what Amorph is for.” Interestingly, it can also allow for happy accidents and unintended results, Leo says. “The errors inside the prompt can sometimes bring this creative serendipity.”

Amorph is still in open beta, but it’s getting closer to version one. At the time of publication, it was at v0.99, which includes the ability to generate a custom UI, offers sample-accurate MIDI timing, and gives users access to The Hub, an integrated cloud browser to preview and install community-created patches.

Artists in DSP is hoping to integrate communication with AI agents into the plugin in the official release, so you no longer have to copy and paste back and forth. You’ll also be able to use your own API key, as with ChatDSP. “You get your API key and you can insert it inside the plugin, and then you’ll have an agent that is not only able to generate the code, but also to change the code directly, iterate, and fix the errors,” says Leo.

Another feature the team is hoping to expand is The Hub. Explains Leo: “Right now we have a quite clear image of what Amorph is going to be. It will become a marketplace, because we want to keep it free. We're pushing it to become a sort of Stripe for audio plugins, for generated DSP plugins.

“You can imagine it as a place where inside the plugin, you have a marketplace and can download free or paid tools that other people are generating. And, if you download that tool from someone else, you can see the code inside of it. You can also modify it. You can transform it.”

Prompt Custom Audio Plugins [Amorph v0.9.9 - Walkthrough] - YouTube

Watch On

Rather than work within your DAW as a plugin or Max device, Pluginmaker.ai uses your web browser as the building and testing platform. And, unlike the other two platforms discussed here, Pluginmaker.ai delivers a unique VST or AU plugin for you to use - or sell - as you see fit.

Described as “the first-ever browser-based AI-powered VST plugin generator,” Pluginmaker.ai creates your instrument or effect idea based on prompts typed into the browser. You can then test it via MIDI, on-screen keyboard or built-in sequencer right there, and adjust parameters and create new iterations until you’re satisfied with the results, after which you can download the completed plugin. Pluginmaker.ai provides a marketplace for you to sell (or give away) your creation, keeping 10% of any sales. You’re also allowed to sell it independently.

To do all of this, you’ll need to subscribe to the platform, with different pricing tiers generating differing amounts of credits that you use when prompting - the standard procedure for working with an AI agent. (Polyend has a similar system for its Endless pedal.)

“It all started when [co-founder] Maks Padalak asked himself a question,” says the other co-founder of Pluginmaker.ai, Dominik Bilski. “‘What if people could generate any plugin they want in minutes for a dollar and preview the result instantly on the web?’” Dominik goes on to explain how Maks, a front-end developer at Nord Security and musician, wanted to build his own plugins but was surprised by “how insanely high the entry bar was to even start.”

Pluginmaker.ai’s goal is to democratize plugin creation and access to plugins themselves. “Right now, building a plugin requires deep knowledge of DSP and C++, which limits creation to a very small group of highly specialized developers,” says Dominik. “That's an enormous entry barrier. And quality plugins are expensive. Most producers, especially bedroom producers just starting out, can't justify spending hundreds of Euros on individual tools. Our goal with pluginmaker.ai is to enable anyone to create whatever plugin they can imagine for a fraction of the price.”

Although the platform is still in beta, users have already created a number of different instruments and effects, all with beautiful graphical user interfaces, something notably lacking from many vibe-coded plugins so far. It’s clear to see how this could seriously disrupt the industry as it develops.

“The software industry is already shifting in this direction and there's no doubt about that,” says Dominik. “VST plugins are ultimately software, and the cost of all software is trending toward zero. We strongly believe that in the coming years, writing code manually won't be a thing. What will be left is the fun stuff - creating things and solving problems. In our eyes this is one of the biggest revolutions in human history. Possibly bigger than the internet.”

AI Audio Plugin Generation Talk @ Ableton HQ Berlin - pluginmaker.ai - YouTube

Watch On

While the three platforms discussed here offer new ways to engage with music production, they do bring up some thorny ethical issues. Data centers are not only gobbling up RAM and other tech that could otherwise end up in things like cars and laptops, they’re incredibly harmful to the environment, with large facilities swallowing up to five million gallons of water a day, the equivalent to a town of 50,000 people.

As one commenter on ChatDSP's walkthrough video wrote, “Using the power of the omniscient LLM, we can now make the most basic of Operator patches within a matter of hours of trial and error using the yearly energy consumption of a mid-size European city.” This is one reason why Dillon gave users the option of switching to a local AI model in the device.

Another issue to keep in mind when using a platform like the ones profiled here is the source of the code that's used to create the plugins. AI companies across the board have been criticized for training their models on copyrighted material, whether that be photos, videos, text, or music.

Artists in DSP, for their part, have written pre-made code in Amorph for the AI to work with. “You can think of it like building Legos,” explains Leo Biasca-Caroni. “You have Lego building blocks. And those are like oscillators. You have filters. And you just put them together and give the AI in the system some graphical knowledge.” In this case, then, the AI is working with ethically sourced code, compiling it into something that you can use in your DAW.

Lastly, there is the possible impact on traditional software developers. Are their livelihoods at risk? While this is still cutting-edge technology, it’s developing rapidly, and Dillon Bastan sees the writing on the wall: “Unless AI and technology collapse in the near future, the way we work with technology will become more abstracted because of AI. I suspect the developer industries will change rapidly.”

For Leo of Artists in DSP, he believes that developers will need to get on board or be left behind. “If you don't follow the rhythm of AI, there will be a big change. You have to adapt to the way music production is working with technology nowadays.”

Pluginmaker.ai also views AI as the future of software development. “As software developers, we were genuinely worried about our futures at first,” says Dominik Bilski, “until we quit our jobs and decided to do something about it instead of sitting around being scared. We chose to use the AI revolution to create something special rather than watch it happen to us.”

Perhaps there’s a future where traditional plugin development and vibe coding can exist side by side in harmony. After all, we all know how to cook our own food, yet the restaurant industry remains profitable. Not everyone will want to spend the time creating plugins (even with the help of AI) when they’d rather be making music. And many discerning producers may reject AI-made plugins, much as we choose not to listen to AI-generated music. Only time will tell.

ChatDSP is priced at $10 and is available from Dillon Bastan’s website.

Amorph is available for free from the Artist in DSP Gumroad page.

Pluginmaker.ai offers two pricing levels as well as a free option to get started and check out the marketplace.