It used to be you could spend an afternoon drumming up a home movie with your little sister, soundtrack it with your favorite mixtape cuts, and upload it to the Internet for sharing without a care. But as bot-driven copyright tools started scanning new uploads and society generally became more concerned with proper song licensing, using whatever track you desired for some publicly available Web video became less of an option.
What's an amateur-at-best musician to do? I may have marched on a collegiate snare line (and therefore understand rhythm, phrasings, and tempo), but my ability to create melody probably stopped with middle school recorder lessons.
Luckily, we musically challenged filmmakers and podcasters now have robots. A number of high-profile AI composition initiatives have surfaced in recent years—perhaps most notably, Sony's Flow Machine released its debut album in January—and slowly but surely these tools are moving from the research labs and professional production studios into publicly available spaces.
So when I was recently left by myself for a weekend with just my dog for company and a desire for late night tacos, I decided the time had come to reenter the short film arena. The above documentary on Ernie the Shih Tzu is scored entirely by smart composition tools you can access right now. Would-be children's publishers: feel free to email me directly about his expanded universe. For everyone else, here's the lowdown on four song-generating programs you can use to help people get down.
Chrome music maker
Released on March 1, Chrome's Song Maker represents the newest of these tech-y composition tools. Technically, this isn't an artificially intelligent composer. Instead, it's a tool that simplifies composition to the point where anyone can do it regardless of your familiarity with pitch or rhythm. It greets users with a grid representing beats on the X axis and tones on the Y, and filling in any individual square will generate a sound. You have five options for tonal instruments (piano, synth, strings, marimba, woodwind) and four for percussion (electronic, blocks, kit, conga). You can manually point and click; you can add through a MIDI keyboard or by singing into a mic; you can even let your keyboard keys do the songwriting if desired.
Song Maker comes from Chrome Music Lab, a Google initiative to build browser-based tools for learning basic music principles. The Music Lab has a dozen or so tools already focusing on everything from oscillators to rhythm to arpeggios. So while this may be aimed literally at the elementary school demographic most of the time, Song Maker has enough customization options for any novice composer to be effective. You can make your loop up to 16 bars, change into funky time signatures like 5/4 or 12/8, break the note subdivisions into triplets or sixteenth notes, choose any key you'd want by alternating scale and starting note, and extend your range up to three octaves.
The lone issue for my purposes? Song Maker only allows you to choose a single percussion and tonal instrument pairing for your composition. I wanted a meatier, spy thriller-inspired ditty for the moment my dog reveals his day job (roughly the 1:21 mark of the video). Ideally, I could combine the basic beat I crafted with a higher-octave string line. But even if you're cautious enough to choose the same amount of bars and the same tempo, there doesn't appear to be a way to layer multiple tracks on top of one another for fuller songs within Song Maker. That requires a basic sound-editing program and the ability to capture sound from a browser in this instance. The limitations of Song Maker therefore prove to be both positive and negative—things are simplified enough that it would be hard to create something truly unlistenable, but compositions have a complexity ceiling, too.
Google’s Magenta tools
This being Google, of course there are multiple initiatives in progress to upgrade the music-making process. Compared to Chrome Music Labs, the Magenta initiative represents the headier approach.
Introduced back at the 2016 Moogfest (increasingly a bucket-list event for music tech fans), Magenta wants to leverage artificial intelligence and machine learning to empower anyone to be musically creative.
“The goal of Magenta is not just to develop new generative algorithms, but to 'close the creative loop,'" the team wrote when introducing N Synth last year. "We want to empower creators with tools built with machine learning that also inspire future research directions. Instead of using AI in the place of human creativity, we strive to infuse our tools with deeper understanding so that they are more intuitive and inspiring."
N Synth, a tool that allows you to essentially interweave two instruments and generate a new sound, is likely a little above my pay grade. The same goes for the goods available to anyone who really wants to tap into Magenta's work—the team releases all their tools and models in open source on GitHub. For my purposes, I instead stuck to two of Magenta's available prebuilt tools: Infinite Drum and AI Duet.
Infinite Drum is essentially a drum machine composed of everyday sounds that were organized through machine learning. "The computer wasn’t given any descriptions or tags—only the audio," the GitHub description reads. "Using a technique called t-SNE, the computer placed similar sounds closer together. You can use the map to explore neighborhoods of similar sounds and even make beats using the drum sequencer."
From a user perspective, the tool becomes incredibly easy to use. You scroll around or search for a general category of sound, ultimately settling on four to combine. You can choose to shuffle things up or change the tempo, but otherwise you simply press play from there to get a looped beat.
AI Duet, on the other hand, will help you craft a perfect melody. Tap away however you'd like with whatever skills you might have, and Google machine learning takes over to respond with a sensible melody. The AI leverages machine learning by ingesting tons of compositions, and then its neural network responds to your individual inputs (including key and rhythm). As developer Yotam Mann puts it, "It's even fun to just mash the keyboard. The neural network tries to deliver something coherent from whatever you give it."
Like the Chrome Music Labs offerings, these individual tools function more as singular instruments than full composition platforms at the moment. So coming up with a final product—at the 2:55 mark, you can hear my Homeland-inspired avant-garde jazz closing track—once again requires a separate audio editing and capture program.
If the tools above sound too complicated, fear not. As Google pushes forward with Magenta and Sony puts its AI-powered FlowMachine in the hands of real musicians, there is a true AI composition option out there that's available right now to the rest of us: Amper AI.
Amper may not be the first publicly available AI composition tool (the British company behind Jukebox dates back to 2015, for instance), but it's the best combination of ease, customization, and ultimate quality. Amper made some headlines last year when it teamed up with YouTuber Taryn Southern for a single that felt indistinguishable from what you'd hear on Top 40 radio. Now, Southern will even release an entirely AI-driven album, called I Am AI, in May. Different songs will leverage Amper, IBM Watson, Aiva, and Google's Magenta.
“Our ultimate goal is to write music as well as John Williams [and] have it sound like it was recorded at Abbey Road Studios and produced by Quincy Jones,” Amper CEO Drew Silverstein told me back in December. "That’s the musical standard we hold ourselves to, but admittedly we’re not there. There’s a lot of work to be done on the music side of the project before Amper’s product is indistinguishable from human music, at which point it’d be the most valuable tool it could be for creators."
Musicians, not programmers, started Amper the company, so Silverstein and co. view their AI ultimately as a tool and not a replacement (hence the collaboration with Southern rather than promoting a single entirely of the bot's work). They trained their AI on the work of real-world composers much like how AlphaGo learned through observing real-world matches. The result is a tool that, even in its simplified beta iteration, offers a lot of customization while generating almost fool-proof usable results.
In fact, Amper currently has two separate interfaces to choose from: Simple or Pro. The former merely asks you to choose from some pre-set stylistic options and set a duration. The latter allows you to begin with similar stylistic parameters, but you can dig down to change seemingly everything: the number of instruments involved, the type of instruments involved, the tone of instruments involved, the overall key, the overall time signature, etc.
Roughly 30 seconds into my video, you can hear what Amper's simple tool does when instructed to craft "modern folk." And around the 2:00 mark, you experience my inspirational cinematic composition created via Amper Pro in order to score a montage of my dog walking. I may have envisioned "Gonna Fly Now"-style brass lines, but realistic brass sounds are something that eludes all these tools at the moment. Still, this particular composition swells and has soaring strings I can live with.
Am I going to win a best original song Oscar any time soon thanks to new AI instruments? Probably not. But even in the early stages, these tech-y layman's composition tools work. They're easy enough that I'm not spending all day on 30 seconds of sound, and I don't need to understand a lick of music theory. Yet, these tools can do a passable enough job to wash away any regrets I have that I can't get Vagabon to soundtrack my next Ernie short. Bands like that will likely always get called before the bots for projects of substance, but things for the weekend auteurs will only get better from here.