How AI Music Generation Works

Speech is one voice saying words in order. A song is a drum kit, a bass line, chords, a melody and a singer, all locked to a beat, repeating with variations over three or four minutes. A model has to get the millisecond right and the minute right at the same time. That double job explains most of what AI music does well, and most of how it fails.

Three research milestones

The commercial companies keep their architectures private. Published research shows the main ideas they build on.

System	Year	Idea	What it showed
Jukebox (OpenAI)	2020	Compress raw audio into discrete codes at several levels of detail, then predict codes with transformers, conditioned on genre, artist and lyrics	Songs with singing could be generated from raw audio at all. Slow and rough, but proof of concept.
MusicLM (Google)	2023	Treat music generation as a hierarchy of sequence-to-sequence steps from text to audio at 24 kHz	Music consistent over several minutes from a text description, and melody conditioning from humming or whistling. Released MusicCaps, 5,500 music-text pairs.
MusicGen (Meta)	2023	One transformer predicting several interleaved streams of codec tokens, conditioned on text or melody	A single-stage model could match cascades, in mono and stereo, with open code and weights.

Some other systems use diffusion, the noise-removal method that image generators use, on a compressed form of the audio. The two approaches are converging, and for a user the difference matters less than three shared facts. They learn from huge amounts of recorded music. They are steered by text and sometimes audio. And they produce an average of what they learned that fits your prompt.

From prompt to song

Lyrics add a hard sub-problem. The model must assign each syllable to a note it is composing in the same breath, so stress patterns of the words and the beat have to agree. When they do not, you hear the familiar tells: a word stretched over three notes, a syllable swallowed, the stress on "re-PORT" landing as "RE-port."

Section tags like [Verse] and [Chorus] help because they give the model a map. A model trained on millions of songs has a strong prior for what a chorus does: lift in energy, repeat the hook. Tags let you call up that prior where you want it.

Where the artifacts come from

Artifact	Likely cause	Fix
Tiny ticks and clicks	Decoding glitches; SoundStudio's theory page names Suno among sources of tiny broadband ticks	SoundStudio click finder, Suno ticks setting (lesson 6)
Swishy, smeared cymbals	Heavy compression of high frequencies in the codec	Gentle EQ; regenerate if severe
Mumbled or wrong lyrics	Syllable-to-note alignment failure	Rewrite lines with clearer stress; region edit
Song that wanders or ends abruptly	Long-range structure is the hardest part	Explicit section tags; extend or crop
Generic sound	The model outputs a statistical middle of the genre	More specific instrumentation and production words

The main services as of October 2026

Service	What the provider says now	Downloads and rights
Suno	v6 family (v6, v6-wild, v6-mini) in its help center as of September 2026; up to 8 minutes per generation; Suno says v6 is the first model released in partnership with artists and producers including Warner Music Group and BMG. Stem separation into up to 12 stems on paid plans.	Free accounts get up to 7 lifetime trial downloads for personal use. Pro gets 20 downloads a month, Premier 60. Songs made while subscribed carry commercial use rights.
Udio	Settled with Universal in October 2025 and with Warner in November 2025, and committed to licensed models.	Per a September 2026 trade summary, downloads have been off on every plan since late 2025, pending a licensed relaunch built with Universal that had no confirmed date.
Mureka	The model list in Jeff's MusicStudio shows mureka-9.5 as newest, plus mureka-9, mureka-8, mureka-o2 and mureka-7.6.	Mureka's FAQ says it is subscription-only and that subscriptions include full commercial usage rights; free users get daily credits for a few songs on an older model.
ElevenLabs Music	Launched August 2025, made "in collaboration with labels, publishers, and artists"; tracks from 3 seconds to 5 minutes, with section-level editing.	ElevenLabs says it is cleared for nearly all commercial uses, with terms varying by plan; the API is for paid subscribers.

Treat every line of that table as perishable. In twelve months this market went through lawsuits, settlements, a model generation and new download rules. Check the provider's help center the same week you release anything or repeat a figure from it.

The training-data fight, fairly

Every one of these models learned from recorded music. The dispute is over whose music, how it was obtained and whether permission was needed.

The labels' position. Universal, Sony and Warner sued Suno and Udio in June 2024, arguing that copying millions of recordings to build a product that competes with those recordings is infringement. In the Suno case, in federal court in Massachusetts, the judge in August 2026 let the labels add a claim that Suno got recordings by ripping them from YouTube, circumventing its protections. Suno's September 2026 answer, as reported by Music Business Worldwide, admits obtaining YouTube audio with a tool called yt-dlp and training on "tens of millions" of recordings.

The companies' position. Suno and Udio argued from their first answers in August 2024 that training is fair use: the model learns patterns, the copying is an internal step, and the output is new music that does not reproduce the recordings. Suno also notes, per the same report, that the labels do not appear to claim its outputs copy the recordings at issue.

The Copyright Office's view. Its May 2025 pre-publication report on training, still marked pre-publication as of October 2026, said many training uses are likely transformative, and that fairness depends on what works were used, from what source, for what purpose and with what output controls. It also said that commercial use of vast troves of works to produce content competing with them in existing markets, especially through illegal access, goes beyond established fair use boundaries. That is the Office's reading, not a court ruling.

Where it stands. No court had ruled on fair use in a music-generation case as of October 2026. The settlements point in one direction in practice: the companies that settled moved to licensed models, and Suno describes v6 as built in partnership with Warner and BMG. Lesson 8 has the case-by-case status.