Music models have one annoying property: either the result works, or you throw it away and generate again. There is nothing to fix, because between your lyrics and the finished recording there is nothing you can reach into. YuE2 adds that missing step — and it is the most interesting thing I have seen in open music generation this year.
It writes the score first, then sings
YuE2 is an open model from the M·A·P group (people from HKUST, NYU, Stanford and other institutions are behind it). You give it lyrics and one sentence about the style — something like "English, jazz-funk, warm lead vocal, Rhodes, bass and drums" — and it returns a complete song with vocals and accompaniment in 48 kHz stereo.
The difference is what happens in between. First it generates a plan of the song: melody and chords in ABC notation. That is plain text you can read, play back and rewrite — and only then is the recording sung from it. The authors call it white-box generation, and for once that label fits.
One switch controls it:
cot="full"— the model plans melody and chords, the default for a new song,cot="melody"— it keeps the melody and invents the accompaniment freely, recommended for covers,cot="off"— straight from lyrics and style, like other models,abc=…— you supply your own plan.
That last line is where the magic is. Export the plan, change its harmony, tempo or form (by hand, or let an agent rewrite it) and run the generation again. Just keep in mind that editing produces a completely new recording — the original audio outside the edited part is not preserved.
What it sounds like
This is a complete song from YuE2 3B — nothing edited, a single run exactly as it came out of the machine:
The file still carries the full ComfyUI request, lyrics and seed 41688042516701 included, so you can take it apart and repeat it.
Covers from a recording, with no training
Add the second model from the same workshop, SheetSage2, and YuE2 can transcribe an existing recording into a score and then play it in a different style. The numbers the authors measured on 948 works make the point: with a score, the cover holds the melody at 0.647 CLEWS mAP, without a score at 0.006. That is the difference between "the same song done differently" and "something else entirely".
How it compares to Suno
The authors published measurements on the WildSongBench set (192 prompts, measured 12 September 2026). The rows worth your attention:
- YuE2 (best-of-8) — 6.9632 (the highest mean in the whole table)
- Mureka 9 — 6.9377
- Suno v5 — 6.8721
- YuE2 (standard setting) — 6.7316
- Suno v6 — 6.5562
- ACE-Step 1.5 — 6.0118
- YuE 1 (last year's version of this family) — 4.9165
Two notes on that. First, "best-of-8" means the model generates eight candidates and the best one is picked — the competition is scored the same way, but on your own machine it means eight times the waiting. Second, the authors themselves write that the gaps between the top systems are too small to be statistically significant. The way I read it: an open model you download and run at home has reached the level of a paid service. That is the news, not a few hundredths in a table.
What hardware it takes
The official route from the repository is honestly documented and demanding: Linux, Python 3.12 and an NVIDIA card with 24 GB of VRAM and BF16 support. Model files download from Hugging Face on first use and the output is unquantized.
For an ordinary computer the second route matters more. ComfyUI supports YuE2 natively from version 0.36 (released 15 September 2026) — no third-party nodes to install. It uses a quantized int8 build; the checkpoint yue2_3b_int8_convrot.safetensors is 3.9 GB and belongs in models/checkpoints. That shrinks the 24 GB requirement down to a card plenty of people already own. Mine runs on an RTX 4060 Ti with 16 GB (Intel Core Ultra 7 265K), and I timed it: a four-minute track (240 seconds) is done in about two minutes — faster than it takes to listen to it. The first run is slower, because the model has to load first.
I added it to PocketComfy
PocketComfy is my Android app that turns your phone into a remote control for your own ComfyUI at home. The Music card has been in it from the start, but a single model ran underneath it. From version 3.48 you choose:
- ACE-Step 1.5 Turbo — fast (eight steps) and it sings Czech. You set the length, tempo, key and singing language yourself.
- YuE2 3B — more musical, because it writes the score first. It was trained on English and Chinese, but as the demo above shows, it sings Czech too. A run takes considerably longer.
For each model the card shows only the fields that model actually uses. YuE2 takes no tempo, key or language on input — it decides those from the style and the lyrics. What it does have is the plan setting (no plan / melody / melody and chords), and the length is only a ceiling: the track ends where the song ends, not where the timer runs out.
Where it hurts
Licence. This is the most important sentence in the article. The code and documentation are Apache 2.0, but the model weights are CC BY-NC 4.0 — non-commercial only. You can compose whatever you like at home, but you cannot put YuE2 music into a client's ad, into a podcast carrying ads, or into a video on a monetized channel. When you need music you can sell, reach for ACE-Step (the whole repository is Apache 2.0) or for a paid service where commercial use is covered by contract.
Time. Planning and singing are two passes through the model, so compared to the eight-step ACE-Step Turbo you wait longer — two minutes against a few dozen seconds on my machine. For one song you will not throw away it is worth it; for thirty attempts in an afternoon it is not.
Where to start
Do not start with your own lyrics. Let the model make one song with a melody-and-chords plan, open the generated score.abc file and play it back — even if you do not read music, you will see immediately whether the model understood what you asked for. If the plan is wrong, there is no point rendering audio; fix the plan. That is the whole difference from generators where all you can do is press "again".
The app with the Music card is on GitHub (PocketComfy), the model itself lives in the multimodal-art-projection/YuE repository.
