AI music generation model that synthesizes music with lyrics and vocals based on a reference track.
Model Overview
A powerful AI music generator that creates up to 60 seconds of music, complete with lyrics and vocals, in the style of a reference track you provide. It's designed to be intuitive for creatives, allowing you to guide the musical output with your own lyrics and sonic inspiration.
Best At
- Style transfer: Mimicking the musical style, instrumentation, and vocal qualities of a reference song.
- Lyric-to-music: Generating vocal melodies and accompanying music directly from your provided lyrics.
- Quick demos: Rapidly creating short musical pieces for soundtracks, AI singer compositions, or musical reinterpretations.
Limitations / Not Good At
- Output length: Currently limited to 60 seconds of audio. (3 minutes planned for the next release).
- Reference required: Needs a reference track (song, voice, or instrumental) to learn the desired style.
- Lyric length: Maximum of 350-400 characters for lyrics.
Ideal Use Cases
- Creating background music for videos or podcasts.
- Experimenting with different musical styles for song ideas.
- Generating unique vocal tracks for AI singer projects.
- Producing quick musical sketches or demos.
Input & Output Format
Input:
lyrics (string, optional): Your song lyrics, with newlines for line breaks and pauses.
song_file (audio, optional): A reference song (.wav or .mp3, >15s) for overall style.
voice_file (audio, optional): A reference voice (.wav or .mp3, >15s) for vocal style.
instrumental_file (audio, optional): A reference instrumental (.wav or .mp3, >15s) for accompaniment style.
voice_id (string, optional): Reuse a previously uploaded voice.
instrumental_id (string, optional): Reuse a previously uploaded instrumental.
sample_rate (integer, optional): Desired output sample rate.
bitrate (integer, optional): Desired output bitrate.
Output:
- A URI (string) pointing to the generated music file (MP3).
Performance Notes
- Generates up to 60 seconds of music quickly.
- Style learning is based on the provided reference track(s).
- Handles multiple genres including classical, pop, rock, and electronic.