AudioCraft is Meta’s free model kit for text-to-audio tasks



summary
Summary

With Audiocraft, Meta releases three AI tools for music and audio generation for research purposes.

Audiocraft consists of Meta’s MusicGen, an AI model introduced in June 2023 that can generate melodies and musical pieces from text and other music. Also part of Audiocraft is AudioGen, a Transformer-based generative AI model introduced in October 2022 that can generate sounds to match text input from scratch or extend existing audio files.

Meta’s audio tokenizer EnCodec, which breaks audio files into smaller pieces for AI processing, is the third part of Audiocraft and is now available in an enhanced version that Meta says produces higher-quality music with fewer artifacts.

Model kit for AI audio experiments

According to Meta, the Audiocraft family of models can produce high-quality, consistent, and longer audio using only natural language interaction. The release provides full access to Meta’s research in generative audio AI over the past several years, according to the company.

Ad

“There are nearly limitless possibilities once you give people access to the models to tune them to their needs,” Meta writes.

With Audiocraft, musicians or sound designers, for example, would have professional tools for faster inspiration, brainstorming, or refining existing compositions.

MusicGen example: Earthy tones, environmentally conscious, ukulele-infused, harmonic, breezy, easygoing, organic instrumentation, gentle grooves


Audiogen example: Whistling with wind blowing

Generative audio to lower entry barriers to music and audio

The Meta research team continues to work on generative audio, specifically high-quality audio based on diffusion models, the same technique that has led to huge quality improvements in image generation.

Recommendation

Audiocraft’s code is available here.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top