Stable Audio Open published by Stability AI as an open source text-to-audio generator

Stability AI has released an open source artificial intelligence (AI) model called Stable Audio Open. Users can use the model to generate up to 47 seconds of samples and sound effects. Users can use it to create samples of musical instruments or ambient sounds. The AI ​​model also allows users to generate different variations and styles of a previously generated pattern. The open-source model is separate from AI’s Stable Audio platform, which allows users to generate tracks up to three minutes long, and is only available as part of a subscription.

Stability AI is published by Stable Audio Open

Stable Audio Open works similarly to many available AI models on the market. Users can write a text query for a sample or sound effect and it will generate up to 47 seconds of sound. AI stability mentioned in a news post that the AI ​​model has been released in open source to empower sound designers, musicians and creative communities.

However, it has restricted the use of Stable Audio Open to research and non-commercial use. In order to obtain commercial rights, users will need to purchase a Stability AI membership.

In terms of features, it can generate drum beats, instrument riffs, ambient sounds, foley recordings and other audio samples. Furthermore, users can also fine-tune the model using their custom audio data.

Giving an example, the AI ​​company says a drummer can train the AI ​​on recordings of his drumming and use the model to generate new beats. Although the model can generate short audio samples, it is not optimized for whole songs, melodies or vocals.

To train Stable Audio Open, the company used a dataset of 4,86,492 audio tracks from FreeSound and the Free Music Archive. It added: “We did due diligence to ensure there was no unauthorized copyrighted music in our training data before we started training.”

However, Stability AI also said that the data set lacks diversity and that not all cultures are equally represented. As a result, the generated samples will reflect biases from the training data. To access the AI ​​model, users can go to the company’s Hugging Face list, where the model’s open weights are currently located.