Facebook owner Meta announced on Friday that it has built a new AI model called Movie Gen that can create realistic video and audio clips in response to user input, claiming it can rival tools from leading media generation startups such as OpenAI and ElevenLabs.
Samples of Movie Gen’s creations provided by the Meta featured videos of animals swimming and surfing, as well as videos that used actual photos of people to show them performing actions such as painting on canvas.
Movie Gen can also generate background music and sound effects in sync with video content, Meta said in a blog post, and use the tool to edit existing videos.
In one such video, Meta used a tool to insert pompoms into the hands of a man running alone in the desert, while in another she changed the parking lot where the man was skateboarding from dry ground to one covered in a puddle.
Meta’s Movie Gen AI model allows users to edit videos using text queries
Photo author: Meta
Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. He shared data showing blind tests showing the model performed favorably against offerings from startups including Runway, OpenAI, ElevenLabs and Kling.
The announcement comes as Hollywood has grappled with how to harness generative AI video technology this year, after Microsoft-backed OpenAI first showed in February how its Sora product could create feature-length videos in response to text queries.
Technologists in the entertainment industry are eager to use such tools to enhance and speed up moviemaking, while others worry about embracing systems that appear to be trained to copyright works without permission.
Lawmakers have also raised concerns about how artificial intelligence-generated forgeries, or deepfakes, are being used in elections around the world, including in the US, Pakistan, India and Indonesia.
Meta spokespeople said the company was unlikely to release Movie Gen for open use by developers, as it did with its Llama series of large-language models, saying it considered the risks individually for each model. They declined to comment on Meta’s assessment specifically for Movie Gen.
Instead, they said, Meta is working directly with the entertainment community and other content creators to use Movie Gen and will build it into Meta’s own products next year.
According to a blog post and research paper about the tool published by Meta, the company used a combination of licensed and publicly available datasets to create Movie Gen.
OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, though there are still no reports that any deals have come from those talks. Concerns about the company’s approach increased in May when actress Scarlett Johansson accused the maker of ChatGPT of mimicking her voice without permission for its chatbot.
Lions Gate Entertainment, the company behind “The Hunger Games” and “Twilight,” announced in September that it was giving AI startup Runway access to its film and television library to train AI models. In return, it says, the studio and its filmmakers can use the model to scale up their work.
© Thomson Reuters 2024
(This story was not edited by NDTV staff and was automatically generated from a syndicated feed.)
