On September 25, 2024, Meta CEO Mark Zuckerberg delivered a keynote speech while holding a smartphone at the Meta Connect annual event held at the company’s headquarters in Menlo Park, California, USA.
Manuel Orbergozo | Reuters
facebook owner Yuan announced on Friday that it has built a new artificial intelligence model called Movie Gen that can create photorealistic video and audio clips based on user prompts, claiming it is comparable to tools from leading media generation startups such as OpenAI and ElevenLabs .
Samples of Movie Gen creations provided by Meta show videos of animals swimming and surfing, as well as videos that use real photos of people to depict them doing things like painting on canvas.
Meta said in a blog post that Movie Gen can also generate background music and sound effects synchronized with the movie content, and use the tool to edit existing movies.
In one such video, Mehta had the tool insert pompoms into the hands of a man running alone in the desert, while in another, it transformed a parking lot where a man was skateboarding from dry ground into into a parking lot that was splashed with puddles.
Meta says Movie Gen can create videos up to 16 seconds long and audio up to 45 seconds long. It shared data showing that blind tests showed the model outperformed products from startups such as Runway, OpenAI, ElevenLabs and Kling.
The news comes as Hollywood has been working hard this year on how to use generative artificial intelligence video technology. Microsoft– Supported by OpenAI, which first showed off in February how its product Sora can create movie-like videos based on text prompts.
Technicians in the entertainment industry are eager to use such tools to enhance and speed up film production, while others are wary of adopting systems trained on copyrighted works without permission.
Lawmakers also highlighted concerns about how artificial intelligence-generated forgeries, or deepfakes, could be used in elections around the world, including in the United States, Pakistan, India and Indonesia.
A Meta spokesperson said that the company is unlikely to release Movie Gen for open use by developers like the Llama series of large language models, and said that the risks of each model will be considered individually. They declined to comment specifically on Meta’s assessment of Movie Gen.
Instead, they said Meta is working directly with the entertainment community and other content creators on the use of Movie Gen and will incorporate it into Meta’s own products sometime next year.
According to a blog post published by Meta and a research paper on the tool, the company used licensed and publicly available datasets to build Movie Gen.
OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although those talks have reportedly yet to result in any agreement. Concerns about the company’s practices intensified in May when actor Scarlett Johansson accused the makers of ChatGPT of impersonating her voice for its chatbot without permission.
Lionsgate EntertainmentThe studio behind “The Hunger Games” and “The Twilight Saga” announced in September that it would allow artificial intelligence startup Runway to access its library of movies and TV shows to train artificial intelligence models. In return, studios and their filmmakers can use the model to enhance their work.
About The Author
