Luma AI launches Dream Machine AI-powered text-to-video platform

Luma AI on Wednesday globally launched its artificial intelligence (AI) video text generation model, Dream Machine. The platform can generate videos of up to five seconds from simple or descriptive text queries. The AI ​​model can generate videos in different styles such as cinematic, animated, realistic, etc. The AI ​​company claims that Dream Machine is fully trained on videos and is capable of generating “physically accurate, consistent and meaningful footage”. The platform is currently free to access and use, but will likely have a daily generation limit.

Luma AI debuts Dream Machine

According to the website, the Dream Machine AI model is built on a transformer model and trained directly on videos. Typically, large language models (LLMs) are trained on text and images and then moved to videos because they require a deeper understanding of space and motion. “Dream Machine is our first step towards building a universal imagination engine,” the company added.

Dream Machine joins video generation platforms such as Runway AI and Pika 1.0 which are also available in the public domain and offer video generation between three to five seconds. Gadgets 360 tested the platform and found the platform’s fast adherence to be below par. It struggles with multiple characters or when the query is overwritten. However, compared to the other two, it is capable of generating higher quality cinematic videos.

The AI ​​platform takes 120 seconds to generate a video that the company claims will have 120 different frames. Dream Machine is also said to understand how people, animals and objects interact with the physical world and creates videos with accurate physics and character consistency.

However, Luma AI also highlighted several limitations in the current mode such as motion, text, morphing and the famous Janus problem, where instead of showing a consistent 3D output, the AI ​​model shows multiple canonical views of an object in different directions.

Luma AI’s Dream Machine creates a video about Stormtroopers
Photo credit: Luma AI/Dream Machine

Luma AI has not shared any technical details about the AI ​​model, so information on parameter size, benchmarks, architecture, and training methods is unknown. The company also did not share any details about the acquisition of training data. In fact, Gadgets 360 managed to generate several videos with copyrighted characters.

To try out the platform, enthusiasts can go to the website and click on the ‘Try Now’ button. Users will need to sign up for the platform before they can generate videos.