While we tinker around with AI tools to best understand how we can use them to our benefit, the new Google Lumiere AI model just added another realm of possibility to our user experience. As we’ve grown more comfortable with chatbots and text-based interactions, the Google AI video generator has been working or refining the more complex video experience we can get with artificial intelligence, through a new diffusion model that creates brief but highly customized videos. Announced on 23 January 2024, the Google Lumiere release date was merely a reveal of the tool rather than a release for public use. The AI is reportedly capable of generating five-second 1024×1024 pixel videos that are low-resolution but quite realistic if their announcement video is to be believed.
Image – Google
After the much-awaited and slightly controversial release of the Google Gemini AI chatbot, the company appears to be pacing itself with its AI launches without allowing too much time in between, just to ensure no one forgets that it is still very actively a player in the game.
What is the Google Lumiere AI Model?
With the increasingly narrowed attention spans we have these days, it’s no wonder we aren’t tired of the multiple AI resources that keep dropping every few months. Even though we don’t have a public Google Lumiere release date, the technology looks quite efficient so far. This assessment is based on the multiple videos the company dropped along with its research paper and official announcement. The Google Lumiere AI model is slightly different from its competitors, with its Space-TIme U-Net architecture. What this means is that while other video models create distant keyframes first and then enhance the temporal details, the Google AI video generator does it all in one pass.
Other AI video generation models define key points or keyframes in a video, and these are used as anchors for the rest of the imagery to be filled in between. The temporal details refer to the changes that occur over time in the video, with the subtle shifts that take you from one frame to another—the elements that add life and fluid movement to the generated video. In the Google Lumiere AI model, both elements are considered in one go instead of setting anchor points first and filling in the gaps. It looks at the big picture as a whole instead of working in parts, which reportedly allows it to improve the quality of the final generated video. Using spatial and temporal down- and up-scaling, Google Lumiere plays around with the resolution and is able to “directly generate a full-frame-rate, low-resolution video by processing it in multiple space-time scales.”
Image – Google
The test-to-image diffusion model is pre-trained to turn simple prompts into realistic videos or even edit existing video samples according to the input prompts. The Google AI video generator release note was paired with a huge collection of samples of what the AI tool can do and the myriad ways it can make the most of your prompts.
- Text-to-Video: Turn your prompts into very realistic elements that look like they were shot on camera. Some are more realistic than others but the quality overall is quite impressive
- Image-to-Video: Turns a still image into a moving video based on the prompt you provide with it (we’re personally obsessed with how famous paintings like The Girl with a Pearl Earring and The Great Wave off Kanagawa were brought to life)
- Stylised Generation: Images can be used as a style reference, such as “line drawing, 3D rendering, or watercolor painting” to generate more videos along the same lines, bringing uniformity to your projects with ease
- Video Stylization: Source videos paired with an input prompt can be used to edit elements in the video in a specified style, such as converting a car in a video to be made of “Origami folded paper art, colorful toy bricks, or flowers”
- Cinemagraphs: The Google Lumiere AI model allows you to add an image and mask a section with a specific instruction for animation while the rest of the image stays still
- Video Inpainting: By selecting specific areas of the video, you can add instructions for elements you want to alter or add to the video, such as adding sunglasses to the video of an owl
Image – Google (Image-to-video feature)
Is Google Lumiere Free?
The features of the Google Lumiere AI model appear quite interesting to experiment with and could mean a whole new era of video production and editing but there is no news currently on how the company is planning to release access to the tool. “Is Google Lumiere free?” Well, we don’t know yet as the company does not have a trial model out or any plans to release the AI tool anytime soon. While the Google Gemini AI chatbot was available via free access plans on the Google Vertex AI Studio soon after the announcement, there are no indicators if Google Lumiere will follow the same path.
Image – Google (The source video on the left is edited with multiple different prompts)
Google has already experimented with text-to-video AI tools before, its Imagen Video offering a pretty decent starting point for the company when Meta’s Make-A-Video AI editor was first announced back in 2022. The latest tool does seem to be a step up from the previous iteration which shows just how quickly—or slowly depending on how impressed you are by the tool—technology is evolving.
👀 I think #Google is going to hit it big with #Lumiere #AI is evolving so fast 🚀 I can’t wait, and you? 📽️🤖 pic.twitter.com/LeVDovzArG
— KillerOfSupply (@KillerOfSupply) January 30, 2024
“Our primary goal in this work is to enable novice users to generate visual content in a creative and flexible way,” the company states and if the tool actually works as effectively as the video suggests, the company appears well on its way to reaching that goal. Google also acknowledges the safety issues that come with such a tool and we’re quite wary of the innumerable ways in which this kind of video generative technology can be misused. There is no dearth of people who are quite willing to both exploit the vulnerabilities of said tech and exploit other people just as quickly. We’ll have to see how the landscape of AI continues to evolve and the kind of checks that can potentially be set in place to ensure that these tools are used safely without risking harm to the population at large.