If you’ve been away from the internet for a few weeks, you might find yourself wondering, “What is OpenAI Sora and are we already over ChatGPT?” It’s a fair question considering the constant flood of chatbot names. Each AI isn’t drastically different from previous versions but they still carry some element of unfamiliarity. The OpenAI Sora text-to-video generator is the company’s next step in its vision for AI, not intended to replace ChatGPT but instead to work as a companion tool. Learning how to use OpenAI’s Sora tool will require you to take the prompt generation skills you’ve already developed and put it to good use on a different medium but it will be a while before you get to try it out.
The OpenAI Sora release date has been a matter of great discussion and while for a brief period it felt like we would not see a public release this year, the plans have changed at the San Francisco company. While we wait for more details, it can’t hurt to know a little more about what OpenAI’s Sora is and how it works.
Image: Pexels
What is OpenAI Sora?
The first time the OpenAI Sora release date was discussed was back in February when they released an in-depth introduction to the AI. The OpenAI Sora text-to-video generator is a service that takes a basic text prompt and invents a minute-long video from the ideas shared with it. These videos range from being hyperrealistic and life-like to showcasing a more animated design format, but one thing that remains constant is the quality of the videos. To help the world at large find answers to their pressing questions on “What is OpenAI’s Sora?,” the company released a stream of videos to celebrate their next advancement in AI.
The OpenAI Sora text-to-video generator not only interprets the prompt it is provided with, but it also understands the physical nature of the generated object and its place in the real world, making it a very efficient tool to have on hand. Additionally, it is also able to take a pre-existing image and turn it into a video based on a text prompt. It isn’t perfect, however, and there are still gaps in what it can do. The closest we saw a company come to being an efficient text-to-video generator before OpenAI’s Sora was Google’s Lumiere tool and even that hasn’t seen a public release yet.
Is Sora AI Open to the Public?
If you’re wondering how to use OpenAI Sora and test the AI out for yourself, then you might not have to wait for much longer. After the initial announcement in February, the Sora tool was only released for red teamers who were asked to put the AI to work and test its security limitations and technical aspects before a public release. The OpenAI Sora text-to-video generator was also shared with a few visual artists, designers, and filmmakers who could test the creative capabilities and limitations of the AI. These limited releases were intended to help the team get an outsider’s perspective on the product they had sewn together so far.
The wait has been a brief one and we will likely only have a few more months left before the OpenAI Sora release date in 2024. OpenAI Chief Technology Officer Mira Murati told The Wall Street Journal that there was discussion of a public release of the Sora AI in a few months, possibly sometime this year. While it’s still a vague, non-commital promise that could amount to nothing, we’re hoping it’s a reliable estimate of when we will see the AI.
How Does OpenAI Sora Work?
An in-depth understanding of how OpenAI works will be hard to come by as there are some trade secrets that the company will keep close to its heart but it has been forthcoming about the major research points behind the AI. The AI relies on a diffusion model that uses static noise in a frame and converts these into images that make up the collection of video frames that make up the final results. It uses the visual patches of noise and reshapes them into the approximate structure of what is requested in the prompt. These shapes are refined further until a clear image with the necessary elements that the AI learned about is present in the image. The company’s text-conditional diffusion models are trained on videos and images with varying features and the AI “learns” how to take elements of these to devise into a final video.
According to The Verge, one of the sources for these training videos is the licensed Shutterstock content that the company has access to, but there could be many other data sets that help Sora learn things like what a helicopter is or where the wheels go on a bus. CEO Sam Altman has been encouraging users on Twitter/X to reply to him with prompts for testing Sora’s capabilities and the results are quite fascinating.
https://t.co/rmk9zI0oqO pic.twitter.com/WanFKOzdIw
— Sam Altman (@sama) February 15, 2024
https://t.co/qbj02M4ng8 pic.twitter.com/EvngqF2ZIX
— Sam Altman (@sama) February 15, 2024
What are the Advantages of Sora?
If you’ve understood what OpenAI’s Sora model is, then you might be able to envision many applications for such technology already. The quality of the videos designed by the OpenAI Sora text-to-video generator is already impressive, and will likely get even more refined with future updates. From content creators to filmmakers, all those who work with video content could now have a way to add elements to their videos that they might find hard to film. It could also help avoid unnecessary expenses. The video generator is also capable of adapting videos to suit the design aesthetic and format of the user, expanding on the potential applications of the AI.
Even with the brief one-minute length of these videos, it could be so much easier for a video maker to find the perfect transition shot or create a background for the story they’re working on. If refined, the potential applications of AI would only expand even further. With the option of longer videos, videographers would be more incentivized to use AI for creating their videos instead of getting a whole filming crew to record them. OpenAI CTO Mira Murati also revealed that they have been working out how to add audio to the videos, which will only make it more invaluable when it is executed successfully
What are the Limitations of Sora?
While learning how to use OpenAI Sora appears to be quite an interesting prospect, it is also for the best that the AI isn’t perfect yet. Despite the extensive training, some elements of object interaction still appear to escape the AI, which can result in uncanny, odd videos every once in a while. Due to certain limitations of the OpenAI Sora text-to-video generator, physics seems to be one aspect of the application that will require further training.
For example, when prompted “In an ornate, historical hall, a massive tidal wave peaks and begins to crash. Two surfers, seizing the moment, skillfully navigate the face of the wave,” the AI is able to generate quite a realistic video of what that would look like, but during the last few seconds, one of the surfers defies the laws of physics to skirt the side of the wave very unrealistically.
Even when asked to generate videos of people walking, some turn out quite beautifully while others feature the subject walking with an unnatural gait or hovering over the ground. These elements show that the AI is not perfect and that it has some ways to go before it fully learns how to give the user exactly what they are looking for without compromising on any aspect of the device.
Security and Privacy Concerns Abound
Considering how OpenAI Sora works, there are extensive concerns about how this technology might be misused by people to spread misinformation or generate false content that can be circulated as the truth. With how easy it seems to be to make content go viral, once a video is put out into the world, it can become impossible to let each person who sees it also learn that the video is fake. Security and privacy concerns abound and while there is some talk of a watermark to indicate that it is AI-generated content, it is easy enough to get rid of such marks even without AI. We’ll have to wait and see how OpenAI plans to tackle these problems and what solutions they come up with before release.
Now is OpenAI Sora going to be free? Highly unlikely considering that the company will probably try to limit who has access to it to a degree. Especially considering the high costs of designing and running such a demanding AI, it is unlikely that the tool will be free even after a public release.
