OpenAI introduces Sora, its text-to-video AI model

*The Deadman* · 02-16-2024, 10:10 PM

https://www.theverge.com/2024/2/15/2...xt-to-video-ai

OpenAI is launching a new video-generation model, and it’s called Sora. The AI company says Sora “can create realistic and imaginative scenes from text instructions.” The text-to-video model allows users to create photorealistic videos up to a minute long — all based on prompts they’ve written.

Sora is capable of creating “complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background,” according to OpenAI’s introductory blog post. The company also notes that the model can understand how objects “exist in the physical world,” as well as “accurately interpret props and generate compelling characters that express vibrant emotions.”

The model can also generate a video based on a still image, as well as fill in missing frames on an existing video or extend it. The Sora-generated demos included in OpenAI’s blog post include an aerial scene of California during the gold rush, a video that looks as if it were shot from the inside of a Tokyo train, and others. Many have some telltale signs of AI — like a suspiciously moving floor in a video of a museum — and OpenAI says the model “may struggle with accurately simulating the physics of a complex scene,” but the results are overall pretty impressive.

A couple of years ago, it was text-to-image generators like Midjourney that were at the forefront of models’ ability to turn words into images. But recently, video has begun to improve at a remarkable pace: companies like Runway and Pika have shown impressive text-to-video models of their own, and Google’s Lumiere figures to be one of OpenAI’s primary competitors in this space, too. Similar to Sora, Lumiere gives users text-to-video tools and also lets them create videos from a still image.

Sora is currently only available to “red teamers” who are assessing the model for potential harms and risks. OpenAI is also offering access to some visual artists, designers, and filmmakers to get feedback. It notes that the existing model might not accurately simulate the physics of a complex scene and may not properly interpret certain instances of cause and effect.

Earlier this month, OpenAI announced it’s adding watermarks to its text-to-image tool DALL-E 3, but notes that they can “easily be removed.” Like its other AI products, OpenAI will have to contend with the consequences of fake, AI photorealistic videos being mistaken for the real thing.

*IMJ* · 02-17-2024, 10:40 AM

So I've messed around with Open A.I. a bit and I love it. But I don't understand how some of the stuff I've seen online is getting out there - the versions I use are locked or won't do certain things.

For example, a guy made an unreal and awesome mashup of Alice In Chains "Nutshell" but used A.I. to make it sound like Stone Temple Pilots Scott Weiland was singing it in his "Core"-era vocals. It was amazing. I've also seen people making cool little video clips (like two pirate ships fighting inside a cup of coffee).

How do you get access to do stuff like that and keep it or download it or use it?

*IndigoErth* · 02-17-2024, 12:22 PM

I'm not sure how I feel about this, sounds a lot more risky to blurring reality and causing trouble with people gullible enough to believe things, or the equivalent of yelling "fire" to purposely freak people out with fake "events."

Though I look forward to someday using a VR headset with this capability to hang with the Turtles, that is all. (Creating sharable VR things like that could actually be a pretty fun fan thing.)

02-16-2024, 10:10 PM	#1
The Deadman Foot Elite Join Date: Aug 2009 Location: Maryland Posts: 4,001	OpenAI introduces Sora, its text-to-video AI model https://www.theverge.com/2024/2/15/2...xt-to-video-ai OpenAI is launching a new video-generation model, and it’s called Sora. The AI company says Sora “can create realistic and imaginative scenes from text instructions.” The text-to-video model allows users to create photorealistic videos up to a minute long — all based on prompts they’ve written. Sora is capable of creating “complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background,” according to OpenAI’s introductory blog post. The company also notes that the model can understand how objects “exist in the physical world,” as well as “accurately interpret props and generate compelling characters that express vibrant emotions.” The model can also generate a video based on a still image, as well as fill in missing frames on an existing video or extend it. The Sora-generated demos included in OpenAI’s blog post include an aerial scene of California during the gold rush, a video that looks as if it were shot from the inside of a Tokyo train, and others. Many have some telltale signs of AI — like a suspiciously moving floor in a video of a museum — and OpenAI says the model “may struggle with accurately simulating the physics of a complex scene,” but the results are overall pretty impressive. A couple of years ago, it was text-to-image generators like Midjourney that were at the forefront of models’ ability to turn words into images. But recently, video has begun to improve at a remarkable pace: companies like Runway and Pika have shown impressive text-to-video models of their own, and Google’s Lumiere figures to be one of OpenAI’s primary competitors in this space, too. Similar to Sora, Lumiere gives users text-to-video tools and also lets them create videos from a still image. Sora is currently only available to “red teamers” who are assessing the model for potential harms and risks. OpenAI is also offering access to some visual artists, designers, and filmmakers to get feedback. It notes that the existing model might not accurately simulate the physics of a complex scene and may not properly interpret certain instances of cause and effect. Earlier this month, OpenAI announced it’s adding watermarks to its text-to-image tool DALL-E 3, but notes that they can “easily be removed.” Like its other AI products, OpenAI will have to contend with the consequences of fake, AI photorealistic videos being mistaken for the real thing. __________________ Last Movie Watched: Infested (2023). Last TV Show Watched: Ghosts US (S1:E10).

02-17-2024, 10:40 AM	#2
IMJ Emperor Join Date: Aug 2006 Location: Midwest, U.S.A. Posts: 6,997	So I've messed around with Open A.I. a bit and I love it. But I don't understand how some of the stuff I've seen online is getting out there - the versions I use are locked or won't do certain things. For example, a guy made an unreal and awesome mashup of Alice In Chains "Nutshell" but used A.I. to make it sound like Stone Temple Pilots Scott Weiland was singing it in his "Core"-era vocals. It was amazing. I've also seen people making cool little video clips (like two pirate ships fighting inside a cup of coffee). How do you get access to do stuff like that and keep it or download it or use it? __________________ Wave 1 Collectibles Online Market Watch Blog Collection Photography

02-17-2024, 12:22 PM	#3
IndigoErth Team Blue Boy Join Date: Jun 2014 Location: U.S., East Coast Posts: 15,252	I'm not sure how I feel about this, sounds a lot more risky to blurring reality and causing trouble with people gullible enough to believe things, or the equivalent of yelling "fire" to purposely freak people out with fake "events." Though I look forward to someday using a VR headset with this capability to hang with the Turtles, that is all. (Creating sharable VR things like that could actually be a pretty fun fan thing.)