Sam Altman singing in a toilet. James Bond playing Altman in high-stakes poker. Pikachu storming Normandy’s beaches. Mario jumping from his virtual world into real life.
Those are just some of the lifelike videos that are rocketing through the internet a day after OpenAI released Sora, an app at the intersection of social media and artificial intelligence-powered media generation. The app surged to be the most popular app in the iOS App Store’s Photo and Video category within a day of its release.
Powered by OpenAI’s upgraded Sora 2 media generation AI model, the app allows users to create high-definition videos from simple text prompts. After it processes one-time video and audio recordings of users’ likenesses, Sora allows users to embed lifelike “cameos” of themselves, their friends and others who give their permission.
The app is a recipe made for virality. But many of the videos published within the first day of Sora’s debut have also raised alarm bells from copyright and deepfake experts.
Users have so far reported being able to feature video game characters like Lara Croft or Nintendo heavyweights like Mario, Luigi and even Princess Peach in their AI creations.
One user inserted Ronald McDonald into a saucy scene from the romantic reality TV show “Love Island.”
The Wall Street Journal reported Monday that the app would enable users to feature material protected by copyright unless the copyright holders opted out of having their work appear.
However, the report said, blanket opt-outs did not appear to be an option, instead requiring copyright holders to submit examples of offending content.
Sora 2 builds on OpenAI’s original Sora model, which was released to the public in December. Unlike the original Sora, Sora 2 now enables users to create videos with matching dialogue and sound effects.
AI models ingest large swaths of information in the “training” process as they learn how to respond to users’ queries. That data forms the basis for models’ responses to future user requests. For example, Google’s Veo 3 video generation model was trained on YouTube videos, much to the dismay of some YouTube creators.
OpenAI has not clearly indicated which exact data its models draw from, but the appearance of characters under copyright indicates that it used copyright-protected information to design the Sora 2 system. China’s ByteDance and its Seedance video generation model have also attracted recent copyright scrutiny.
OpenAI faces legal action over copyright infringement claims, including a high-profile lawsuit featuring authors including Ta-Nehisi Coates and Jodi Picoult and newspapers like The New York Times. OpenAI competitor Anthropic recently agreed to pay $1.5 billion to settle claims from authors who alleged that Anthropic illegally downloaded and used their books to train its AI models.
In an interview, Mark McKenna, a law professor and the faculty director of the UCLA Institute for Technology, Law, and Policy, drew a stark line between using copyrighted data as an input to train models and generating outputs that depict copyright-protected information.
“If OpenAI is taking an aggressive approach that says they’re going to allow outputs of your copyright-protected material unless you opt out, that strikes me as not likely to work. That’s not how copyright law works. You don’t have to opt out of somebody else’s rules,” McKenna said.
“The early indications show that training AI models on legitimately acquired copyright material can be considered fair use. There’s a very different question about the outputs of these systems,” he continued. “Outputting visual material is a harder copyright question than just the training of models.”
As McKenna sees it, that approach is a calculated risk. “The opt-out is clearly a ‘move fast and break things’ mindset,” he said. “And the aggressive response by some of the studios is ‘No, we’re not going to go along with that.’”
Disney, Warner Bros. and Sony Music Entertainment did not reply to requests for comment.
In addition to copyright issues, some observers were unsettled by one of the most popular first-day creations, which depicted OpenAI CEO Sam Altman stealing valuable computer components from Target — illustrating the ease with which Sora 2 can create content depicting real people committing crimes they had not actually committed.
Sora 2’s high-quality outputs arrive as some have expressed concerns about illicit or harmful creations, from worries about gory scenes and child safety to the model’s role in spreading deepfakes.
OpenAI includes techniques to indicate Sora 2’s creations are AI-generated as concerns grow about the ever-blurrier line between reality and computer-generated content.
Sora 2 will include moving watermarks on all videos on the Sora app or downloaded from sora.com, while invisible metadata will indicate Sora-generated videos are created by AI systems.
However, the metadata can be easily removed. OpenAI’s own documentation says the metadata approach “is not a silver bullet to address issues of provenance. It can easily be removed either accidentally or intentionally,” like when users upload images to social media websites.
Siwei Lyu, a professor of computer science and the director of the University of Buffalo’s Media Forensic Lab and Center for Information Integrity, agreed that multiple layers of authentication were key to prove content’s origin from Sora.
“OpenAI claimed they have other responsible use measures, such as the inclusion of visible and invisible watermarks, and tracing tools for Sora-made images and audio. These complement the metadata and provide an additional layer of protection,” Lyu said.
“However, their effectiveness requires additional testing. The invisible watermark and tracing tools can only be tested internally, so it is hard to judge how well they work at this point,” he added.
OpenAI addressed those limitations in its technical safety report, writing that “we will continue to improve the provenance ecosystem to help bring more transparency to content created from our tools.” OpenAI did not immediately reply to a request for comment.
Though the Sora app is available for download, access to Sora’s services remains invitation-only as OpenAI gradually increases access.
Source link
