A recent study suggests that some of OpenAIโs AI models may have learned directly from copyrighted material โ without permission. This finding adds weight to ongoing legal battles by authors, developers, and other rights-holders who say their work has been unfairly used to build these models.
While OpenAI has argued that using such content is covered by โfair use,โ the study raises new concerns over whether that defence holds up. Researchers from the University of Washington, Stanford University, and the University of Copenhagen worked together to develop a method for checking whether AI models have memorised specific pieces of text during training.
New method reveals hidden memorisation
The study focused on what the authors call โhigh-surprisalโ words โ uncommon words that stand out when placed in certain sentences. For example, โradarโ in the sentence โJack and I sat perfectly still with the radar hummingโ is considered a high-surprisal word. Itโs less expected in this context than words like โengineโ or โradio,โ which might appear more often before the word โhumming.โ
Using this idea, the researchers tested several of OpenAIโs models, including GPT-3.5 and GPT-4. They took text snippets from fiction books and articles published in The New York Times, removed the high-surprisal words, and asked the models to guess the missing word.
If a model guessed the correct word with high accuracy, it suggested that the model had seen that exact phrase or passage during its training โ an indication of memorisation. Since the data included copyrighted books and journalism, this poses serious ethical and legal concerns.
GPT-4 showed signs of memorising books
The tests showed that GPT-4 โ OpenAIโs most advanced model โ appears to have memorised sections from popular fiction. Some of this content came from a dataset called BookMIA, which includes samples from copyrighted ebooks. The model also seemed to recall parts of New York Times articles less frequently than it did with fiction.
These findings point to the possibility that GPT-4 was trained, at least in part, on copyrighted materials. Thatโs a major issue for creators whose work may have been included without their consent.
Abhilasha Ravichander, a PhD student at the University of Washington and one of the studyโs authors, explained the significance of this discovery. โTo have large language models that are trustworthy, we need to be able to audit them and understand how they work,โ she said. โOur study offers one way to investigate that, but it also highlights the urgent need for more transparency around the data these models are trained on.โ
Ongoing legal and policy debate
OpenAI has been the subject of several lawsuits over its use of copyrighted content. It has defended its approach by arguing that training AI with such content is fair useโa legal principle in the US that allows for limited use of copyrighted material without needing permission.
At the same time, OpenAI has tried to show it takes content rights seriously. It has licensing agreements with some publishers and offers an โopt-outโ process so creators can request that their work not be used in training.
Still, the company continues to lobby governments around the world to support looser rules regarding AI training. It wants clearer legal protections that would allow models to be trained on a broad range of online contentโincluding some copyrighted materialโwithout facing legal risks.
But as this new study shows, thereโs a fine line between learning from data and copying it. And until lawmakers draw that line, the debate around fair use in AI training will likely remain heated.
