Open Source AI Models Have a Stolen Goods Problem

For years, the pitch behind open source artificial intelligence was simple: unlike the black boxes built by OpenAI or Google, these models could be downloaded, inspected, and run by anyone, a transparent alternative to corporate AI. That pitch is now colliding with a less flattering discovery. Court filings, class action settlements, and academic audits over the past 18 months show that many of the world’s most widely used AI models, open and closed alike, were trained in part on material that was pirated, scraped without consent, or in at least one documented case, illegal.

The problem is no longer speculative. It has a paper trail, in federal courtrooms in California, in a High Court judgment in London, and in a Stanford research lab that found child sexual abuse material embedded in one of the largest open datasets in the world.

The Meta files

The most detailed account of how a major AI company allegedly acquired its training data comes from Kadrey v. Meta, a copyright case originally brought by authors including Sarah Silverman, Richard Kadrey, and Ta-Nehisi Coates. Court documents unsealed during discovery showed that Meta engineers torrented data from Library Genesis, known as LibGen, and Z-Library, two so-called shadow libraries that host pirated books and academic papers, and that the decision to use LibGen was ultimately escalated to Mark Zuckerberg himself, according to internal messages cited in the plaintiffs’ summary judgment filing. 

One engineer downloaded roughly 81 terabytes of material through torrenting, The Register reported, even as colleagues raised objections internally. “I feel that using pirated material should be beyond our ethical threshold,” one Meta researcher wrote, according to the unsealed filing.

Meta won the first round. In June 2025, Judge Vince Chhabria granted Meta summary judgment on fair use grounds, finding the use of the books “transformative” enough to clear the legal bar, largely because the plaintiffs had not shown the market harm the law requires. But the fight is far from over. 

In May 2026, a group of five major publishers, whose catalogs span academic, educational, and trade publishing, filed a new class action against Meta and Zuckerberg personally, arguing that the earlier ruling ignored evidence of market substitution that this case can now supply. Legal analysts tracking the space count more than 50 AI copyright lawsuits filed in the United States since 2023, with roughly 30 still active.

The most expensive admission yet

Anthropic, the maker of the Claude chatbot, chose a different path than Meta: it settled. In September 2025, the company agreed to pay $1.5 billion, roughly $3,000 for each of about 500,000 books, to resolve Bartz v. Anthropic, a class action brought by authors Andrea Bartz, Charles Graeber, and Kirk Wallace Johnson. 

Court records showed Anthropic had downloaded more than 7 million digitized books from LibGen and a second pirate repository called Pirate Library Mirror. “As best as we can tell, it’s the largest copyright recovery ever,” said Justin Nelson, a lawyer for the authors.

The ruling underlying that settlement drew a distinction that is likely to shape every AI copyright case that follows. Judge William Alsup found that training a model on legally acquired books was, in his words, “quintessentially transformative” and protected under fair use, but that downloading and retaining pirated copies was not. 

In other words, paying for a book and training on it is probably fine. Pirating the same book is not, no matter how the resulting model is used afterward. Anthropic has agreed to destroy the pirated files as part of the settlement, though it admitted no liability.

What “open” hides

If the Meta and Anthropic cases are about how models are built, the case of Stable Diffusion is about what happens when a model’s training data is public by design, and still turns out to contain material nobody should have collected in the first place.

In 2023, researchers at the Stanford Internet Observatory examined LAION-5B, an open dataset of billions of image-text pairs used to train Stable Diffusion and other image generators, and identified more than 3,200 suspected instances of child sexual abuse material, of which 1,008 were externally validated by the Canadian Centre for Child Protection, according to the Stanford report. 

LAION, the German nonprofit that maintains the dataset, took it offline within days, citing a zero tolerance policy for illegal content. The episode became a case study in a problem researchers had been warning about for years: open datasets assembled by scraping billions of images from the open web inherit whatever was on that web, without anyone meaningfully checking first.

Stability AI, the company behind Stable Diffusion, faced a separate and more conventional copyright fight from Getty Images, which alleged the model had been trained on roughly 12 million of its photographs without a license. In November 2025, the UK’s High Court delivered a split verdict: it rejected Getty’s copyright claims on narrow, technical grounds, ruling that Stable Diffusion’s model weights do not “store” the images they were trained on and therefore cannot be treated as an infringing copy, while still finding limited trademark infringement over Getty’s watermark appearing in generated images.

Getty called the outcome only a partial resolution. “We remain deeply concerned that even well-resourced companies such as Getty Images face significant challenges in protecting their creative works,” the company said in a public statement, noting the ruling turned in large part on the fact that Stability’s training and development took place outside UK jurisdiction, not on whether scraping copyrighted images for training is lawful in principle.

The open washing problem

None of this would matter as much if buyers of these models, or the researchers trying to audit them, could simply check what went into them. Increasingly, they cannot. A growing body of academic research has coined a specific term for the gap between what “open” AI companies claim and what they disclose: open washing. 

A 2024 paper in Nature by researchers David Gray Widder, Meredith Whittaker, and Sarah Myers West argued that many so-called open AI systems are, in practical terms, closed where it counts most, withholding the data, code, and documentation needed to actually reproduce or audit them, even while marketing themselves as open.

Meta’s Llama models have drawn particular scrutiny on this point. The company has released model weights publicly while keeping the composition of its training data undisclosed, a pattern researchers argue defeats the purpose of calling a model open in any scientific sense. 

“Many ‘open’ models don’t disclose what’s in the training set,” said Mike Lieberman, cofounder and chief technology officer of the software supply chain security firm Kusari, warning that vendors using undisclosed data may bear hidden legal exposure that gets passed downstream to whoever builds on top of their model, as he told LeadDev.

The Open Source Initiative tried to resolve the ambiguity in October 2024 by publishing a formal Open Source AI Definition, requiring not just open weights but sufficient information about training data to allow a model to be substantially recreated. 

Almost no major “open” model released since then fully meets that bar, according to researchers who track compliance against the standard, because the training data itself, not the code or the weights, remains the dimension of openness companies are least willing to share.

Why it keeps happening

Taken together, the pattern across Meta, Anthropic, Stability AI, and the open source ecosystem more broadly is not that any single company behaved uniquely badly. It is that scraping the internet at scale, whether the result is called open or proprietary, has consistently outpaced anyone’s ability, including the companies themselves, to verify what was swept up in the process. Shadow libraries offered a shortcut to the scale of data frontier models require. 

Open image datasets offered a shortcut to diversity and volume no single organization could license fast enough. Both shortcuts left the same trail: material nobody had the right to take, sitting inside systems now generating billions of dollars in value.

Regulators are only beginning to catch up. The European Union’s AI Act now requires general purpose AI providers to publish a summary of the content used to train their models. The UK government is due to publish its own report on copyright and AI training by March 2026, following the unresolved territorial questions the Getty ruling left open. 

In the United States, the outcome still rests almost entirely on courts deciding cases one lawsuit at a time, an approach that has so far produced a genuinely split verdict: paying for data and transforming it is probably legal, and taking it for free from a pirate site, no matter how sophisticated the model built on top of it, is a bill that eventually comes due.

For companies still building on scraped or pirated corpora, the settlements so far offer a strange kind of comfort: none has stopped development, and none has forced a company to delete a trained model outright. What they have done is put a price on the shortcut. 

Stay ahead of the stories shaping our world. Subscribe to Impact Newswire for timely, curated insights on global tech, business, and innovation all in one place.

Dive deeper into the future with the Cause Effect 4.0 Podcast, where we explore the ideas, trends, and technologies driving the global AI conversation.

Got a story to share? Pitch it to us at info@impactnews-wire.com and reach the right audience worldwide