Seven Hundred Papers, Nobody Checking
The AI industry has a new habit. Ship the output. Skip the part that makes it usable.
Start with mathematics. OpenAI posted 722 mathematics manuscripts to GitHub, grouped into 372 results and produced by an internal model the public can’t use. OpenAI’s own repository warns that some of the results that haven’t been formally checked “could have issues.” By The Verge’s count, only about four in ten of the main results come with a machine-checkable proof. Mathematicians told The Verge it could take years just to understand what’s there. A referee reading one paper a day, every day, would need nearly two years to get through them.
The field had asked for something different. Weeks earlier, twenty-five Fields Medal winners signed an open letter warning that AI labs racing to solve famous problems were threatening the work itself. Their concern was solutions announced without proper write-ups, without citation of prior work, and without anyone able to explain them. OpenAI withdrew its sponsorship of a Caltech math event after researchers there criticized it. And a week before the release, an independent advisory group of mathematicians at the Institute for Advanced Study, one OpenAI itself consulted, asked labs to disclose the model, the prompts and the compute behind each result. OpenAI didn’t name the model, published no prompts, and gave compute only as an average.
The people downstream felt it immediately. NYU mathematician Tristan Buckmaster told Business Insider that entire research programs were wiped out in a day. Early-career researchers had built grant proposals around problems that now have claimed answers. OpenAI says the release includes a process for revisions and citations, and plans to fund workshops to help researchers work through it.
Some of it may well be real. That isn’t the question.
It isn’t only math. In its own report on unintended model actions, Anthropic disclosed that during testing, Claude Haiku 4.5 filled in a Philadelphia police tip form about a murder case, with no name and no contact details. “I may have information regarding this case,” it wrote. Its agents also filed 20 incomplete visa applications on a State Department form, the New York Times reported. None were processed. Anthropic says the cases had minimal real-world impact, and it disclosed them itself. But a police department still had to deal with a tip that came from a test.
Different fields. Same shape. The output shipped. The preparation the receiving side depends on didn’t.
So why does the preparation keep getting skipped?
Cheap to Make, Costly to Check
Every one of those norms exists for the same reason. They make whoever produces the work carry the cost of checking it.
Peer review asks an author to write a result up so others can follow it. Responsible disclosure asks a bug hunter to confirm the bug before filing it. That arrangement held because producing the work was expensive too. Nobody wrote seven hundred papers in a month.
AI broke that balance on one side only. Generating got nearly free. Checking didn’t get any cheaper.
You can watch it happen in open source. Linus Torvalds told kernel developers that the flood of AI reports has made the Linux security mailing list almost entirely unmanageable, as The Register reports. Different people run the same tools, find the same bugs and file the same reports. Maintainers spend their time forwarding them, or pointing out that the fix shipped weeks ago. Torvalds called it “pointless make-believe work.” His ask to reporters is the old norm, restated: read the documentation, write a patch, add something the AI didn’t.
Now look at what happens when a producer actually counts. Anthropic’s free OSS Scanner sends model-written vulnerability reports to the open-source projects that sign up, with no human review, and Anthropic warns some will be wrong. Anthropic also publishes a dashboard for its broader disclosure program. By its own numbers, it has disclosed 6,157 vulnerabilities across 591 projects. Maintainers acknowledged 5,103. Five hundred sixteen have been patched upstream, about one in ten. Even so, the finding was the easy part. The fixing is still on the maintainers’ side of the ledger. And Anthropic itself notes that a released fix doesn’t mean it’s installed. Every day between found and fixed is a day the bug is known and still open.
In plain terms, the producer’s books record the output: papers released, bugs found, reports sent. The checking, the fixing and the explaining land on somebody else’s calendar, and nobody records them. That’s why the output looks free. The cost was moved, not removed.
And it doesn’t only move to mathematicians and maintainers.
Follow that cost far enough, and it reaches a service desk.
The Cleanup Lands On You
For a small business, the people downstream of AI output are your clients. Their files get exposed. Their staff lose the hours. And the MSP is usually the only one in a position to see it.
On this show, Bryan Lukralle, chief strategy officer at Charter Technology Solutions, a Brooklyn MSP serving charter schools and nonprofits, described what’s landing on his team. Clients are connecting Claude and ChatGPT to their shared files. In their minds, it’s their own staff reading their own files. But the AI finds the file buried twenty folders down that was never locked down properly, and he says things have been exposed. The cleanup is permissions work, policy work and assessment work. And CTS is giving it away. He hopes to have a paid offering by January. Next on his list is how to staff support for apps built by teachers and administrators who aren’t developers. Nobody upstream counted any of that. It lands on his schedule.
Here’s what pushing back looks like. Daniel Stenberg leads curl, open-source software that moves data inside an enormous range of products. Its bug bounty paid out more than a hundred thousand dollars for eighty-seven confirmed vulnerabilities. Then Stenberg measured what was arriving. In earlier years, more than fifteen percent of submissions turned out to be real. Starting in 2025, it fell below five percent. Not even one in twenty. He described the time spent debunking the rest as “completely wasted.” So he ended the bounty, removed the money that rewarded volume, and kept banning people who submit AI slop. He didn’t absorb it. He counted it, and he changed the terms.
That’s the choice in front of an MSP. Count the work AI output moves onto your clients: the assistant they connected, the vendor’s model-written alert, the app somebody built over a weekend. Then show it to them, with terms attached, so that work arrives with a named person who checked it, or the client sees what skipping that step costs. Or absorb it quietly, and your clients never learn what the tools they chose actually cost, while you become the free verification layer for every producer upstream.
Before you pick a side of that choice, consider what each one does to your position.
The fair objection is that absorbing a little cleanup is just good service, and that the MSP who pushes back looks difficult next to the competitor who says yes to everything. But a cost nobody manages doesn’t stay small. The volume of unprepared AI output only goes up, and the shop quietly absorbing it is subsidizing every producer upstream out of its own margin. The separator won’t be who says yes. It will be who can show a client, in hours, what that client’s AI habits cost last quarter, and set terms from that number.
What to Consider
- Measure it before you argue about it. For one quarter, tag every ticket caused by AI output nobody prepared: the assistant a client connected, a model-written alert, a tool somebody built over a weekend. Without that number, pushing back is an opinion. With it, it’s a line in the renewal conversation.
- Change the incentive, not just the tone. Stenberg didn’t ask reporters to try harder. He removed the reward for volume. For an MSP, that means AI-originated work arrives with a named person on the client side who checked it, and work that doesn’t is scoped and billed as a project instead of disappearing into the flat fee.
- Make preparation the precondition. Lukralle’s clients learned what their permissions allowed when an assistant read them. Require a permissions review before any assistant is connected to shared files, priced as its own engagement. The cleanup then happens on your schedule as preparation, not on the client’s as incident response.
If this trend continues: Within the next year, the MSPs that can show a client a measured AI-cleanup number will be setting the terms, and the ones that can’t will find the cost in their own margins first.