Home / Blog / OpenAI wants to stop deepfakes from overwhelming child…
Tech News

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

OpenAI says its CSAM classifier flags 48 percent fewer images and marks AI-generated abuse, with free access for vetted groups and a 24-hour ChatGPT pause.

By Dillip Chowdary • Nov 17, 2026 • Source: The Verge

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines
Image: The Verge

At a glance

Product
OpenAI CSAM classifier
Access
Free for vetted organizations
False positives
48 percent fewer images flagged
Identified images
Twice as likely to contain abusive material
2025 reports
400,000 generative AI CyberTipline reports
First offense
24-hour pause on ChatGPT access

The source has a clear five-part order.The draft file was overwritten by another process. I'll rewrite it to a separate path and check the counts again.OpenAI announced a classifier that it says will help child sexual abuse investigators cut through noise and find harmful material they can act on quickly. At a Monday press briefing in Washington, DC, the company said the tool can better detect abusive imagery of kids, including whether it is AI-generated, and that vetted organizations will get free access. The Verge's report, published by senior policy reporter Lauren Feiner on November 16, 2026, also covers more developer guidance and law enforcement training as policymakers accuse OpenAI and other companies of facilitating harm to children.

This piece sets out the CyberTipline totals cited at the briefing, the classifier's reported change in false positives, and the ChatGPT warning that follows an attempt to generate child sexual abuse material. It is for investigators, developers, and policymakers who need the names, counts, and legal rules from that briefing in one place.

01Monday briefing in Washington, DC

Vetted organizations are being offered free access to a classifier built to detect abusive imagery of kids, including whether an image is AI-generated. Developers get more guidance on how to detect this type of content. OpenAI is also helping to train law enforcement on how AI can aid their investigations.

The Monday event in Washington, DC, comes as policymakers and others push for more safety restrictions on AI tools. They accuse OpenAI and other companies of facilitating harm to children. OpenAI says the classifier is meant to help investigators cut through the noise and find harmful material they can act on quickly.

02CyberTipline's 400,000 AI reports

In recent years, AI-generated sexual images of children have threatened to overwhelm systems like the National Center for Missing and Exploited Children (NCMEC) CyberTipline, which collects and triages reports of child sexual abuse material (CSAM). The CyberTipline received 400,000 generative AI-related reports in 2025. That was an exponential increase from the 67,000 reports it received in 2024.

Researchers have warned that the influx could make NCMEC's workload "unmanageable." The US government has stated that both AI-generated and non-AI-generated CSAM is illegal, and 48 states have laws that address sexually explicit AI deepfakes. The tools that make those images are widely available, and researchers say a flood of fake images can make it harder to tell when a child is actively experiencing harm.

03Task force, statute, and lawsuits

Advertisement

Tech Pulse Daily

Get tomorrow's pulse first

Join engineers who read Tech Pulse before stand-up. Free, weekday mornings.

North Carolina Attorney General Jeff Jackson, who cochairs the AG Alliance's AI Task Force, said at OpenAI's event that the influx of AI-generated CSAM reports to NCMEC is "explosive." If the reports keep going that way, he said, "it is going to mean that we fail in our mission." Tech companies face a range of policy proposals that could require them to take more action against CSAM and other potentially harmful content for kids.

The Take It Down Act now requires social media platforms to remove real or AI-generated nonconsensual intimate images within 48 hours. The newly elected Congress could seek to take stronger action against AI companies like SpaceXSI, whose Grok chatbot went on an undressing spree earlier this year. Last month eight states sued OpenAI over a separate set of alleged harms to children, citing lawsuits alleging ChatGPT encouraged illegal substance use and suicide.

04Fewer flags and a generation flag

OpenAI says the classifier dramatically cuts down on false positives, flagging 48 percent fewer images, while the images it did identify were twice as likely to contain abusive material. It is designed to catch abusive AI-generated images produced by a variety of tools, including open-source models and nudifying apps. John Buckley, OpenAI's head of child safety public policy, said it is the first broadly available CSAM classifier with a specific flag for AI-generated material.

Buckley told the briefing, "It's important for industry to be able to understand what images may contain a fictional child versus a real child." He added, "That triage is critical in law enforcement reporting flows." He also said laws should recognize the harm of AI-generated CSAM, because "it potentially erodes social norms in offending against children, and it's often found colocated with child sexual abuse imagery of real children."

05ChatGPT pause and specialist help

OpenAI's own ChatGPT image generator is designed to deny user requests for CSAM. Users who attempt to produce it will now see a new warning, developed with expert guidance, that tries to encourage them to seek help. A first offense triggers a 24-hour pause in accessing ChatGPT, warns that the account could be permanently closed, and offers support resources to combat the behavior.

OpenAI says it reports "all instances of CSAM" to NCMEC. In a statement, Buckley explained that the 24-hour pause is based on expert advice for best practices "and international evidence on what works to prevent and respond to CSAM seeking vs CSAM generation." The point, he said, is "to interrupt harmful behavior and direct people toward specialist help to stop it."

Note: Jackson said North Carolina and many other states now have laws that cover synthetic CSAM, but partnering with OpenAI to help deter its creation is a fairly novel approach to an emerging issue. "Most of what AGs do is accountability work," he told reporters, because "most of our jurisdiction kicks in after the fact to hold people accountable for harm." He said they wanted to see if it was possible to develop a model of working together to prevent that harm, and that "it's still basically an experiment."

Frequently asked questions

What classifier did OpenAI announce for child abuse imagery?

OpenAI announced a classifier that it says detects abusive imagery of kids, including whether the image is AI-generated. Vetted organizations get free access. John Buckley called it the first broadly available CSAM classifier with a specific flag for AI-generated material.

How many generative AI reports did the NCMEC CyberTipline receive?

The CyberTipline received 400,000 generative AI-related reports in 2025, up from 67,000 in 2024. Researchers warned that the influx could make NCMEC's workload "unmanageable."

By how much did OpenAI's classifier cut false positives?

OpenAI says the classifier flags 48 percent fewer images, and the images it did identify were twice as likely to contain abusive material. It is built to catch output from open-source models and nudifying apps.

What happens on a first ChatGPT attempt to generate CSAM?

The image generator is designed to deny the request and show a new warning developed with expert guidance. A first offense triggers a 24-hour pause in accessing ChatGPT, a warning that the account could be permanently closed, and support resources. OpenAI says it reports all instances of CSAM to NCMEC.

What did Attorney General Jeff Jackson say about the report surge?

Jackson, who cochairs the AG Alliance's AI Task Force, called the influx of AI-generated CSAM reports to NCMEC "explosive" and said that if it continues, "it is going to mean that we fail in our mission." He described the joint prevention work with OpenAI as "still basically an experiment."

Sources

Dillip Chowdary

Author

Dillip Chowdary

Writes Tech Bytes coverage of AI, engineering, and the tools that actually ship. Editor of Tech Pulse Daily.

Related on Tech Bytes

Advertisement

5-min tech signal

Weekday briefing for engineers who skip the noise.

No spam · Unsubscribe anytime

Advertisement

✈️ CareerPilot

Your AI job-search copilot

Match your resume against live Ashby, Greenhouse & Lever openings — fit scores, job-specific resume optimization and email alerts.

Find matching jobs →

Free Tools

Browse all tools →