🔗 Share this article

Tech firms and child safety agencies will receive authority to evaluate whether AI tools can generate child exploitation material under recently introduced British legislation.

Substantial Increase in AI-Generated Harmful Material

The announcement coincided with findings from a safety monitoring body showing that reports of AI-generated CSAM have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025.

New Regulatory Structure

Under the changes, the government will allow designated AI developers and child safety organizations to examine AI systems – the foundational systems for chatbots and visual AI tools – and ensure they have sufficient safeguards to prevent them from producing images of child sexual abuse.

"Ultimately about stopping exploitation before it happens," stated Kanishka Narayan, noting: "Experts, under rigorous protocols, can now detect the risk in AI systems promptly."

Tackling Legal Obstacles

The amendments have been implemented because it is illegal to produce and own CSAM, meaning that AI developers and other parties cannot generate such content as part of a testing regime. Previously, officials had to delay action until AI-generated CSAM was uploaded online before dealing with it.

This legislation is aimed at averting that problem by helping to stop the creation of those images at their origin.

Legislative Framework

The amendments are being added by the government as modifications to the criminal justice legislation, which is also implementing a ban on owning, producing or sharing AI models designed to generate exploitative content.

Real-World Impact

This recently, the minister visited the London headquarters of Childline and listened to a simulated call to counsellors involving a report of AI-based abuse. The interaction portrayed a teenager requesting help after being blackmailed using a explicit deepfake of themselves, created using AI.

"When I learn about young people facing blackmail online, it is a source of intense frustration in me and justified anger amongst families," he said.

Concerning Statistics

A prominent online safety foundation stated that cases of AI-generated exploitation content – such as webpages that may contain multiple images – had more than doubled so far this year.

Cases of the most severe material – the gravest form of abuse – increased from 2,621 visual files to 3,086.

• Girls were overwhelmingly victimized, making up 94% of prohibited AI images in 2025
• Depictions of newborns to two-year-olds rose from five in 2024 to 92 in 2025

Sector Response

The law change could "represent a crucial step to guarantee AI tools are secure before they are released," stated the chief executive of the internet monitoring organization.

"Artificial intelligence systems have made it so victims can be victimised all over again with just a simple actions, providing criminals the capability to create potentially limitless amounts of sophisticated, photorealistic child sexual abuse material," she added. "Material which further commodifies victims' suffering, and makes young people, especially girls, more vulnerable on and off line."

Support Session Data

Childline also released information of counselling sessions where AI has been referenced. AI-related harms mentioned in the conversations comprise:

• Using AI to evaluate weight, physique and looks
• AI assistants dissuading young people from consulting safe guardians about harm
• Being bullied online with AI-generated material
• Online extortion using AI-manipulated pictures

During April and September this year, the helpline delivered 367 support interactions where AI, conversational AI and related topics were mentioned, four times as many as in the equivalent timeframe last year.

Half of the references of AI in the 2025 sessions were related to psychological wellbeing and wellbeing, encompassing utilizing AI assistants for support and AI therapeutic applications.