New AI tool detects child sexual abuse material with ‘99% precision’
Safer is already being used by Flickr and Slack
How Safer works
The system detects abusive content on a platform by searching for digital fingerprints called hashes.
When it finds these hashes, it compares them to a dataset of millions of images and videos of abusive material. If the content hasn’t been previously reported, algorithms trained on abusive material determine whether it’s likely to be CSAM. It’s then queued up for review by the platform’s moderation teams, and reported to theNational Center for Missing and Exploited Children, which adds the content to its database.
Among Safer’s early clients is photo-sharing service Flickr. The company recently used the tool to detect a single image of abuse on itsplatform. A law enforcement investigation that followedled to the identification and recovery of 21 children, ranging from 18 months to 14 years old. The perpetratoris now in federal prison.
In total, Thorn says it took down nearly 100,000 CSAM files during its beta phase.The tool is now available to any companies operating in the US, with current customers including Imgur, Slack, and Vimeo. Next year, Thorn plans to offer the product to overseas firms, after integrating it with their reporting requirements.
The non-profit’s ultimate goal is to eliminate child sexual abuse material from the open web.
“We see a future where every child can just be a child, the internet is a safe environment, and victims who may have been victimized 10 years ago … will not be re-victimized when their images are reshared or resurfaced,” said Borgman. It’s really just disrupting the supply chain. It’s stopping it in its tracks.”
Story byThomas Macaulay
Thomas is a senior reporter at TNW. He covers European tech, with a focus on AI, cybersecurity, and government policy.Thomas is a senior reporter at TNW. He covers European tech, with a focus on AI, cybersecurity, and government policy.
Get the TNW newsletter
Get the most important tech news in your inbox each week.
