Welcome to dystopia – or perhaps better spelled as dystopAI: harassment, humiliation, grooming, sextortion, suicide, and fraud cases exceeding $1.5 billion in 2025. Because humans have an extremely difficult time spotting the difference between authentic images and deepfakes, the crisis is all the more difficult to stop.
And it’s massive. As Gov.UK reports, a 2023 investigation “found 276,149 sexual deepfakes on a single dedicated site … with 96 percent depicting women,” and noted that globally, “nudification” sites in September 2023 alone attracted 24 million visits.
“‘Nudification’ apps are not used for harmless pranks,” says Jess Phillips, the UK’s Minister for Safeguarding and Violence against Women and Girls. “They devastate young people’s lives ... We must stop these images being created and shared while tackling the root causes of negative influences on young men in their schools, homes and online.”
So, what’s the solution to this devastation? In the United Kingdom, ministers have pledged to make it impossible for children in the UK to take, share or view a nude image using their phones. But that pledge does little to address the creation of deepfakes themselves, and is meaningless without digital tools that make detecting deepfakes possible.
That’s why so much hope is arising from innovations such as RealorRender, the product of researchers at Germany’s Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation (IOSB), as funded by the German Federal Office for Information Security (BSI).
In addition to detection, RealorRender explains why it classifies images as authentic or AI-generated, allowing developers to increase its accuracy and transparency. The IOSB’s robust, novel, hybrid approach integrates deep learning classification with an assessment of how reliably a generative model can reconstruct (deepfake) an image.
As senior scientist in the Video-Assisted Security and Assistance Systems research group, Andreas Specker explains, “First, we use an AI image generator to reconstruct the image. Then an AI model takes over the classification and carries out a hybrid calculation of the reconstruction error. We finally obtain an estimate displaying recognition accuracy as a percentage.” When the AI image generator successful clones a source image, that success is itself proof that the source image itself was a deepfake.
So far, RealorRender’s success rate starts at 85% and peaks at 91%, although as Specker explains, “It can certainly be higher in individual cases.”
Key to RealorRender’s value is that it reveals the reasons for its conclusions. “We use explainable AI (XAI) methods for this, which allows us to highlight the image areas and structures that contribute to the model's decision,” says Nadia Burkart, Head of the Applied Explainable AI group at Fraunhofer IOSB, ”Distinctive textures or characteristic frequency patterns can hint at the synthetic origin of an image. XAI helps to show which features the model uses in its decision.”
To explain its results, RealorRender identifies the suspicious regions of images as heatmaps (grids of colored squares indicating intensity, as with topographical maps) and segments of contiguous image elements. Offering analysis of a large dataset of images, RealorRender’s hybrid system generates greater accuracy than previous approaches in detecting deepfakes, while its explanations increase transparency, offering a powerful weapon even against ever-more-powerful deepfake engines.
For everyone currently or potentially targeted by personal, commercial, or political deepfakes, such new detection means increased safety, which may also reassure Minister Phillips, who says, “Every child deserves to grow up safe, and we will do whatever it takes to make that a reality ... That’s why we will join forces with tech companies to stop predators online and prevent the next generation from being exploited by sexual extortion and abuse ... We will ensure those who create or supply [sexual deepfakes] face real consequences.”
Source: Fraunhofer
