Systems designed to detect deepfakes (videos that manipulate real-life images through artificial intelligence) they can be fooled, as this study suggests.
Researchers have shown that detectors can be defeated by inserting adversarial examples into each video frame. Adversarial examples are slightly manipulated inputs that cause AI systems, such as machine learning models, to make an error.
Attacking blind spots
In deepfakes, a subject's face is modified to create realistic and convincing images of events that never happened. As a result, typical deepfake detectors focus on the face in the videos: first they track it and then pass the data from the cropped face to a neural network that determines if it is real or fake.
For example, the blinking of the eyes It doesn't play well in deepfakes, so detectors focus on eye movements as a way to detect that the video is fake.
However, if the creators of a fake video have some knowledge of the detection system, they can design inputs to target the detector's blind spots and avoid it.
The researchers created a confrontation example for each face in a video frame.. But while standard operations, such as compressing and resizing a video, typically remove adversarial examples from an image, these examples are designed to resist these processes. The attack algorithm does this by estimating over a set of input transformations how the model classifies images as real or fake. The modified version of the face is then inserted into all video frames. The process is then repeated for all frames of the video to create a deepfake video.
To improve the detectors, the researchers recommend an approach similar to what is known as Adversarial Machine Learning o adversarial training: during training, an adaptive adversary continues to generate new deepfakes that can bypass the current state-of-the-art detector; and the detector continues to improve to detect new deepfakes.
–
The news
For the first time, deepfake detectors can now be fooled and that is a problem
was originally published in
Xataka Science
by
Sergio Parra
.