Abstract
Deepfakes, or artificially generated audiovisual renderings, can be used to defame a public figure or influence public opinion. With the recent discovery of generative adversarial networks, an attacker using a normal desktop computer fitted with an off-the-shelf graphics processing unit can make renditions realistic enough to easily fool a human observer. Detecting deepfakes is thus becoming vital for reporters, social networks, and the general public. Preliminary research introduced simple, yet surprisingly efficient digital forensic methods for visual deepfake detection. These methods combined convolutional latent representations with bidirectional recurrent structures and entropy-based cost functions. The latent representations for the video are carefully chosen to extract semantically rich information from the recordings. By feeding these into a recurrent framework, we were able to sequentially detect both spatial and temporal signatures of deepfake renditions. The entropy-based cost functions work well in isolation as well as in context with traditional cost functions.
However, re-enactment based forgery is getting harder to detect with newer generation techniques ameliorating on temporal ironing and background stability. As these generative models involve the use of a learnable flow mapping network from the driving video to the target face, we hypothesized that the inclusion of edge maps in addition to dense flow maps near the facial region provides the model with finer details to make an informed classification. Methods were demonstrated on the FaceForensics++, Celeb-DF, and DFDC-mini (custom-made) video datasets, achieving new benchmarks in all categories. We also perform extensive studies to evaluate on adversaries and demonstrate generalization to new domains, consequently gaining further insight into the effectiveness of the new architectures.
Library of Congress Subject Headings
Deepfakes; Computer vision; Machine learning; Digital video--Editing; Fake news
Publication Date
6-13-2020
Document Type
Thesis
Student Type
Graduate
Degree Name
Computer Engineering (MS)
Department, Program, or Center
Computer Engineering (KGCOE)
Advisor
Raymond Ptucha
Advisor/Committee Member
Matthew Wright
Advisor/Committee Member
Shanchieh Yang
Recommended Citation
Chintha, Akash, "Employing optical flow on convolutional recurrent structures for deepfake detection" (2020). Thesis. Rochester Institute of Technology. Accessed from
https://repository.rit.edu/theses/10468
Campus
RIT – Main Campus
Plan Codes
CMPE-MS
