Types of deepfakes: The dark side of AI-generated content

Have you come across Barack Obama dubbing Donald Trump a "complete dipshit," Mark Zuckerberg bragging about holding "total control of billions of people’s stolen data," and Jon Snow apologising for the Game of Thrones finale? Then, buddy, you’ve been excellently fooled by deepfakes. 

A modern-day evolution of Photoshop, deepfakes use artificial intelligence, specifically deep learning, to create fabricated images of events that never occurred. The world of deepfakes is vast and often ventures into the explicit; a 2019 Deeptrace study found a surge to 15,000 deepfake videos, nearly doubling in nine months, with a staggering 96% delving into pornography. 99% of these explicit deepfakes imposed female celebrities' faces onto adult film actors.

The deepfake issue reached a boiling point when a morphed video of Rashmika Mandanna surfaced on the internet. Even Prime Minister Narendra Modi expressed concerns, prompting global discussions on AI and its ethical regulation. 

But do we know how to differentiate between a real and a deepfake video? Unfortunately, a study by Christopher Doss revealed that only 46% of adults can make the distinction.

Deepfakes are evolving faster than our ability to spot them. But that doesn't signify we're powerless. This blog decodes the varied types of deepfake matrices. Read on to know more.

Face-swapping deepfakes

This type of deepfake stands out as one of the most prevalent and easily recognisable forms. It revolves around replacing a person's face in a video or image with another, resulting in a natural-looking and often convincing transformation. The intricate process behind face-swapping deep fake videos involves several key steps–

Facial recognition algorithms

Deepfake creators use advanced facial recognition algorithms to analyse and map the facial features of both the source (original) and target (desired) individuals. These algorithms identify essential facial elements such as the eyes, nose, and mouth.

Machine learning models

Once the facial features are identified, machine learning models take the stage. These models undergo extensive training using vast datasets, enabling them to generate lifelike facial movements and expressions that seamlessly align with the target individual.

The applications of face-swapping deepfakes are diverse, serving both malicious intentions and entertainment purposes. Some notable instances include:

Political manipulation

Deepfake videos have been manipulated to superimpose the faces of politicians onto unrelated individuals, leading to the creation of false narratives or the dissemination of misinformation.

Celebrity impersonations

For more lighthearted pursuits, some individuals use face-swapping deepfakes for entertainment, swapping their own faces into movies or music videos, effectively impersonating celebrities.

Also Read

Even a two-second voice clip or one image is enough to create deepfakes, say experts

Textual deepfakes

Textual deepfakes represent AI-driven systems proficient in generating human-like written content, spanning articles, poems, and blogs. These systems use advanced natural language processing (NLP) and natural language generation (NLG) techniques to analyse and produce text aligned with specified styles, topics, or tones.

A prime example of textual deepfake technology is GPT-3, a creation of OpenAI. GPT-3 is a text-generating system capable of crafting stories, news articles, poems, and responses that closely mimic human-written text. Beyond mere text creation, GPT-3 showcases versatility by answering questions, generating code, and executing diverse tasks based on natural language input.

While textual deepfakes find positive applications in creative and educational spheres, such as writing essays, novels, and summaries, their misapplication poses significant concerns. These technologies can be exploited for spreading misleading information, like fake news, propaganda, and phishing emails.

Voice cloning deepfakes

While face-swapping is focused on visual deception, voice cloning deepfakes take the manipulation of audio content to new heights, aiming to deceive listeners by replicating someone's voice through training a model on their existing voice recordings.

Voice cloning deepfakes employ advanced algorithms and machine learning models to generate someone's voice. Key techniques employed in crafting deceptive audio content include:

Text-to-speech synthesis

This involves converting written text into spoken words using artificial voices generated by machine learning algorithms. Parameters like tone, pitch, intonations, and emphasis can be manipulated, resulting in a synthetic voice closely resembling a specific individual.

Speaker adaptation algorithms

These algorithms analyse a target speaker's unique vocal characteristics and adapt a pre-existing model to mimic their voice. By training the model on a dataset of the target speaker's speech patterns, the resulting deepfake can imitate their voice with great accuracy.

Voice cloning deep fakes introduce significant concerns, particularly in areas like fraud and impersonation, leading to potential implications like supporting illicit activities.

Platforms like Lyrebird AI and Deep Voice allow users to create digital voices based on a few minutes of recording. Using deep neural networks, Lyrebird AI learns the characteristics of a person's voice and generates new speech based on it. While deepfake audio has positive applications such as communication and accessibility, translating languages or creating audiobooks, the range of misapplication is greater.

Live deepfakes

The most advanced deepfake technology, live deepfakes pushes the boundaries by manipulating reality in real-time through the integration of artificial intelligence. These synthetic media form streaming technology and generative adversarial networks (GANs), to produce live videos, images, audio, or text that dynamically respond to user input or environmental changes.

A noteworthy example of live deepfake technology is Neuralink, an innovative venture led by Elon Musk in brain-computer interfaces. Neuralink envisions the development of implantable devices capable of establishing a direct connection between the human brain and computers, facilitating instantaneous communication between the two.

Live deepfakes find applications in immersive and interactive domains such as gaming, virtual reality, and augmented reality, offering a potential avenue for enriching user experiences. However, their capabilities also raise concerns about potential misuse, including exerting control over an individual's actions or manipulating their thoughts.

Content creators, particularly on platforms like YouTube, have embraced the transformative possibilities of live deepfakes. For instance, DeepFaceLive, an open-source artificial intelligence software, allows users to alter their facial appearance during videoconferences and streaming sessions. This feature has gained traction among streamers on platforms like Twitch, offering broadcasters and developers across various media outputs a tool to explore creative and engaging possibilities in real-time content transformation.

Deepfakes blur the line between truth and fiction, leaving us questioning, “Can we even believe anything anymore?” Though deepfakes may lurk in the shadows, their deception won't be eternal. Learn to spot these digital chameleons before they blend in.