Skip to content

Exposing AI Deepfakes Is Critical to Combating Fraud in 2024 — Here’s How

AI deepfakes

The rapid advancement and widespread adoption of deepfake technology marks a pivotal moment in the development of the technological landscape, introducing a higher degree of uncertainty to the online ecosystem and putting individuals, institutions, and businesses that fail to adapt at an increasingly greater risk. Read on to understand the far-reaching implications behind AI deepfakes and learn how LegitScript can help you uncover risky deepfake offerings among your merchants or sellers.

AI Deepfakes Are a Convincing Part of the Online Information Landscape

Deceptive videos and images created using generative artificial intelligence (AI) are rapidly becoming a more common and convincing part of the online information landscape. The potential ramifications extend beyond individual privacy, with disruptive implications in the arenas of commerce, society, and national security.

The term “deepfake" refers to a piece of media, such as a video, audio clip, or image, that has been manipulated and/or synthesized to misrepresent someone as doing or saying something they did not.

AI deepfakes leverage advanced machine learning techniques, particularly deep neural networks, to create highly sophisticated and realistic manipulations of media that require little investment of time and energy. With billions of dollars of venture capital flowing into image-generating software powered by artificial intelligence, it has never been cheaper or easier to create convincing photos and videos of things that never happened.

Developments in Deepfake Technology Have Far-reaching Implications

These developments in deepfake technology have far-reaching implications for manipulating public perception and eroding trust in institutions. Deepfakes can fabricate convincing videos of public figures and media personalities making false statements, potentially influencing public discourse, damaging reputations, and challenging our understanding of reality. Businesses and brands are no more immune to the risks posed by AI deepfakes than individuals or public institutions.

The Potential for Economic Sabotage Is Substantial

With the fallout of fabricated media having a real impact on operations and public image, AI deepfakes have also added a new dimension to concerns surrounding personal privacy. The ability to superimpose someone's likeness and/or voice onto explicit or compromising scenarios poses a significant threat to individuals' personal and professional lives. With the click of a button, malicious actors can create content that tarnishes reputations and inflicts irreparable harm. This use of AI deepfakes, more than any other, has captured the public’s attention in recent years as the associated ethical and legal questions tie into the popular discourse surrounding systemic issues such as consent, rape culture, and violence against women.

A 2023 study noted a 550% increase in the total number of deepfake videos online in the past four years, with deepfake pornography accounting for almost 98% of those videos. Women in the entertainment industry are disproportionately affected due to their high visibility and the large volume of images, video, and audio available to them online. Of the videos analyzed in the study, 94% of the individuals targeted were women working in the entertainment industry.

As of December 2023, there is no federal law criminalizing the creation or distribution of fake pornographic images. Several states, including Texas, Minnesota, and New York, have recently passed legislation criminalizing nonconsensual deepfake pornography. Efforts to address the issue legislatively have been ongoing since 2018, with some states taking steps to criminalize nonconsensual deepfake porn.

LegitScript has observed a notable increase in merchants offering deepfakes and/or the software to produce them over the past year, and card brands appear to be on the lookout for bad actors in the industry. More legitimate operators in the industry may offer services such as AI-assisted personalized videos and other content. It is not always evident, however, whether or not a merchant has adequate controls in place to stop their users from creating deepfake and/or adult content.

Bad actors, on the other hand, operate services designed to enable users with no prior technical knowledge to generate adult content from a handful of public or private images. LegitScript is also on the lookout for merchants turning toward transaction laundering methods to obfuscate their deepfake adult services.

LegitScript Analysts Uncover Deepfake Services in Their Monitoring Work

Technological advancements often outpace regulatory frameworks, demanding nimble response and rapid adaptation from the individuals, businesses, and institutions affected and involved. LegitScript remains committed to monitoring regulatory updates and trends related to the development and dissemination of AI deepfakes and stands as a partner in identifying and reporting bad actors operating within our shared online ecosystem.

If you’re concerned about the potential effects of AI deepfake services impacting your merchant portfolio, e-commerce marketplace, or ad platform — contact us.



Recent Blog Articles

AI and ROI in trust and safety.

Navigating Marketplace Risk: AI and the ROI of Trust and Safety

Every year, professionals from around the world come together at the Marketplace Risk Management Conference to discuss issues of risk on online platforms and other technology. Explore the most critical takeaways from 150+ industry-leading speakers spanning 70+ sessions. Then contact us to see how Le...
Pride Month Addiction Treatment Certification application fee waiver.

This Is How LegitScript Is Celebrating Pride Month

LegitScript is celebrating Pride Month by waiving application fees for a limited number of new applicants who provide specialty care for the LGBTQ+ community - along with addiction treatment services. Let's unfurl Pride Month's origins, and discuss why the LGBTQ+ community needs support for addictio...
Levels of risk.

What You Need to Know About the Different Levels of Merchant Risk

Many risk mitigation approaches focus on high-risk merchants, but what about medium-risk merchants? Being aware of the various levels of risk and which category your merchant falls into is a critical step in quantifying risk. Let's delve into what low-, medium-, and high-risk merchants are and the v...
Northern Colorado Hemp Exposition (NOCO)

Navigating Compliance: LegitScript’s Insights from NOCO

Join LegitScript on a journey to the 10th annual Northern Colorado Hemp Exposition (NOCO), where we immersed ourselves in the heart of the CBD industry. LegitScript shared critical compliance knowledge, exchanged insights with industry peers, and absorbed vital regulatory updates from the FDA and US...