What’s Leaking Through Your Controls?

The Problem of Scale

Even the best controls aren’t perfect, and bad actors increasingly rely on volume to overcome even the smallest error rate. New technology has supercharged these efforts, allowing fraudsters to spin up new websites, products listings, and ad campaigns in seconds. 

One platform reported removing more than 159 million scam ads as well as nearly 11 million accounts associated with criminal scam centers. Those numbers show both the valiant effort many platforms are making to detect and remove problematic content and the staggering scope of the problem. It’s proof of how platforms are being barraged with violative content, and a reminder that a large takedown number and a large leakage number can both be true at the same time.

The AI Problem

Bad actors testing for gaps in a platform’s defenses is not new. What has changed is the speed at which they can run that test-and-adapt loop. Research on AI-driven content moderation now frames the relationship between a platform and an adversary as an ongoing adversarial game rather than a one-time classification problem. Attackers who once needed real technical skill to slip past a filter can now automate the search for whatever still works.

The numbers are stark. One study found that certain text-obfuscation techniques can clear the majority of keyword-based moderation systems outright, and some multi-turn manipulation techniques succeed as much as 90% of the time against commercial safeguards. 

What that means for platforms: a system trained on last quarter’s fraud patterns is defending against last quarter's fraud. The bad actors have already moved on.

The Fraud Problem

The problem with the scam industry is that it’s good business. The Federal Trade Commission recently said that Americans reported losing $2.1 billion to scams that started on social media in 2025, an eightfold increase since 2020. Nearly 30% of everyone who reported losing money to a scam last year said it began on social media. Shopping scams, the kind where someone orders a product from a site impersonating a well-known brand, were the single most reported type.

Zoom out and the number gets bigger. The FTC’s Consumer Sentinel Network logged a record $16 billion in reported fraud losses in 2025, up from $12.5 billion the year before. That’s a jump of roughly a quarter in a single year, and that’s only what got reported.

None of that fraud is happening off platform. It’s happening inside the ad systems, the seller onboarding flows, and the content feeds that internal trust and safety teams are already monitoring. Bad actors are highly motivated because the payoffs are proven; they will continue to show up where their potential victims spend time, looking for creative ways to get around controls.

The Network Problem

Most detection systems are built to catch individual violations: a listing, a post, an account. But sophisticated bad actors know this, and they plan for it. They know that some of their efforts will get caught by the controls, so they invest in building out a connected network.

Think of it like a hydra: When one account gets taken down, another pre-prepared account goes up in its place, often within hours. When a platform bans a keyword, the operation shifts to a new one. When enforcement gets close to the source, the storefront moves, the domain redirects, or communication drops into an encrypted app.

Individual enforcement wasn’t built to counter this approach. It leads to “whack-a-mole” efforts that never yield any meaningful results and has enforcement teams running in circles. 

The Regulator Challenge

While platforms are responsible for staying ahead of regulatory changes, it’s not always easy to predict regulatory priorities. Platforms may be focusing their enforcement efforts in one area only to discover that regulators have homed in on another. 

For example, while platforms regularly battle against listings and ads for illegal drugs, psychoactive products, and weapons, the European Commission recently turned its focus to consumer products such as unsafe baby toys and small electronics that could cause health risks, injury, or even death. In February 2026, the commission opened formal proceedings against Shein over the systems it uses to limit illegal products, following an earlier finding that Temu hadn’t properly assessed the risk of illegal products on its own marketplace.

Ultimately, it’s not only scale that’s a problem, but breadth. The diversity of product types and the multitudinous regulations that govern them globally make it difficult to stay abreast of all regulatory changes at all times.

Furthermore, regulators are increasingly looking at the problem of violative content and products as a systemic issue that platforms are ultimately responsible for. Regulators aren’t just asking platforms to remove more bad listings; they're asking whether the underlying system was ever built to catch them.

Why In-house Monitoring Stalls Out

Ask any trust and safety team what’s wrong with their internal systems and you’ll often hear a version of the same answer: too much noise, not enough signal. Analysts working under sustained alert overload become desensitized. As a result, they can start missing the alerts that actually matter, and start second-guessing whether an escalation is worth raising at all. The tools built to catch problems end up eroding the very judgment needed to act on them.

PwC’s Trust and Safety Outlook for 2026 puts numbers to what practitioners already feel: faster detection and faster response are now the top investment priorities across the industry, ahead of nearly everything else. That’s not a sign the old model is being fine-tuned. It’s a sign the old model, built for content that was predictable and volume that was manageable, no longer matches the job.

Industry surveys of platform moderators describe teams stretched thinner even as the range of harms grows: coordinated disinformation, AI-generated spam, and synthetic impersonation, arriving faster than moderators can be trained to recognize them. A moderation stack tuned for yesterday’s harms will wave through the ones nobody’s written a rule for yet.

It’s not that internal teams aren’t working hard. It’s that internal monitoring, by definition, monitors what it was built to see. Everything outside that scope is invisible until it shows up as a chargeback, a regulatory inquiry, or a headline.

The Solutions: Technology Paired With Expertise

In-house monitoring can struggle to see around corners because it lacks the wider context of behavior on the internet. Closing that gap requires seeing what’s happening across the entire landscape, not just inside one platform’s four walls. That’s precisely the kind of visibility an outside partner, purpose-built to track bad actors as they move between platforms, sellers, and payment rails, can bring.

Technology and Data

Advanced technology, particularly AI-powered monitoring capabilities, are increasingly fundamental to managing the problem of scale. According to PwC, these systems can automatically catch and remediate as much as 97% of violations with little or no human involvement. With the flood of fraudulent ads and content that hit platforms daily, AI technology is best equipped to identify and stop malicious AI-produced content at scale. This frees up trust and safety teams to focus on complex edge cases.

Data is another essential component. AI monitoring systems are only as good as the data they’ve been trained on. The best monitoring technology requires massive, accurate data sets that have captured bad actor activity across the internet, not just on a single platform. And the data must be continually updated as trends change.

Regulatory and Policy Expertise

As technology handles an ever-larger share of the monitoring work, humans still play a crucial role in the trust and safety ecosystem. As global regulations evolve, regulatory experts help to track laws, interpret their meanings, and decode priorities. External regulatory experts help to develop and refine policy, evaluate complex or novel scenarios, and help teams understand how comparable platforms manage the same risk areas. In-house monitoring alone can’t do that.

Furthermore, policy experts track emerging risks and help platforms make informed decisions about which activities to support, restrict, or prohibit. This input can help platforms seize growth opportunities while more confidently managing potential risks around them.

Grow More Confidently With LegitScript

LegitScript's Platform Risk Solutions were built for exactly this problem: finding what internal controls miss, monitoring the sellers and listings that evade static rules, and giving platforms the external, always-on visibility that in-house systems alone can’t provide. 

If you’re wondering what's leaking through your controls right now, that’s a question worth answering before a regulator, a payment partner, or a customer answers it for you. Contact us to learn more about how LegitScript can help you catch what you’re missing, and help you grow where you’re not.