On-Demand! Marketplace Risk Global Summit Panel

Transcript

David: I'll go ahead and say hello and welcome since everyone just stopped talking. Welcome.

My name is David and I am representing LegitScript. Happy to have you here at the Marketplace Risk Global Summit presented by Incognita and LegitScript.

LegitScript, if you are unfamiliar, we help online platforms, also payment service providers, build compliant and trustworthy ecosystems. And we do that by combining artificial intelligence and human expertise to identify risk with confidence. So if you have any questions about that, come talk to us. We've got a table out there.

And just some little notes here. We're pretty full, so we don't need to move people forward. There's a recording in the back, just so you know. But the questions at the end, which we'll try to leave some time for, will be edited out.

So no worries about asking silly questions, I guess.

And with that, we'll go ahead and get started today. And I assume it's the recording system start recording.

Let's start off by introducing our lovely team of experts here.

We've got Ted James from Legiscript. They're not in order here, but you can see they can hold up their hands too. Associate general counsel at LegitScript, Laura DeBenedetto, who works for Meta in the policy team, Tom Cook, also from LegitScript, and our Chief Technical Officer.

And Irina Andrei, she works in policy for Vintage. So big thanks to all of them for joining us today on our talk on artificial intelligence and the way that it's affecting policy. When we got together and talked about this talk and what we wanted to talk about, we realized that we're coming from artificial intelligence really in a couple of different ways. The way that artificial intelligence is creating challenges for teams as far as the user generated content and as far as creating policy within artificial intelligence in the sense that it's also creating opportunities and helping us more accurately and more efficiently identify risk.

And so today, we're gonna talk about sort of both sides of that, the challenges of artificial intelligence and also the opportunities that artificial intelligence is providing. With that, I thought it would be fun to start off with a polled audience question. So my question to you is this, and we're just gonna have you raise your hand because I'm curious about this.

Since you all work sort of in the platform risk space, from a risk perspective, AI technology right now, do you see it more as a threat and a challenge to your work, or do you see it more as an opportunity to make your work more efficient and more effective? Or is it just you're sort of somewhere in the middle? So who sort of sees it right now more as a threat or a challenge to the work they do? Hands raised.

One couple people. Okay. Yeah. Yeah. Yeah. Yeah. Yeah.

Who sees it more as an opportunity?

Okay. Majority. And who's just sort of in the middle, sort of like, equally? Okay. So probably both people here. That's why you're in this session because we're gonna talk both about the challenges and opportunities. Awesome.

Perfect. Before we start getting into sort of our talk around policy and how some of our marketplace experts are approaching that, I wanted to just start off by having Tom talk a little bit about because when we talk about artificial intelligence, it's just sort of a really broad topic and sort of broad definition. And so I wanted to set some definitions about what we're talking about when we talk about AI. So, Tom, can you just walk us through some of the most common types of AI that we're talking about and sort of what they mean?

Tom: Yeah. Sure. So hi, everybody. Good morning. So, actually, artificial intelligence is a phrase that is about seventy years old.

The first use of that term was just about seventy years ago. So it's been around for a long time, and it's a little, well you know, you've heard about it a lot. They have these advances and then there's what they call AI winters over the decades.

Things like neural networks, machine learning, image recognition, image analysis. There's a lot that's been developed and used for some time. I mean, I've been using AI for probably a couple decades. But what's changed, what changed everything really ia ChatGPT in late twenty twenty two, early twenty twenty three.

And that's when what we call the democratization of AI happened. So with advances in natural language processing and being able to understand process language, and then actually be able to respond in that way, now, just about everybody can use it. And so it took off, I think in one month maybe for a hundred million users to start using ChatGPT. And, you know, if you're like me, you know, you asked for things like, hey, give me the history of the Peloponnesian War in the style of Doctor Seuss. And you get astonished because it actually could do it. And so that was the early use of ChatGPT. But then, you know, then people started to figure out how to actually make good use of it, do the research, and so on.

And then these things called agents came. And so agents are now able to actually act with context and actually do some things.

And this is where both the threat and the opportunity exist because AI and agents in particular are excellent at mimicry. So when you hear about deepfakes and you hear about those are the things that we need to worry about hitting us. You know the deepfake IDs and businesses and merchant websites, product listings, you know, LinkedIn profiles, you know, they're all over the place. We have all seen them. And so that's the mimicry part.

The opportunity is, well, how do you use AI now to defend against that?

And that's where, you know, we'll talk about I'm like, there's a maturity model for AI that that people often refer to. There are five levels and, you know, the first was like, very early on, it was a very static kind of chatbot, and then there's one chatbot that has context. And then you have these agents that are kind of cognitive workers, and then you have autonomous agents, that's four, and then you kinda have an ecosystem of agents. And generally, we're at three.

So we're at level three maturity. There are some appearances of level four where you have autonomous agents working. But generally speaking, in the world we have these cognitive workers. The best way to think about them, you know, is a phrase called “well read interns.”

If you have someone who is just thinking they're out of college, they're very smart, they're educated, they have no context, and they don't know the business, and they don't know what they're doing. Right? And so when you think about agents, now you think about building your agent force, you need to be aware that the agents really have no context. They don't know what they're doing, and they don't know your business. So you can't over trust them, and that's the big challenge we have today.

David: Thanks. So we'll talk a little bit more about autonomous agents later in the talk, but probably what most of our marketplace folks are seeing right now is more generative AI as far as the content you're seeing on your platforms. And so I wanted to ask Laura and Irina just what are some of the most common issues of as far as AI that you're seeing on your platforms, especially when it comes to, like, some of the more, like, problematic stuff you're seeing, but just what's what are some of the things that you're seeing in your work on a daily basis or anything like Excel?

Irina: Yep. I can start?

So I can speak on the generative AI used for generating images because that's my focus for now. So, yeah, I would split the uses or the way people are using generative AI to generate photos in kind of two types of users, like harmless use or harmless from, yeah, everyone's perspective and malicious use.

And we are what we are mostly seeing right now based on the perception of the users, but also what we're seeing internally, what we're experiencing, what kind of images we're seeing being generated on Vinted is harmless use. And there are either to enhance the quality of an image and to kind of, I don't know, delete the background or make the image more appealing or to protect their privacy because people, understandably don't wanna be online that much or don't wanna show themselves online that much.

And, yeah, that would be one of the harmless reasons for why they use it, use AI, because they wanna protect their privacy and they wanna generate an image where they display the item that they are trying to sell without, yeah, giving away their privacy or their online identity.

It might be like modeling a jacket, and yeah, they want to, like, leave their face out.

And, also, I would add, like, when we are thinking outside of the fashion area and fashion items, we're also noticing that when someone is trying to sell a product that it's still packaged, it's like a bulky item, let's say a piece of furniture, they will try to or they will use AI just to help them display the final product that they are trying to sell, which may be basically impossible to display otherwise. So this is what I would qualify as harmless use.

David: Okay. Yeah. Laura, what about you?

Laura: Sure. I think that there's a lot of similarities between what you're seeing on your platform and what we see across Meta. There's a lot of just harmless use cases of AI, that goes everything from, like, enhancing photos, on marketplace or just on your Instagram profile and things like that. I deal primarily in the safety space, and so what we're most concerned about is how AI is used to perpetuate harms on the platform and how we can combat those.

And so that's everything from AI generated CSAM to, you know, really the democratization of really sophisticated, like, frauds and scams that, like, maybe ten years ago was something that, like, you would need, a really sophisticated network to to do to deal with, is now something that, like, one person might be able to do if they have the right AI tools. And so a lot of the work that we do is really around understanding what those kind of malicious use cases are and how we can then use, you know, AI on our side to help to identify those and to categorize those harms and to be able to, you know, mitigate those as quickly as possible.

And you mentioned, obviously, malicious AI like CSAM and things like that.

David: There's also sort of, like, small like, what are some of the malicious use cases, Irina, that you're seeing on Vinted?

Irina: So I would say we're not seeing them yet, so I can talk from the perspective of what we might see as people using AI to generate images of the item they received to make it look as if it's a faulty item.

Or I guess this goes more into the security of it all and security implications, but they might also use generative AI to act like yeah. As Laura was saying in a more complex way. So not at the personal level, not just someone trying to let you scam another person, but just for them to, like, do more systematic scamming.

David: Yeah. So you're looking ahead in the future to sort of see what you think how it could be used and sort of project ahead?

Irina: Yes. I would say we're trying to do that. We're trying to cover ground on what could potentially happen, which we're not really seeing right now, but for sure, it leaves the gates open to a lot of malicious use.

David: Yeah. And, Laura, you mentioned, you know, one person being able to to do maybe what a group was doing. Talk a little bit more about the democratization of AI and how that's impacting your approach.

Laura: Yeah. So I think it really comes back to scale, which I think we've briefly touched on. But I think that's different, you know, AI tools really enable bad actors to scale out their harms, and that's something that we have to really be cognizant of, as you know in the trust and safety space, where we wanna make sure that we are also keeping pace with that and being able to identify and remove those bad actors from the platform and, like, kinda recognize the harm that they're perpetuating. And I think that, as these, like, you know, different, like, scam archetypes continue to evolve, we need to keep pace with that and evolve our tooling and, you know, our approach to that. So it's definitely sort of an iterative process where we're always kind of, like, trying to understand the ecosystem better because it evolves so rapidly and I think even more so now with, with AI, the use of AI.

David: Let's talk now a little bit about policy creation because I'm sure that's top of mind for a lot of folks who are looking at how do we create good policy around artificial intelligence that is used. Irina, what are some of the challenges that you're facing as you go through a process of creating policy?

Irina: Oh, I would have a lot to say about that. I will try to keep it short.

I think one challenge is to try to define the terms I was mentioning earlier, like what is harmless use, what is malicious use? How do you quantify after which attempts are harmless using AI? Do you consider it malicious use? And also developing a policy that stands the test of time because we might develop a policy that covers the use cases that you're seeing right now. But you might find out in two months that something new comes up, like a new way of using AI, a new way of, yep, producing scams.

And then you have to have a policy that adapts to and it's scalable and adaptable to new use cases.

David: Yep. Laura, anything to add to that or ethical considerations or Yeah.

Laura: I would just say that I think at least it's, like, kinda speaking from the safety space, the policies themselves and, like, the spirit of the policies, I don't think have really altered that much over the last few years. I mean, like, safety harms like CSAM, terrorism, things like that, that's not really what's evolving, but the way that we have to enforce against those harms has evolved, quite a bit since, you know, we've seen more use cases of AI. So I would say that that's been where a lot of the work has happened and how we identify these, like, different harms and how we're able to take action against them.

David: Yes. That's a good point, maybe the policies haven't changed, but the technology has. So, I guess how do you express policy in such a way that it's, you know, effective that teams can practically use it for enforcement? I guess this is a question for if anyone has some thoughts on, you know, ways to craft policy when we're dealing with a technology and a sort of a rapidly changing space.

Tom: Yeah. So I can talk about that a little bit. So one of the challenges that you can face in some, you know, think now in certain risky products is that you can get a lot of regulatory changes or you get new products or you can get new versions of products that show up. And so the beauty of the agents is that you can get scale. Right? The challenge of the agents is that you have to explain everything.

Right? So again, go to the well read intern. They don't know your business and, you know, they don't understand what you want.

So you want your agents when you're thinking of scaling if you want your agents to be very good at finding things, but then you don't want it then you don't want them to be very opinionated. Right? They you want your agents to refer back to your policy to make a decision about whether or not something needs to be enforced.

Right? And that and just the trick there is, you know, building very capable agents of finding things. And by finding things, you know, Ted will have some examples of, like, how challenging things are to be found sometimes, such in recall and and and such. But but but often often things are okay if there's the right text with an image, for instance.

Ted: Think of cacti, you know, San Pedro cacti. Right? It's fine if it's not for human consumption, but you need the image and the text. So you need something that can find that. And then you also need to be able to parse out for IP infringement and various other things. So you need these capable agents, but then you need to express your policy in a granular way so that the agents know what to do.

And so that's the trick. And I think what Irina was saying too is, like, if you can't explain to yourself or to a person, like, what you're looking for or what the difference is, there's no chance an agent will get it. Right? So that part of the process is to be able to describe what it is that you're seeing and then in such a way that agents can if you have an agent, you know, a bunch of agents are running for the scale, that they can interpret it. And one of the beautiful things about Gen AI and agents is that you can actually ask an agent if it understands what you're saying.

So you can ask an agent or develop a prompt by saying, okay. Now ask me questions. Like, this is what I want. Ask me questions.

What are you unsure about? And so as you develop these agents, you can figure out how to best speak to an agent so that the agent can understand what your policy is. But the key is to have it, and there's there's techniques called RAG and other things that that you might have heard of, which is allowing context to be provided so the agents can stay the same and run, but but you always can change your context with RAG and other forms of that so that the agents don't have you don't have to keep retraining them, but they just keep referring to a policy. So you can have a very dynamic policy that's granularly expressed that agents understand as long as your agents know how to find stuff.

David: Anyone else?

Laura: I would also just add that I think that it the the sort of use cases differs per harm type and so there are definitely some harm areas such as like pornography, CSAM, like often like the visual based, harms that is much easier to identify and we can really leverage, AI to, you know, almost like prevent these things on upload and kind of leverage all these different types of of tools, whereas some of the harms such as like, you know, think think about things like human exploitation or, things that have a much more nuanced set of signals that might differ per region is is much harder when you think about it's it's difficult to train people on those things. It's really hard to also, you know, train an AI agent on that as well. So that's something that I think a lot of the work will focus on going forward.

Ted: There's only three mics before Bill. And the one thing we have found is we put our AI agents to work and put our automated systems to work. These agents, like Tom was mentioning, they're well trained in terms of it, but they don't do nuance well. And in the platform space and in the payment space, there's a huge problem with miscategorization of products. And sometimes that's due to bad actors.

You know, because the way a product is categorized is gonna determine how much scrutiny that product gets. Many platforms charge different prices for different categories of products. And that's true in the payments industry. It's true in the platform industry.

Or like the, you know, retail platforms and whatnot. And sometimes it's as simple as Viagra being sold as a book. You know, books don't get much scrutiny. So prices are lower to list books.

So you'll see Viagra being sold as books. We see firearm parts being sold as automotive parts. That's fairly easy, but, when you start talking about regulatory categorizations, you know, I'm an attorney by trade. And, you know, often as an attorney, your answer is, well, it depends.

AI does not do well, it depends very well. So if you look at, you know, often we're looking at what, say, the difference is between a cosmetic and a medicine, or a cosmetic and a medical device, or a psychoactive plant and a non psychoactive plant. Often it'll hinge on some very nuanced regulations and nuanced interpretation of regulations. For example, if you see those, fat dissolving injectables, if you go on YouTube or TikTok, you'll see the videos where they inject the stuff into a slab of bacon, and it eats all the fat out and leaves the red parts behind.

In the US, those are classified as a drug, and they're an unregistered drug. And they've sent out multiple warning letters to various platforms warning them against selling In Sweden, it's classified as a drug, and they've sent out warning letters. But here in the UK, the General Pharmaceuticals Council has sent out plenty of warnings about these things saying, hey, these are dangerous. They start eating fat throughout your body.

You don't know when it's going to stop. But, so far, the MHRA has not classified it as a drug. They've not classified it as a medical device. They're pretty much a gray area right now.

So if you're looking at some of these products, you have to decide how they're gonna be classified. And the AI kind of struggles with that because these classifications depend on intended use. And so you're looking at, you know, not so much how it's classified on the site, but also what other evidence is there. And so that's an area where AI kinda stumbles.

And it's an area where we are finding more and more, the line is crossed where our system freezes up and it has to involve a human being. You're getting to that point where, you know, it's pretty easy to say, hey, this is a switch for a Glock pistol. It should not be in, you know, the automotive section. It's harder to say, hey, this product is actually not cosmetic.

It's a medicine because it's making these claims even though it claims it's a medicine. So that's where the AI kind of stumbles on the nuance.

David: Good. So we've just naturally started transitioning over into how AI is being used as an enforcement tool and how it's used to help enforce policy. Tom, do you wanna and you already touched upon this a little bit. Do you wanna express any bit more just about how we're using AI tools to enforce how some of this technology works or anything like that. I feel like you covered it pretty well.

Tom: Okay. Cool. I think I covered that.

David: Yeah. Okay. Good. Good. You said that, you know, you mentioned that, you know, you talked a little bit about AI agents and them being used as a monitoring tool. You know, how should we think about getting the best value out of them as far as, you know, approaching them, making use of them, and so forth?

Tom: Yeah. The way we look at it is, you know, we build systems that we ask the agents to be sure of.

And if they're not sure, which is a real trick, sure to make sure that we're involving a human in the loop. Right? So we have a lot of analysts, and those analysts are being fed what we call the risk signals or or cases. You gotta look at this. You need to look at this. And and a lot of a lot of these, you know, you know, for us, there are things that maybe we haven't seen. Like, agents aren't gonna be good at new things.

They're good at recognizing existing patterns.

But but we all know in fraud, it's, you know, as soon as as soon as someone detects some counter pressure, they'll just, like, take two degrees step to the right, and then they'll just make a little change, and it'll it's just enough to defeat whatever is is now put in place to to ward against that fraud. And so those new things come up and so what you want is that this is not where you want hallucinations. You don't want your agents to say, I got it. I know what to do with these things. You want them to say, I don't know what this is. And then and then it can go into the, you know, the pool of analysts that we have you would have.

And then the analyst can then classify and say, oh, this is an interesting pattern. This is now something new, and here's how I can set policy against it. Here's how I'm going to tweak this. Here's a new evasion tactic. And, again, you can use these, you know, the rag techniques and other things to inform the agents of a new evasion tactic or a new product, and it's really this classification. Or, hey, there's a regulation change.

Right? And and so, you know, it's difficult sometimes to propagate regulatory change across analysts.

But if you have that baked into a policy, then your entire agent pool picks it up just like that and it'll start enforcing right away. But it all kinda hinges on how you wanna express that policy and and just, you know, what you wanna do with that and the changes in terms of your risk posture.

So it's important to understand, like, you need to always talk about governing the agents or monitoring the agents. That's generally true, like, in call centers even where you don't generally have people trying to call into a call center and try to defeat the agents. You know? It just doesn't happen. People are there for a reason. But, boy, is there a lot of fraud in our world. Right?

People are actively trying to defeat all the mechanisms. And so you just need, like, monitoring in call centers, but you need even more monitoring for people that are where you have a lot of bad actors.

So it's all about monitoring, training, finding new things, expressing new rules and and and policy. And so it's just an ongoing battle.

David: You mentioned wanting a, like, a neutral agent saying, like, I don't know if it comes across something. What's the danger of having I've heard you mentioned this before, having, like, an opinionated agent?

Tom: Well, there's two problems with opinionated agents. You know, one is hallucinations. Like, oh, I know what this is. It's good when it's clearly not good.

The other part is that it may be bad and it could be objectively bad. Like, I've found these things, but, you know, like, you're in your business, there is, you know, here's what I found. Now here's what I wanna do about it. Right? So call that your risk posture. It's like, well, how aggressive do you wanna be in enforcing these things?

Right? Because, you know, you only have so much bandwidth of enforcement. It's like, do I wanna enforce everything? Probably not. Right? Because some things are gonna get you in a lot of trouble, some things are gonna get you in a little bit of trouble.

And so you have to make those trade off decisions. And so what you don't want are your agents making all those decisions because then effectively, the agents have expressed your policy and your and your risk posture. And it's generally gonna be very conservative because, you know, agents only know one thing. They're very literal. Right? So when you think about your policy, you could say, want you to find everything. Now I want you to refer to me and my policy to say whether or not I wanna take action on that.

Right? And then you can change whether I wanna take action or how aggressive I wanna be in the expression of your policy.

Okay. So you don't want your agents to make those decisions because that means you probably won't be selling anything.

But you want them to refer back. So that's the problem. You hallucinations and then kind of an over aggressive posture.

David: Did you want to add anything?

Ted: Oh, I just wanna say, like Tom was saying, it's almost like the line from Yates. The, you know, the worst of us are full of passionate intensity. And then, you know, the AI does not do nuance well. And it just does not, it's not a reasonable person.

If you look at the standards, you know, courts around the world use, especially in the US and the UK, for regulated industries, often it'll hinge on, you know, intended use of a product, intended marketing or intended targeting of a product into which jurisdiction. And a lot of time, it'll hinge on a reasonable person's standard. That's what the courts in the US use. That's what the FDA uses.

That's what the MHRA uses. You know, what would a reasonable person think this product is gonna be used for? And the reason they use reasonable people is because often you cannot, you know, add you know, just make it a binary decision. You can't just say, you know, this amount of information is worth this many points.

This is worth this many points. So when they give, like, jury instructions into the US and they're trying to say, alright, is this product intended for human use? For even though, you know, the seller says it's not intended for human use, was this product really intended for human use? What to tell the juries is, your decision You have to make your decision based on what a reasonable person would decide based on all the relevant information.

It's not add up this, add up that, add up that. It's what a reasonable person really says, you're looking at this website or looking at this listing. What are they really trying to do? And that's where AI kinda kinda stumbles.

AI is unable to be a reasonable person. You know, it's like your grandmother thinking a bong is a vase. You know, you will look at it and say, that's not a vase. That's a bong.

Whereas your grandmother, the AI, says, oh, it says vase. It says vase. Like, there's flowers in it. It's a vase.

It's obviously a vase. So, you know, we see that a lot like, you look at, think Tom mentioned, you know, something like the psychoactive cacti where certain cacti, the San Pedro cacti, it carries mescaline, you know, just like like peyote and whatnot. And on one hand, it's illegal to sell if you're selling it for psychoactive use, but it's also one of the most popular ornamental plants sold in the United States. It's sold on every Home Depot, every website, it's sold at every garden center.

So when you're analyzing it, you have to look at it and say, okay, how are they really selling it? Is this guy just selling a cactus to put around your house? Or are they kinda hinting that you're supposed to eat it to get high? So you have to kind of, and AI struggles with that because it looks for keywords.

And the word the, you know, the use of binary keywords, I mean, that's an anachronism. You know, ten years ago, we had keywords that were binary.

You know?

If this was here, it triggered this. But you have to look at the whole picture because the goal is to reduce seller friction and to reduce the number of products you're actually taking off the platform. And that's where, you know, the AI kinda struggles with that reasonableness. Where it's not a reasonable person.

Tom: Yeah. And if you're wondering, Ted does do all of the product testing.

Ted: Yeah. I do.

David: He has a ball in the court.

Ted: Yeah. Exactly. Yeah.

Irina: I wanted to add a couple of words from a policy development perspective, like, to kind of summarize what they shared, but also, yeah, to add it yeah. To add the policy development layer that I think it's important to develop. When you develop a policy, not just to define what is allowed or prohibited or yes or no, you have to also define the gray areas and what are the edge cases. I mean, we try to do that, me, my team.

But, yeah, this should be the standard. Let's say, like, in which cases you are allowed or it's expected that you don't take a decision and you escalate, and then you have to also have an enforcement and moderation, a layer, an in build layer, whether it is, like, human moderation or agenda moderation where they can say, okay. This is, like, not my decision level. I need some support to make this decision.

And I think this is how you mitigate the risks of some of the risks of making incorrect decisions.

David: Yeah. Those edge cases are the trickiest part. Yeah. Laura, do you have anything to add? Is Meta using AI monitoring tools and Yeah.

Laura: No. I I would say we face a lot of the same issues basically that, when you have those gray area cases, making sure that there's a way to articulate that those need to be escalated to, like, a subject matter expert for review is is always gonna be a challenge because, like, the nature of gray areas is that there's not necessarily, like, one type of gray area. And so I think this is just sort of a perennial problem that most platforms face.

Tom: Yeah. I'd I'd say almost to your analysts are are there to de gray the area. Right? So, like, when you when you find those gray areas and then eventually, you can make it a little more black and white, but but the gray area is where the analysts will live.