LegitScript Marketplace Risk NYC Panel Recap: AI Blind Spots: The Gray Areas of Risk, Policy & Regulation

Transcript

Kim: Awesome. Well, it's nice to see everyone. Thank you for joining. Hopefully, everyone's well fed and well caffeinated to talk about AI and the gray areas of risk, policy, and regulation.

I'm Kim Mintemeyer with LegitScript. I lead our account management and customer success organization. And with me today is Tom Cook, our chief product and technology officer at LegitScript, as well as Ted James, associate general counsel at LegitScript. So before we talk about the use of AI in blind spots, I think it would be good to set some definitions because the term AI is very broad. I know it was talked about a little bit earlier today. But Tom, can you walk through the type of AI we're going to be talking about and very broadly how it works?

Tom: Yeah. Sure. So I'm assuming everyone saw the keynote. Most people see that. Right. Okay. So, you know, we'll focus mostly on the, AgenTic side because that's kinda where we are today. And so when what we're gonna be talking about is more workflow automation. So what you saw a lot about this morning was mimicry. And so when you look at LLMs and you look at agents, they're very capable, especially capable of mimicry. Right? And so a lot of that was very capable. When you get to workflows, you have to think about it slightly differently. And so the current state today is best captured by saying an agent is like a well read intern.

So that's kind of the general capability of an agent. And so if you keep that mental image in your head while we're talking, some of these things will make more sense. So when you think about what you need to do with a well read intern, like, well read means it's based on an LLM. It's like read everything in the Internet. Right? It's read everything. So that's great. And it's got capabilities. Even now, like, you can look at agents and they're multimodal, which means they can look at images and text together so they can get that context. But so they're kind of capable, like, think Ivy League agent, but they're new to everything. They need a ton of context, and they need very explicit instructions. Right? So when you start to think about that agent and you start thinking about policy, you know, Ted, you have some examples. So we're gonna talk about just like where and, you know, and I would encourage everyone to look into agents because they're very powerful, but where do they start to like intersect with the gray area in policy?

Ted: Yeah, so I work in research department and what we're doing all day long is we're helping our clients analyze their risk. We're trying to analyze their, regulatory risks. We do products and services. We, analyze their regulatory risk, their enforcement risk, their, litigation risk, and the reputational risk in general. And so we use our AI tools to monitor their platforms. We also monitor, merchants for the payment platforms and whatnot. So we're constantly going through and using the AI to surface that. And like you said, it's like a very well trained intern, but the moment you start getting into gray areas, the moment you start getting into where decision making gets a little more confusing is where we run into issues. The days of, keyword lists are five years behind us. You know, when you had definitive keywords list, you could surface things with keywords and you could have a definitive decision. That we still do quite a bit of that, and that's the low hanging fruit. I mean, we seldom see products, you know, marketed as a BAW or products marketed as LSD. You have to go into, like, when you surface these products, you have to spend more time analyzing what we're seeing. So one of the things we do is we, you know, under the DSA, very large platforms in Europe are required to consult, databases of illegal goods or services.

One of the databases you have to consult is the database of recalled products. The customs agents in Europe post regulatory notices about product recalls, unsafe products and whatnot. The problem, one of the problems we run into is the pictures they post are terrible. The iPhone cameras, smartphone cameras killed the entire photography industry, but for some reason, customs agents are unable to take good pictures of the products they wanna recall.

So one of the things we come up with so they spit out the pictures like you see above. You know, it's a bag of balloons, and our technology is able to go out and match several data points in these images, but you still need a human being to come along and confirm that it's actually the same product. Because by the time these products hit the platforms, they're not being sold with a bag of balloons like the picture you see there. I mean, that wouldn't sell any products. And they don't show barcodes, they don't show anything like that. So one of the things we have to do is this is the area where as we do these safety recall works, we find we're having to kick it back to the humans more often than not. Same here. I mean, here's a picture of these are little plastic monkeys, I think they were. But it was at the same time, like, we were able to surface large amounts of listings that match several of the data points in the image, but we still need the humans to come along and help with that work.

Tom: Yeah. And you'd also then talk about, like, the reasonableness. Like, so often when you're now looking at risk, like, where does that go? How would you talk about it?

Ted: So like I said, the days of keywords being definitive are behind us. And you know, two areas we look at, you know, when you look at the way the FDA or even the FTC, analyze products, they're in the DOJ analyze products, one of those standards they use is the reasonable person standard. What would a reason, you know, in jury instructions and in whatnot, you know, what would a reasonable person conclude this product's intended use is, if they had all the information available, and feel free to disregard anything they actually sell or merchant is telling you overtly. So you look at we see a lot of products marketed for research use only. And the problem is AI is great, like you mentioned. But you have to program it. It's like an intern. AI is not a reasonable person. It doesn't have a lot of common sense. So when we start looking at these products, we look at products marketed for research use only. We have to look at all the information, and weighing it, there's no specific formula for how to weigh it at any given time.

It's kind of like, you know it when you see it, and AI is really bad at you know it when you see it. You look at, say, San Pedro cactuses, probably one of the most common ornamental plant, you know, in the western US at least. It's sold in every nursery across the country. But it does contain mescaline. And it's really easy to identify San Pedro cactuses on a platform. I mean, RAII does it all the time, but AI or San Pedro cactus only becomes problematic when it's marketed as a psychoactive. I mean, otherwise, it's just a fine ornamental. You can sell them all day long. But the problem is, you know, our technology can surface when it's overtly marketed as a psychoactive. Psychoactive. This contains mescaline. Take it as a psychoactive. But it's not so good at looking for those non overt clues. It's not as good as finding, you know, background imagery. It's not as good as surfacing to give the vibe of a merchant.

So we're looking at a merchant, you know, maybe the merchant does mention that it contains mescaline, but it's intended as an ornamental plant. We need to be able to parse that, and that's where the common sense, what would a reasonable person think this product's being sold as, And that's where we still need the human beings. We still need reasonable people.

Kim: Shifting gears a little bit, we see some platforms using AI for regulatory research, but that can be very problematic. Can you speak to that a little bit, Ted?

Ted: Yeah. I mean, so one of the products that we run, we analyze merchants, for our merchant monitoring portfolio. We we run AI to start analyzing what they're doing with, you know, the practice business practices. But where we really run into trouble is with AgenTeq AI, having that agent then go out and do regulatory research. And the biggest problem we run into is, especially in the gray areas, is the information ecosystem is being completely poisoned by blogs, by, the merchants themselves, by their own websites with the information they're putting out. And even when you try to restrict the AI agents to, only consulting official governmental sources, it still becomes a kind of a, like an oral boros of information that's usually not correct, or very often not correct. I come from a financial services background, and for example, there's a new business practice that's been around for a few years. It's for personal trading accounts, financial, trading platforms. It's called the prop trading desks. I don't know if you guys know proprietary trading, and what they're doing is it's really a game. You deposit a certain amount of money, and the promise is that if you do well with a certain amount of money you deposit, you're gonna get a real job. But no one ever gets a job.

These companies all market themselves as proprietary trading desks, but they're not. And what they're doing is no relation to the regulatory definition of proprietary trading. The problem is, if you try to go research this, every one of these companies, and there's hundreds of them, probably have thousands of blogs out there assuring you that it is the type of proprietary trading that's regulated, and what they're doing is perfectly fine. So, we see it same with gambling. If you try to go out and do a lot of regulatory research in gambling, a lot times you just keep hitting the same blogs, the same blogs assuring you that whatever this one merchant is doing is perfectly legal or not regulated at all. So that's what we run into. So you really need a human being to actually do that research and kind of have the experience in the space to be able to know where to research and how to interpret the research.

Kim: We touched on a little bit in some of these examples about how a hybrid AI human model can be the most effective. And again, you touched on some examples here, but what are some other things that we should be thinking about having AI doing versus what human should be focused on?

Tom: Yeah. So the way you know, if you, again, you think the well read intern in here, you really want the intern, the agent, in this case, to do what it does very well. So you can surface a lot of this data, but you don't necessarily want it to make the final judgment. Right? So this is where, like, from a policy perspective, you'd want the agents to surface what it can find and then refer back to your policy. Right? So you don't if you make a if you kinda tip over to making your agents smarter and smarter, you're really delegating your policy because they'll start deciding. And so, again, think about mimicry and learning. You know? It's like the policy will actually be driven by the agents and not by your stated policy. And so there's a balance that you need to strike, but what these agents will be really good at is pulling tests down and images. And if it's very clear, then they can make a decision just like, again, an intern.

If an intern would see something that's clearly violative, then it could make a call or clearly not violative, then it could make a call. If there's a gray area, you'd really want the agents to then say, I don't know, human review, and then kinda send it up. The agent needs to be explicitly told to go do that because, generally, agents are not really good at gauging themselves and their uncertainties, so you have to actually track some of that. So when you wanna look at AI and scale and, you know, and transaction speed, you know, it's really good at surfacing patterns and data. You know? And, again, like, if your policy is clear in these sense, then it can actually make a call. But, generally, this is the balance that you wanna strike. Right? You wanna have a do what only can do. You don't wanna get too smart so that you can really set your risk tolerance explicitly.

Ted: And I think that's one of the things we've not struggled with, but, you know, maybe struggle is the right word, is you wanna have the agents be confident, but not too confident. You know, AI is very full of passionate intensity. Like it knows what it knows and it's gonna assure you of it. But having the AI, just like an intern, know when it's time to kick it out to human review. I mean, goal is not to eliminate risk. Our goal is to help our clients know what their risk is, analyze it, and mitigate it. We're trying to help everyone find their you know, exactly where is your risk right now. And it's dynamic for us as well. Like, you know, of the things we track, everyone tracks it, like enforcement actions around the world. You're constantly tracking the enforcement actions from the various regulatory agencies. At the same time, you're kind of trying to see, mean, use the cliche, where the puck is going. You want to know, you know, what have they been really cracking down on lately?

What are they likely to move on to next? Like, you know, you look at, like, FDA this last year has really been started cracking down on the so called legal highs. You know, they went after seven Oh, they went after Tia Neptine, they cracked down on people selling poppers, they cracked on people selling nitrous oxide, and all their warning letters mentioned something about legal highs being sold online and in gas stations and how they kind of felt like they missed the boat with, fentanyl. So you're kind of trying to look for a pattern and we want to be able to work with our product and have it tell us what it doesn't know when it doesn't know something, but also be looking forward and trying to see where the next where the risk is really going. And so that we need it to be very dynamic. So we need it to be both, secure in knowing what it doesn't know and dynamic at the same time because we're constantly adjusting our risk parameters as things change.

Kim: And how does a company's risk tolerance and risk judgment inform its use of AI? And conversely, how can AI increase a company's risk tolerance?

Tom: Yeah. So that's a great question. So again, kinda going back to the like, if the agents are good at surfacing and the outcome is clear. So I can imagine you're, like, operationalizing your policy is kind of what we're talking about. So in these certainly in regulated industries where there's a lot of gray, and you have to look at ingredients and you have to look at products and you have to look at jurisdictions and you have, like it's kind of a vast policy statement, though, when you wanna try to operationalize it. So what you want the agent to do is actually go find stuff, then refer back to the list that you have of, like, yeah, we're allowing this, not allowing this, you know, so it's clear. So if it's clear for an intern, it should be clear for an agent. And then that's where you set your risk. So you could just say, like, yes, no, yes, no, or this class of things, yes or no. Then the agent can respond to it or a human can respond to it. Because when you're looking at the operationalization of this, it's transaction speed. Like, it has to go really quickly. There's not a lot of time for you don't want an agent to think too much, and then even the humans can't think that much. It's just they don't have time. So when you think about the expression of your policy for the operations team, this is where you wanna be careful about making sure that the yes, no list is something you explicitly set and you don't let the agents decide. That's where you lose control of your policy. But then if you set this up and and now you wanna go into, now thinking top line, you know, you're getting the pressure to say, right.

Like, how do we open up another vertical, or how do we dip our toe in this? Well, you can apply the same thing. If you have that same model, you can go through and now say, okay. Like, we're gonna open this one up. Here's my yes, no list. Right? And it's informed by jurisdictions and policies and enforcement and future enforcement. You know? So it's a yep. When you look at the regular regulatory industry, that's kind of like a compliance towards, but the risk is often grayer. Right? It's like, yep. Right. But, you know, it's not being enforced. So how much teeth is in that? And so when you look at the business and opening it up, you wanna be able to express that risk as the yes, no list for your team or for your, like, the agents or, you know, your individuals. And so if you have that kind of model in place, then when you go into another vertical, then you'd have a pretty small list of allowed things and then a big list of no things. You know? So as these agents or the people are looking at it, they know how to respond. Then when you wanna change your risk tolerance, you wanna open it up, your yes, no list changes. Then the beauty about agents is that you don't have to retrain them. They respond to your policy. So if you have your yes, no list and you change it, the agents will pick that up, and then it'll start going. If you wanna change it and be more restrictive, they'll pick it up and they'll go just like people.

So that's how that's kinda like the right model when you think about this. I think I'll just say, like, if the agents, though if you make the agents too opinionated by letting the agents go scrape and learn, kinda look at all the different regulatory environments and jurisdictions, it's going to form its own opinion, and the opinion is gonna tend to be pretty darn conservative. So even if you wanna move and kind of adjust your risk tolerance to be a little more permissive, you're gonna start battling the agents if they're too opinionated. It's always this is gonna be this will be interesting as we go forward because that will be the conflict.

Ted: And I think, you know, like your point being, you don't want them to be content moderation is not, and regulatory risk and analysis risk, is not necessarily binary. It's something that we want to be, it's dynamic. Risk is always dynamic. It's changing at all times.

Know, our clients' exposure to various risks are changing, the risks themselves are changing, The you know, and so we constantly want to be able to have all these agents we constantly want to be checking how we've got them programmed, exactly like what the yes, no list is, and constantly updating, because they don't do well changing really their own yes, no list. And like you said, like our clients, you know, sometimes you wanna explore a more risky vertical. Sometimes you've got to make your quarterly numbers at the end of the quarter. And sometimes you just products that are really problematic, that are regulated, that are prohibited, really carry with a very little risk. I mean, if you're going be selling, you know, knockoff Neosporin or, you know, mosquito repellents, some of that, there's really not a ton of down well, not a ton of downside. I don't how to say it. There's not a ton of risk there, so you gotta evaluate your risks in real time. But suddenly those risks could change, like we saw with the Latisse, the eyelash cream that grows your eyelashes, but can also change the color of your eyes if you're not careful.

You know, the last time the FDA cracked down, I believe was ten years ago. So, and it's, they sell it over the counter at Nordstrom's. They sell it down the street and go pick some up and grow your eyelashes. But, the, but just last year, there was a big, class action lawsuit, your litigation risk goes up. So as we're evaluating these products, and as we have these, we're programming these agents to go out, we want to be able to say, Hey, yes, as the agent goes out and checks the FDA statements and stuff and sees that the stuff, you know, is prescription only or prohibited in whatever jurisdiction, yes, that's true, but really they haven't enforced it, but oh, now there's some lawsuits and now we're going to take that back. So we wanna be able to have these things be dynamic and be constantly monitoring risk in real time. And that's, you know, these AI agents, you gotta program them. They're only, or they only exist, they only operate on existing information. And so we don't want to have them looking back. We want to have them looking forward. And that's kind of what we work with every day is how do we get these agents to look forward.

Kim: And speaking of looking forward, as you think ahead to the next few years, what risks may exist in a few years? Are there risks that are caused by AI or risks that AI will need to address? What are your thoughts on kind of what's to come? Curious from both of you.

Tom: Yeah. My big concern right now is security. You know? So if you have, a bunch of agents that are screening things, you know, there could be there's a lot of prompt injections. So there's a lot of ways to kinda just start spoofing and driving the outcome and the behavior of these, agents. And so it's just you know, I mean, think everyone knows that, but the reality of dealing with it and the implications to either your, you know, your risk posture or your revenue, like because you'll drive it one way or the other. Like, if you get could drive it to be more you know, have a lower risk threshold, which will then impact revenue or it can make it more permissive. You know? So it's the like, do you have control of your agents, and do you know what they're doing? You know? Because we've seen a lot of if do read the papers, there's a lot of the agents have self preservation, kind of signs. So they're exhibiting that kind of behavior. And so as you try to control them, sometimes, if you're trying to shut them down or move, sometimes they're gonna resist that. And so making sure that you know what your agents are doing at all time is going to be a big thing. And then too, you're going to start seeing, if you're thinking about onboarding of products or ads or, you know, merchants or sellers, you know, the that onboarding process is gonna be now governed by agents too. You know? So now you're gonna have agent to agent discussions, which we heard a little about this morning.

As you go forward, it's now it's not just the people that you have to deal with, but it's like the agents who are working as an agent of a person that's now going. And, you know, and then how does those things get negotiated, and what's the behavior there, and how do you govern it? You know, there's, we're early on. Right? There we're early on. We're learning about these things. So it's just you know, it's always the way we look at it is, you know, be curious, test everything. Make sure you know exactly what's happening at all times because, you know, like, we don't really, really know what these things do, at scale. But there's a lot of power there just to have to be going in with eyes wide open.

Ted: Yeah. I mean, for me personally, I mean, one of the things I worry about is, you know, these agents are amazing at surfacing information. I do worry that oftentimes they tap out a little too early, almost creating more work for us on the human side. Because it's amazing how much, you know, how much data they can process and how much it's revolutionized, like, our ability to process large data sets, but at the same time, have to keep in mind that a lot of this stuff operates at the edge of the gray area where you still do require humans. And, I mean, just personally in terms of businesses across the country, whether you're seeing them, trust and safety departments rely more and more on the agents and less and less on the people and reducing headcount, it's going be you're going to have more and more of this data surface and more and more problematic products caught than you would have in the past, but you're gonna have less people to deal with those gray areas. And, that's one area that is certainly of concern.

Kim: We're coming up on time here. So in thirty seconds or so from each of you, curious what kind of final parting words you have on this topic. What do you want people to walk away from this session with?

Tom: Yeah. So I'll just say, like, the, you know, AI has, it's a big hype curve, Gen AI, LI, you know, the like, LLMs and, you know, all of the DMCP's. There's it's advancing a lot, but just keep in mind well read intern and think about what you would do with a well read intern, and then you'll be good.

Ted: And just to go along with that, AI is not a reasonable person. It doesn't have common sense. And that's what we need to figure out a way is to how to get interact with a way where the reasonableness and the common sense is ingrained with the AI process.

Kim: Awesome. Well, thank you both for your insights and joining us today, and thank you all for joining us today. Hopefully, everyone found this informative.