Welcome to Kernel Panic!, a weekly newsletter by Lily Hay Newman and Matt Burgess from inside the new world of privacy and digital security. To receive this newsletter in your inbox each week, sign up here.
Even as AI supercharges digital scamming and cybercrime, it’s also being recruited in the fight to protect potential victims. Governments around the world have consistently struggled to address the global crisis of online crime, grappling with the limitations of attempting to crack down on criminals beyond their borders. This has inspired alternative efforts, like using automation to attempt to go after scammers en masse—perhaps by spamming the spammers to squander their resources. Now, AI platforms are turning these types of experiments into real tools in the fight against cybercrime.
For the last two years, the Australian company Apate, named after the Greek goddess of deception, has been building a system that diverts phone scammers onto calls with AI bots that are trained to keep conversations going as long as possible without, of course, ever actually falling for the scam. The goal is to give scammers enough hope that they stay on the phone, while also collecting intelligence about the scams they’re running.
“What we really like to think is that we’re building the perfect victims for scammers,” Dali Kaafar, the founder and CEO of Apate, tells us. “A minute that a scammer is talking to a bot or an agent is a minute where you’re probably saving hundreds, if not thousands of possible people being reached out to by that exact same scammer,” he says, referencing the ability for scammers to use automated dialing tools.
Kaafar says Apate’s platform, which is used by banks and supported by telecom companies, has around 350,000 bots. They not only pick up calls, but also infiltrate scam chat groups online and respond to text messages with similar goals of frustrating scammers while collecting intelligence from them. He says the company has collected more than 250,000 pieces of information about fraudsters in real-time, from scam URLs to money mule accounts and bank details.
The bots have different personalities, language skills, and profiles, Kaafar says, so scammers hopefully don’t detect that they’re conversing with an AI. Like real people, “sometimes [the bots] do have WhatsApp, sometimes they don’t. Sometimes they pick up the phone, sometimes they just actually hang up on the scammer saying, ‘I’ll come back to you later,’” he says.
But are they convincing? Kernel Panic tested the tool using a demo version where users play the role of the scammer and talk to one of Apate’s AI “victims.” The personas are designed to express a healthy amount of skepticism but leave enough openings for scammers to continue to try to win them over, a dynamic that we could immediately feel and found intensely frustrating.