How AI-driven identify fraud is causing havoc

11 Feb 2025
 • 
,
4 min. read

Artificial intelligence (AI) is transforming our world in ways both expected and unforeseen. For consumers, the technology means more accurately personalized digital content, better healthcare diagnostics, real-time language translation to help on holiday, and generative AI assistants to enhance productivity at work. But AI is also used to help cybercriminals be more productive, especially when it comes to identity fraud – the most common fraud type today.

Over a third of banking risk and innovation leaders in the UK, Spain and US cite their biggest challenge today as the rise of AI-generated fraud and deepfakes, making it the number one answer. So how does AI-powered fraud work and what can you do to stay safe?

How does AI-driven identity fraud work?

Identity fraud refers to the use of your personally identifiable information (PII) to commit a crime, such as running up credit card debt in your name, or accessing a bank or other account. According to one estimate, AI-driven fraud now accounts for over two-fifths (43%) of all fraud attempts recorded by the financial and payments sector. Nearly a third (29%) of those attempts are thought to be successful. So how is AI helping the cybercriminals?

There are several different tactics we can highlight:

• Deepfake account takeovers (ATOs) and account creation: Scammers are using deepfake audio and video likenesses of legitimate users to bypass the Know Your Customer (KYC) checks used by financial services companies to verify customers are who they say they are. An image or video of you is scraped from the web and fed into a deepfake tool or generative AI. It’s then inserted into the data stream between user and service provider in so-called injection attacks designed to fool the authentication systems. One report claims that deepfakes now account for a quarter (24%) of fraudulent attempts to pass motion-based biometrics checks and 5% of static selfie-based checks.
• Document forgeries: There was a time when fraudsters used physical document forgeries, such as faked passport pages, to open new accounts in the names of unassuming victims. However, they’re more likely today to do so digitally. According to this report, digital forgeries account for over 57% of all document fraud – a 244% annual increase. Scammers will typically access document templates online or download document images stolen in data breaches and then alter the details in Photoshop. Generative AI (GenAI) tools are helping them to do this at speed and scale.
• Synthetic fraud: This is where scammers either create new identities by combining real (stolen) and made-up PII to form a completely new (synthetic) identity, or create a new identity using just fabricated data. This is then used to open new accounts with banks and credit card firms, for example. Document forgeries and deepfakes can be combined with these identities to increase the fraudsters’ chances of success. According to one report, 76% of US fraud and risk professionals think their organization has synthetic customers. They estimate that this type of fraud has surged 17% annually.
• Deepfakes that trick friends and family: Sometimes, fake video or audio can be used in scams that trick even loved ones. One tactic is virtual kidnapping, where relatives receive a phone call from a threat actor claiming to have kidnapped you. They play a deepfake audio of your voice for proof and then demand a ransom. GenAI can also used in these efforts to help the scammers source a likely victim. ESET Global Security Advisor Jake Moore gave a taste of what is currently possible here and here.
• Credential stuffing (for ATO): Credential stuffing involves the use of stolen log-ins in automated attempts to access other accounts for which you may have used the same username and password. AI-powered tools could rapidly generate these credential lists from multiple sources of data, helping to scale attacks. And they could also be used to accurately mimic human behavior while logging in, in order to trick defensive filters.

What’s the impact of AI-based fraud?

Fraud is far from a victimless crime. In fact, AI-powered fraud can:

• Cause major emotional distress for the individual that’s defrauded. One report claims that 16% of victims contemplated suicide as a result of an identity crime
• Make scams more likely to succeed, eating into profits, which forces companies to put their prices up for everyone
• Impact the national economy. Lower profits mean lower tax receipts, which in turn mean less cash to spend on public services
• Undermine public confidence in the rule of law and even democracy
• Undermine business confidence, potentially leading to lower levels of investment into the country

How to keep your identity safe from AI-driven fraud

To combat the offensive use of AI against them, organizations are increasingly turning to defensive AI tools to spot the tell-tale signs of fraud. But what can you do? Perhaps the most effective strategy is to minimize opportunities for threat actors to obtain your PII and audio/video data in the first place. That means:

• Don’t overshare information on social media and restrict your privacy settings
• Be phishing aware: check sender domains, look for typos and grammatical mistakes, and never click on links or open attachments in unsolicited emails
• Turn on multifactor-authentication (MFA) on all accounts
• Always use strong, unique passwords stored in a password manager
• Keep software up to date on all laptops and mobile devices
• Keeping a close eye on bank and card accounts, regularly checking for suspicious activity and freezing accounts immediately if something doesn’t look right
• Install multi-layered security software from a reputable vendor on all devices

Also consider staying aware of the latest AI-powered fraud tactics and educating friends and family about deepfakes and AI fraud.

AI-driven fraud attacks will only continue to grow as the technology gets cheaper and more effective. As this new cyber-arms race plays out between corporate network defenders and their adversaries, it’s consumers that will be caught in the middle. Make sure you’re not next.