AI Wrote the Scam Text, and it is Better Than the Last One

A payment request arrives with the right sender name, the right project and a plausible reason for the change of bank details. It gets read twice. Nothing looks wrong, because there is nothing wrong with it to find.

Most advice about spotting a scam says to look at the writing, check the spelling, watch for the odd phrasing, see whether the greeting fits, and so on. That advice worked for twenty years, and it worked for a reason that had nothing to do with detecting dishonesty. Scam messages were written at volume by people operating in a second or third language, and the errors were a by-product of cheap production.

The FBI stated the consequence in a December 2024 warning, noting that generative AI tools “correct for human errors that might otherwise serve as warning signs of fraud”.

The signal that advice depends on has been removed at the source. 

What are AI scams?

An AI scam is any fraud where generative AI produces or personalises the deception. It can be text from a language model, cloned voice, synthetic video, fabricated documents.

The crime is unchanged. What changed is the unit economics, which is the part that should interest you, because your controls were sized against the old ones.

A team at Harvard Kennedy School measured it. Generic scam mail achieved a 12% click-through rate against their participants. Emails written by human experts using established persuasion techniques reached 54%. Emails researched, written, and sent by an automated tool, with no human in the loop, also reached 54%. Cost per target, including reconnaissance: roughly four cents.

Spear phishing used to be rationed. Research time only paid off against a valuable target, so most of your inbound was generic and most of your workforce was protected by their own unimportance. That is no longer true. Every person on your org chart is now worth a tailored approach, and the org chart is public.

One implication for how you triage: A message of a quality that once indicated a well-resourced actor no longer tells you anything about attribution.

Types of AI scams your controls need to survive

Business email compromise: Still the costliest category. The FBI’s 2025 Internet Crime Report puts losses above $3 billion. No payload, no malicious link, nothing for a gateway to match on. The control is your payment approval process, not your mail filter.

Voice cloning: Used to stage urgency, or to clear telephone-based verification. The same report was the first in twenty-five years to break out AI as a category, logging around $893 million, and the Bureau notes this understates the position because victims rarely recognise AI involvement.

Deepfake video: Microsoft’s 2025 Digital Defense Report recorded a 195% global rise in AI-generated identity forgeries, now convincing enough to defeat liveness checks. If any approval in your environment rests on seeing a face, that approval is weaker than your documentation says.

Service desk impersonation: In both directions, to obtain a credential or a reset.

How to prevent AI scams when the message is unreadable

Accept first that you cannot detect AI-generated text by reading it, and neither can your staff. The Harvard researchers concluded that what separates an excellent fraudulent message from an excellent legitimate one is mainly the sender’s intention.

So move the judgement off the writing and onto the request. Four questions worth embedding in process rather than training:

• Does this move money, change payment details, share a credential or grant access?
• Is it applying time pressure, and would delay actually cost anything?
• Is it routing around the normal process?
• Does it match what this person normally asks for?

And four controls that hold, because none of them depend on anyone judging a message correctly.

• Verify through a channel you established, never one supplied in the message. This is the single control that survives a cloned voice, a spoofed sender and a live deepfake at once.
• Use phishing-resistant MFA, passkeys and hardware keys. SMS codes are relayable in real time.
• Bank details, payroll, and credentials must be confirmed by a second person through a second route.
• Assume the attacker has the org chart. Familiarity is not authentication. If your training implies otherwise, that is now a liability.

None of that is new to you. What is new is that these have moved from good hygiene to load-bearing, because the perceptual layer above them has gone.

Where the residual exposure sits

IBM’s 2026 Cost of a Data Breach report has phishing as the most common initial attack vector for the fourth consecutive year. Its voice and SMS variants carried the highest average breach cost of any initial vector, at $5.29 million. The most expensive way into an organisation now arrives on a phone.

Your stack was built around email and the corporate network. A lure delivered by text message, QR code, messaging app or browser link lands on a device with no gateway in front of it deciding what is allowed through, on the endpoint most likely to be personal, checked in a queue and read quickly. You probably have MDM there. MDM manages configuration. It does not inspect what arrives.

There is a second problem, and it applies to the tooling you do have. Reputation and blocklist approaches lag a destination generated once for one recipient. Corrata’s own detection data shows the decay rate: across customer mobile fleets, 69% of the AI service domains observed had never been seen before. Those were legitimate services, not attack infrastructure, which is what makes the point cleanly. A list that cannot keep pace with software that is not hiding will not keep pace with a lure built to be new.

Corrata inspects traffic on the device itself, at the network layer where traffic from every app converges. Coverage does not depend on which channel the lure arrives through. Detection works from what a connection is doing rather than from a catalogue of what has been seen before, and it reads traffic characteristics rather than message content, so the workforce is covered without anyone reading their messages.

Conclusion

The messages improved because a cost constraint disappeared. Your perceptual defences were built on that constraint, and no amount of additional vigilance restores a signal that no longer exists.

Two things follow. Your procedural controls and AI governance have to carry more weight than they were designed to. And the surface where the most expensive of these lures now lands is the one you have the least visibility into.

Book a demo to see what is currently reaching your mobile fleet.