Deepfake Audio and Video, and the Limits of Detection

At 2:10 on a Tuesday afternoon, a retired woman in the invented town of Pinecrest gets a call. It is her grandson, voice shaky. He has been in a car accident, a police officer is on the line, he needs $4,500 for bail by morning, and please do not tell his parents. She hears his voice crack. It sounds like him. She goes to the bank.

The Federal Trade Commission describes exactly this pattern. Scammers can clone the voice of a loved one from a short clip of audio posted online, then call you pretending to be that person. The FTC's advice is plain: do not trust the voice alone, and call the person who supposedly contacted you on a number you know is theirs to verify the story (checked October 2026). The scammer's demands are a tell too: wire transfers, cryptocurrency and gift cards, the payment methods that are hard to reverse.

What the technology does, in plain words

The last row matters because the cheapest fakes are often the most common. A real speech cut to remove half a sentence, or a real clip from a different year, can fool far more people than a generated face.

Scams that use cloned voices and faces

The family emergency call

A call or voice message from a relative in trouble: a crash, an arrest, a kidnapping, a hospital. The caller asks for secrecy and quick payment. The FBI Internet Crime Complaint Center issued a public service announcement on May 15, 2025 (I-051525) about a campaign using text and AI-generated voice messages to impersonate senior U.S. officials, often to reach their contacts. It notes that cloned voices can sound nearly identical to the real thing, and it advises verifying identity, calling back at a number you find independently, never sharing two-factor codes, not sending money, gift cards or cryptocurrency to people you do not know or have met only online or by phone without independently confirming the request, and creating a secret word or phrase with family (checked October 2026).

The executive or boss request

An office manager gets a voice message that sounds like the owner: "I'm in a meeting, need you to pay this vendor today, details coming by text." Or an employee joins a video call with what look like several senior colleagues, all asking for an urgent transfer. A widely reported 2024 case involved a Hong Kong employee of a multinational engineering firm who was reported to have moved about US$25 million after a video call with what turned out to be fake versions of colleagues (reported by news outlets and catalogued by the AI Incident Database; the figure and details come from press accounts, not from a court filing I could read). The lesson does not depend on the exact figure: a video call alone is no longer proof of who is on the other end.

Fake endorsements and investment pitches

A face-swapped video of a recognizable public figure "recommending" an investment or a product. Check the person's official channels. Real endorsements show up on their own verified pages and in news coverage. Lesson 02 covers lateral reading to check the organization behind the ad.

Political and public-affairs clips

Fake audio of an official saying something outrageous, released shortly before a vote. A well-documented example involved robocalls in early 2024 that used an AI-generated voice imitating a national political figure. The Federal Communications Commission responded with a declaratory ruling announced on February 8, 2024 that calls made with AI-generated voices count as artificial under the Telephone Consumer Protection Act (checked October 2026). Lesson 09 covers election information sources and how to check an election claim.

The defense that does not depend on spotting the fake

Because detection by ear and eye is unreliable, the defense that holds up is procedural. It works whether the fake is mediocre or perfect.

Setting up a safe word

The FBI suggests agreeing on a secret word or phrase with family members (checked October 2026). The FTC's consumer page on family emergency scams, as of the version reviewed, does not mention a safe word, so treat the safe word as a sensible extra layer and the callback as the core. Practical points: choose something that cannot be found online or guessed from your social media (not a pet's name or a street); agree it in person or on a call you control; do not text it; rotate it if it was used or shared; tell older relatives and teens alike; and decide in advance what to do if the caller cannot supply it (hang up and call a number you know). The same idea works in an office: a code phrase that the owner and the bookkeeper use before any unusual payment is released.

Making your own voice harder to clone

Perfect protection is impossible, but you can reduce easy fuel. A public video with clear speech is enough material for many voice cloning tools. Consider who can see long voice clips of you, avoid answering unknown calls with extended conversation, and tell relatives and coworkers that you will never ask for money by voice message alone. If you are a public figure or a local official, publish how your office confirms payment or policy statements.

What detectors can and cannot do

Tools exist that claim to tell whether a clip is synthetic. Some give a percentage; some give a label. They are worth understanding because people lean on them.

What they do

• Look for artifacts left by known generation methods: frequency patterns in audio, blending boundaries, inconsistent physiology.
• Check for watermarks from participating generators. Google's SynthID detector, for example, checks whether media carries a SynthID watermark, and Google lists the products it covers (checked October 2026). A positive result is strong evidence the media came from a participating tool. A negative result means only that no such watermark was found. Content from non-participating sources will not be flagged.
• Combine signals for a score that a trained analyst then interprets.

What they cannot do

• Generalize. A detector trained on yesterday's fakes struggles with tomorrow's generator. In Meta's Deepfake Detection Challenge, the top model reached 82.56 percent accuracy on the public test set and 65.18 percent on a held-out black box set of videos it had not seen, and none of the 2,114 participants reached 70 percent on that held-out set (checked October 2026).
• Handle real-world files. A 2025 study, Deepfake-Eval-2024, collected deepfakes circulated in 2024 (about 45 hours of video, 56.5 hours of audio and 1,975 images from 88 sites in 52 languages). It found the AUC of state-of-the-art open-source detectors dropped by about 50 percent for video, 48 percent for audio and 45 percent for images compared with older academic benchmarks. Commercial and fine-tuned detectors did better but did not yet match human forensic analysts, according to the authors.
• Survive compression. Social media recompression, screenshots and re-recordings of audio over speakers wash out the traces detectors look for.
• Explain. A single number does not tell you what the detector saw or why.
• Prove authenticity. A "likely real" result does not authenticate a clip. It means the detector did not catch it.

Why a failed detector result proves nothing

Say a detector flags a clip of a local official as "98 percent likely fake." That is a clue, so go find out why. A clip can be flagged because it was recompressed three times, recorded in a noisy room, a podcast voice with heavy processing, or because the person's real voice was edited. In the other direction, a clip can pass because the fake was made with a newer tool the detector never saw. Neither result can carry a headline. The numbers below show why even a decent detector produces a lot of wrong alarms when fakes are rare.

Process-based checking

Whether a clip looks real or not, work through the same questions. The first four are the SIFT moves from lesson 02, applied to media.

1. Stop. Do not share. Note the claim: who is said to be speaking, where, and when.
2. Investigate the source. Who posted it first? Is the account new, anonymous, or known for clips like this? Search the name of the person and the event in news outlets and the person's own channels.
3. Find better coverage. Was the event public? Look for other recordings by other people, official recordings (a meeting video, a livestream archive, a press statement) and wire coverage. A real public speech leaves many traces. A fake leaves one clip.
4. Trace to the original. Find the full unedited recording. A short clip is often cut from a longer one; the longer one shows what came before and after. Use reverse video search on keyframes (lesson 03) and archive tools.
5. Check the circumstances. Was the person where the clip claims? Calendars, public schedules, travel, and other footage from that day.
6. Look for provenance. Content credentials, when present, record how a file was made and edited (lesson 07). A valid credential from a known camera or news organization is a strong lead. Missing credentials prove nothing either way.
7. Ask the person or the office. A public official's office can say whether they said it. A denial is evidence to weigh against the clip, and it is stronger when backed by a complete original recording or independent footage.
8. Use a detector last and carefully. If you use one, use more than one, treat a score as a clue, and write your finding in terms of process, not the score.

What you may notice, and what that is worth

Fakes sometimes show telltale glitches: lips that do not quite match, unusual blinking, skin that is too smooth, a voice with odd rhythm, lighting that does not agree between face and background, hands that distort when they pass the face. These are worth noting. They are weak evidence, because the glitches shrink with each model release and compression produces similar glitches in real video. In a live video call, a common suggestion is to ask the person to turn their head sideways or pass a hand across their face. That may expose weak face swaps today. It does not work against a good one, and it is not a reason to skip the callback.

The law in Idaho and federally, briefly

Idaho Code 67-6628A, part of the state's election-related provisions, addresses synthetic media about candidates. As of the text reviewed in October 2026, it defines synthetic media as an audio or video recording of an individual's speech or conduct, created with generative adversarial network techniques or other digital technology, that to a reasonable individual appears to be a real event, action or speech that did not occur. A candidate whose action or speech is deceptively represented may seek injunctive or other equitable relief against publication, and may bring an action for damages against the information content provider. A disclaimer in the statute's required form ("This (video/audio) has been manipulated.") is an affirmative defense, with rules for how long and how often it must appear. The statute also says providers and users of interactive computer services are not treated as the publisher or speaker of another's content, and carves out treatment of licensed broadcasters. This is a general description of the text for education, with the statute linked below so you can read the current wording and any amendments. Federally, the FCC's February 2024 ruling treats AI-generated voices in robocalls as artificial voices under the Telephone Consumer Protection Act, which brings consent requirements into play.

Questions and answers

How much audio does it take to clone a voice?

It varies by tool and quality, and it has dropped over time. The FTC says scammers can use a short clip from content posted online. Do not assume that a voice you have posted publicly is safe, and do not assume a short clone is poor.

My relative called and used a name only family knows. Is that proof?

No. Scammers research families through social media, data broker listings, obituaries and earlier hacked accounts. A private fact the caller could have found is weaker than a safe word agreed in advance and kept off the internet. A callback is better than either.

Why should I not just call back the number that called me?

Caller ID can be spoofed, and some scams keep the line open or route a callback to the scammer. Hang up, then dial a number from your own contacts, a bill or the organization's official site.

I already sent money. What now?

Contact your bank or the payment service immediately and ask about a recall or dispute. Report to the FTC at ReportFraud.ftc.gov and the FBI's IC3 at ic3.gov, and keep the phone number, times and messages. Speed matters. Lesson 10 covers reporting and what to expect.

Can I rely on a video call to confirm who I am talking to?

Not for a high-stakes request such as a payment, a password reset or access to records. Treat the call as one input. Add a callback to a known number and a second approver.

Are there free tools I can use on a suspicious clip?

Yes, including the free InVID-WeVerify browser plugin for keyframes, magnifier and reverse search (lesson 03), and the SynthID detector for media from participating Google and partner tools. Use them to gather leads. See the AI detection course at /journal_tools/issues/45-ai-detection/ for the depth on what text and image detectors can and cannot show.

What if the clip is real but out of context?

That is more common than a fake. Find the longer recording and the date. Lessons 03 and 08 cover tracing the original and checking quotes.

Should schools, clubs and offices have a written rule?

A short one helps: no payment, password reset or records release on a voice or video request alone; always verify through a known number; two approvals over a set amount; and report attempts to one named person. Lesson 10 includes a model correction and reporting policy for groups.

Hard questions

If detection will keep failing, is any digital evidence trustworthy?

Evidence that comes with a trail is. A recording from an official meeting, a livestream with an archive, a clip multiple independent people filmed, and media with valid content credentials all have layers of corroboration. A lone clip from an anonymous account has none. The trend is toward judging the chain of custody rather than the pixels.

Does warning people about deepfakes make them distrust real recordings?

It can. People may dismiss an authentic clip by saying "that's AI." The answer is to teach the verification steps, not blanket doubt. A real public event has many witnesses and records, and you can show them.

Is it fair to blame the victim of a voice clone scam?

No. These calls are built to bypass careful thinking with fear and urgency, and the voice is a good one. Treat reports kindly, since people who fear embarrassment stay silent, and silence helps scammers. Encourage fast reporting.

Will a law stop deceptive deepfakes?

Laws such as Idaho Code 67-6628A and the FCC's robocall ruling give targets and enforcers tools after the fact. They do not stop an anonymous actor in another country from posting a clip, and the statute's own text protects platforms from being treated as the publisher of users' content. Laws help with accountability; verification habits are the front line.