The Five Distortions of OSINT Verification — and the Questions That Defeat Them

2026-07-16 · Philip Choo · AI9OS

In January 2024, a finance employee at a multinational's Hong Kong office joined a video call with his CFO and several colleagues, walked through a transfer process step by step, and wired out roughly US$25 million. Every person on that call was an AI-generated deepfake. He had done the diligent thing — he'd insisted on verifying the instruction face to face. The verification itself was the forgery.

That case is the OSINT profession's problem in miniature. In an era where a convincing lie is pixel-identical to the truth, collection is not the hard part anymore — adjudication is. Any tool can pull ten thousand data points before lunch. The question that decides cases, reputations and sometimes liberty is: which of these findings would survive being attacked?

Over years of casework we've found that when an open-source finding goes wrong, it almost never fails randomly. It fails through one of five predictable distortions. Each has a specific counter-question — and each counter-question only works if the answer is written down before the finding is accepted. (The five-distortions framing is adapted from an argument the strategist Sandeep Swadia makes about critical thinking in the AI era; what follows is how we've operationalised it for investigative verification.)

Distortion 1 — Authority: the halo that replaces the check

Theranos raised US$700 million from some of the sharpest investors alive, with two former secretaries of state on its board — and the core technology never worked. Nobody with blood-chemistry expertise had verified the claim; the calibre of the believers had substituted for verification.

OSINT has the same failure mode in cheaper clothing. A corporate registry entry, a verified social account, a quote in a reputable outlet — each carries a halo. But registries record what was filed, not what is true. Verified accounts get sold. Reputable outlets syndicate unverified wire copy.

The counter-question: what would have to be true for this to be real? Then list the evidence for the claim and against it. If your answer to "why do we believe this?" is a description of the source's credentials rather than the claim's substance, you have a halo, not a finding.

Distortion 2 — True lies: claims that are technically accurate and practically false

Marketing language never lies outright; it lawyered its way there. "Up to eight times faster." "Starting at $37,000." "Clinically proven." Each statement survives a courtroom and still deceives.

Source material in investigations is full of the investigative equivalent: "linked to", "associated with", "reportedly", "sources say", "believed to be". A subject "linked to" a shell company might be its beneficial owner — or might once have shared a mailing address with its registered agent.

The counter-question: strip every qualifier and restate the naked claim with a question mark. "Subject is linked to Company X" becomes "Subject owns Company X?" — and suddenly you can see exactly what evidence you have and what you merely inherited from someone else's hedge. If the claim only survives with its qualifiers, record the qualified version and rate it accordingly.

Distortion 3 — Consensus: a hundred echoes of one voice

In Solomon Asch's famous conformity experiments, participants denied the plain evidence of their own eyes rather than disagree with seven confident strangers — and it took only one dissenter to break the spell.

The open web is a conformity machine. One unverified claim gets scraped, aggregated, reworded and republished until a search returns forty "independent" confirmations — forty echoes of a single origin. Data-broker records are especially prone to this: dozens of people-search sites confidently agreeing, because they all bought the same flawed dataset.

The counter-question: write the strongest case against the finding before accepting it. Trace the family tree of your sources — are they genuinely independent, or one ancestor with many descendants? We make the dissent a mandatory, recorded step: someone (human or AI, explicitly prompted to argue the opposite) must put the best contrary case on the record. Consensus that has never faced a dissenter is not corroboration. It is unexamined agreement.

Distortion 4 — AI offloading: fluent output, absent judgment

An MIT study asked one group to write essays with their own research and another to use a chatbot. Minutes after finishing, the chatbot group couldn't quote a single line of what they had just "written". The tool hadn't assisted their thinking; it had replaced it.

AI is now inside every serious OSINT pipeline — ours included; the platform is AI-native by design. The distortion isn't using AI. It's letting fluency stand in for verification: an LLM summary that reads confidently, a face-match score that looks precise, an entity resolution that feels obvious. Machine output arrives pre-polished, which is exactly why it needs more scrutiny than a messy human source, not less.

The counter-question: what independently checked this output, and what did it return? Be precise; then verify. Run the second tool, the second model, the second source — and log what the cross-check actually said. In our stack, machine output enters at the lowest confidence tier by rule and cannot climb without recorded corroboration. The AI proposes. It never confirms.

Distortion 5 — Motivated reasoning: the source you never fact-check is yourself

The hardest distortion, and the one that ends careers, is the lie that comes from wanting. The client is desperate. The fee depends on a result. The working theory is elegant, and the investigator has defended it in three meetings. From that moment, every ambiguous data point quietly votes for the theory.

There is no tool for this one. There is only a question, asked on purpose, with the answer written down:

What are we refusing to see because the client, the fee, or the theory needs this to be true?

The written answer matters more here than anywhere else. An honest sentence — "we have not been able to falsify the alternative explanation that…" — in the file, dated and signed, is the difference between an investigator and an advocate. Courts can tell the difference. So can regulators.

The discipline: no finding is born confirmed

Five distortions, five questions. The operational insight that changed our practice is that these checks cannot live in anyone's head, because heads are exactly where the distortions live. So we made them structural.

On our platform, every finding enters the evidence locker at an unverified tier, hash-chained and time-stamped. It can be corroborated upward by rule. But the top tier — confirmed — is structurally unreachable until the finding has been adjudicated: all five challenges answered in writing, including the recorded case against the finding, and the verdict sealed into the same tamper-evident chain as the evidence itself. A tick-box without written reasoning is rejected by the software. We call the step Gate 9, and the storage layer simply will not write a confirmed finding without it — not for the AI, and not for the investigator either.

The deliverable changes accordingly. A court, an opposing expert or a client's counsel doesn't just see what we found — they see the examination each confirmed finding withstood, bound to a hash chain that cannot be back-dated. In a world where anything can be faked, that record of surviving attack is the product.

The needle in the haystack was always hard to find. The modern problem is that the haystack is now full of fake needles that look exactly like the real one. More collection just gathers more needles. Judgment — recorded, structured, and enforced — is how you find out which one is real.

General information for practitioners, not legal advice. AI9OS is an open-source-intelligence technology platform; investigation services are conducted solely by licensed agencies under Singapore's Private Security Industry Act.