AI research isn't trash — unaudited research is

Method note · 2026-08-29 · Oroboro Labs

AI-assisted research has a reputation for producing garbage, and the reputation is earned: a confident summary, an invented source, a wall of text that collapses the moment you open the first reference. But the diagnosis is wrong. The problem is not that AI researches. It's that most pipelines research without an audit stage.

We run research agents nearly every day, and the method has one component almost nobody uses: a second agent whose only job is to reject. Here are the five failure modes we've actually hit, and the six-link chain that catches them.

The five diseases

1. Hallucinated source — the model cites what "sounds right", not what exists.
2. Shallow reading — it stops at the first page and generalizes the rest.
3. Artificial ceiling — "the top 20 results" and done, silently cutting two-thirds of the material.
4. Pasted payload — the whole corpus crammed into the prompt. It truncates, and nobody notices what vanished.
5. Unmeasured number — a third-party statistic repeated as if it were a verified fact.

Notice: none of the five is "the model is dumb". All five are failures of method, which means all five are fixable without waiting for a better model.

The six-link chain

1. Index before search. Any decent source has a catalog. Locate the index and read it entirely, into a file. Search only covers what the index doesn't. This alone kills half the garbage.
2. Corpus on disk, path in the prompt. Downloaded material lives in a file; the next agent receives the path, never the pasted content. Pasted payloads truncate silently.
3. No ceiling — with a declared cut. The target is the size of the source's universe, not "the first 20". And when something must be left out, the final product says what was left and why. Undeclared cuts are how scope decisions get smuggled past the reader.
4. The adversarial reviewer. A second agent with a single mission: reject. It checks coverage (confer against the file, list what's missing), depth (reopen a sample and compare with the source), and fidelity (sample claims, verify against origin). A reviewer who can approve everything is not a reviewer.
5. Chained audits. Round N+1 verifies round N's corrections — because fixing things introduces new errors too. A report that came out "corrected" wrong is worse than the first draft.
6. Empty field over guess. Source blocked? It becomes a record with the reason. A documented access failure; never an inference dressed up as a fact.

What changes in practice

• The final product is traceable: every claim resolves to a file. Not "the AI said" — "file X says, verified at the source."
• It takes longer, and that's a feature. Verification costs about twice the effort and avoids the far larger cost of deciding on rotten material.
• You can trust it enough to act — which is the only test of research that matters.

One standing rule worth stealing: a savings number is either measured or it's advertising. If a report quotes a third-party number as if it were our measurement, that's a mislabeled fact, and mislabeled facts poison everything downstream of them.

Start tomorrow

Minimum viable version: source with an index → save the index → filter locally → one agent extracts with written rules → you play the adversarial reviewer with three questions: what's missing? does a sample match the source? is any claim an inference posing as a fact? When volume grows, the reviewer becomes a second agent. Don't start with full automation — start with the audit.

Our Second Brain Starter ships the written-contract + index approach end to end: a vault that both you and your agent can read cheaply, with the reading ladder and tools included.