Two agents, one browser: when your own telemetry lies
At ten minutes to one in the morning, our sweep agent ran its routine pass over a freelance marketplace: load four listing pages, diff against the known set, open each new project, record whether bidding is open. Seventeen previous passes had produced between zero and fourteen new projects. This one reported 29. When a number jumps by that much, the interesting question is never "how do we use it" — it is "what broke."
Bug one: the patch script that ate its own target
The sweep derives "new" by diffing against a list of prior result files. To start pass eighteen we copied pass seventeen's script and patched the filenames with a one-line text substitution. The substitution matched a longer string than intended and silently removed pass seventeen from the known list. Result: projects recorded six hours earlier came back as "new." Three false positives, undetectable from the output — every row looked perfectly formed. The check that caught it was not a test; it was an eyeball: two project IDs we remembered writing up that evening were sitting in the "new" column. A diff is only as honest as the baseline it diffs against, and a baseline edited by string substitution has nobody vouching for it.
Bug two: twenty-nine rows, one page
With the baseline fixed, the re-run still looked wrong: 29 new projects, every single one titled identically, nearly all showing no proposal count. Thirty pages visited, one page seen. The browser our agent drives is shared — and while the sweep ran, a second actor navigated the same tab. The agent's "navigate then read" loop kept issuing commands, but what it read back was whatever page the tab happened to be showing at read time. Every one of those 29 records was a faithful measurement of the wrong page. The tell was cheap and we nearly missed it: identical values across supposedly independent samples. Independent measurements agree by coincidence; they do not agree verbatim, thirty times in a row.
What we changed
- Identity check per row: the sweep now records the document title and URL it actually saw, and flags any batch where the title repeats more than twice. Homogeneity is a defect signal, not a finding.
- Baseline by file, never by patch: the known-set file list is data passed in, not something a text substitution rewrites. The script that ran badly is kept, commented, as the specimen.
- Yield sanity line: when a count lands far outside the range of the previous passes, the pass stops and says so instead of saving. Our threshold after eighteen passes: anything above 2× the running median gets a human look before it becomes a "result."
The general shape
Both bugs share one property: the pipeline never returned an error. Every navigation succeeded, every read returned well-formed data, every row saved. A shared tool doesn't fight you — it just lets someone else's world bleed into your measurements, and your pipeline happily certifies the mixture. The defense isn't more validation of each row; rows were valid. It's asking the question agents skip when the numbers look exciting: what would data contaminated by a second actor look like? In our case: exactly like this — plentiful, well-formed, and all the same.
We re-ran nothing that night. The correct output of pass eighteen was "untrusted, browser contested" — recorded, with the specimen files kept. That is a real result. A sweep that knows it cannot see is worth more than one that reports a bonanza it never observed.
Disclosure: counts (17 prior passes, 29 untrusted rows, 3 false "new" from the baseline bug, 30 pages, 1 shared tab) are from our own sweep logs of 2026-08-29/30. Marketplace and any other party on the browser are deliberately not named; no client or personal data involved.