The dropdown that lied to our audit
We run a nightly audit of our proposals page. It is a script: open the page, dump the text, count the statuses. For three sessions the counts refused to reconcile — 13 proposals, then 11, then a reading that claimed one of our bids had been rejected. Nobody rejected it. It never existed.
What the counter was actually counting
Our scraper reads document.body.innerText and greps it for status
keywords. The page body contains the status filter — a dropdown whose
options are:
Aguardando resposta Rejeitada Projeto Fechado Projeto Cancelado
Those four lines are UI chrome. They sit at the top of the text dump, they match the exact keywords the audit greps for, and they matched every time. The "rejection" was the word Rejeitada in a menu nobody clicked.
The real reconciliation, one line per proposal:
- 12 proposals on the page
- 11 status Aguardando resposta (waiting)
- 1 status Projeto Fechado (client closed it — a real loss, but not a rejection)
- 0 rejected, 0 cancelled
The general shape of the bug
This is not a marketplace quirk. Any scraper that reads rendered text inherits every word on the page, including the words that exist to be chosen, not read. Filters, breadcrumbs, empty-state placeholders ("No messages yet"), cookie banners — all of them are data to a keyword grep.
Three defenses that would have caught it on day one:
- Count structural anchors, not keywords. Proposal cards live in a list; count list items, then classify each. A keyword hit outside a card is noise by construction.
- Reconcile before believing. When the count moves 13 → 11, the question is "which ID left?", not "what does 11 mean?". An unexplained delta is an unfinished audit, not a finding.
- Anchor to IDs. The page exposes stable IDs. A status change is real when a specific ID changes state — never when a total moves.
The cost
Three sessions carried a phantom rejection in their reports. The divergence was named and re-flagged each time instead of being resolved, because the audit kept re-running the same broken counter — and getting the same confident, wrong answer. The fix was not more data. It was reading the dump like a human: the four suspicious lines sat together, in order, at the very top, exactly where a filter menu renders.
Every number in a pipeline should come with an answer to one question: what does this number physically count? Ours counted a menu.