A scraper that breaks loudly is an outage you fix on Monday. One that breaks quietly quotes your customers the wrong price for six weeks.
ANANSI watches B2B distributor catalogues, decides whether a change is a broken scraper or a real commercial event, and repairs it through Bright Data Scraper Studio — without ever letting an unverified fix reach production.
A distributor rolls out a promotion banner. The struck-through list price is now emitted before the real price, sharing the same CSS class.
Every downstream check passes. The value is a number, in range, in the right currency. It is simply the wrong one — 27% high on every discounted line. Schema validation cannot see this, and neither can a human skimming two preview rows.
This is the case ANANSI exists for, and the one every self-healing scraper demo skips.
Three layers run cheapest-first — a declared contract, statistical drift against a known-good baseline, and semantic profiling of each field. They fold into one verdict.
The middle verdict is the one that saves money. A system that cannot tell a genuine 28% repricing from a 28% breakage burns credits rewriting a scraper that was working perfectly, and overwrites correct extraction logic doing it.
A shift across the whole catalogue is a real event. A shift confined to part of it is genuinely undecidable from the numbers — a promotion on some lines and a selector collision on some lines produce identical distributions. So drift refuses to guess, and hands the question to the markup.
heal is called without --auto-approve. The
preview it returns is scored against the same contract that caught the break, and only
a passing preview is approved. A fix can never reach production on weaker evidence than
the check that caught the failure.
Against the live platform, a heal came back with eight of nine fields matching ground truth exactly and an empty object in the ninth on every row. A human reviewing two preview rows would have approved it. The contract rejected it.
A live heal returned a one-row preview, and it passed cleanly. But the defect being repaired was a 15% null rate confined to out-of-stock tiles, and that one row was an in-stock product. It could not possibly have demonstrated the fix.
Seeing only clean rows bounds a null rate; it never proves one. With n clean observations the true rate is bounded at 1 − 0.051/n:
| Clean preview rows | Null rate is at most |
|---|---|
| 1 | 95.0% |
| 2 | 77.6% |
| 10 | 25.9% |
| 30 | 9.5% |
| 59 | 5.0% |
So a two-row preview cannot establish a 5% ceiling — not approximately, not at all. The gate now says so out loud, and confirms on a full run afterwards.
Running against a real distributor, the collector was dropping title,
product_url and image_url on every out-of-stock tile — 46 of
300 rows, a 15% null rate, a clear failure. After the heal, the same contract passed at
1%.
Nothing had been repaired. The distributor had restocked. Two out-of-stock tiles remained instead of forty-six, and they were still just as broken. The population moved and dragged the statistic under the ceiling with it.
A null rate is a population statistic. It can be satisfied by the world changing rather than by the bug being fixed.
So a failure is localised the moment it is detected — a one-level decision stump over the low-cardinality fields, finding what the failing rows have in common — and confirmation re-measures that cohort rather than the average.
A fix counts as confirmed only when the rows that were broken are present in the new run and are no longer broken. A cohort that has vanished demonstrates nothing.
anansi demo runs all seven offline and grades itself against outcomes
fixed in advance. Three of them require not healing, which is the harder half.
| Scenario | Family | Correct action | Result | |
|---|---|---|---|---|
| baseline | — | nothing; the catalogue is healthy | no incident | ✓ |
| rename_price_class | breaking | re-anchor price, approve | healed | ✓ |
| nest_stock | breaking | descend into the new node, approve | healed | ✓ |
| list_price_first | silent | catch the wrong-but-valid price | healed | ✓ |
| repricing | real event | report the move, heal nothing | no_action | ✓ |
| drop_moq | breaking | escalate — the field is gone | contract_stale | ✓ |
| table_to_cards | breaking | reject the partial fix, page a human | escalated | ✓ |
282 tests. Python, FastAPI, numpy and scipy for the statistics, BeautifulSoup for the DOM diff, SQLite for incident post-mortems.
Meridian Components is a 48-part distributor catalogue with a deterministic mutation for each failure mode. The stylesheet deliberately styles the renamed classes too, so a breaking change looks identical to a human and is invisible to anything but a scraper.