ANANSI

A scraper that breaks loudly is an outage you fix on Monday. One that breaks quietly quotes your customers the wrong price for six weeks.

ANANSI watches B2B distributor catalogues, decides whether a change is a broken scraper or a real commercial event, and repairs it through Bright Data Scraper Studio — without ever letting an unverified fix reach production.

Read the source Open the test catalogue pip install -e .  →  anansi demo  ·  no credentials, no credits
The failure

A number that is present, correctly typed, in range — and wrong

A distributor rolls out a promotion banner. The struck-through list price is now emitted before the real price, sharing the same CSS class.

<td class="price-cell"> <!-- taken --> <span class="price price--list">₹5,973.63</span> <!-- wanted --> <span class="price">₹4,594.62</span> </td>

Every downstream check passes. The value is a number, in range, in the right currency. It is simply the wrong one — 27% high on every discounted line. Schema validation cannot see this, and neither can a human skimming two preview rows.

This is the case ANANSI exists for, and the one every self-healing scraper demo skips.

The verdict

Three answers, not two

Three layers run cheapest-first — a declared contract, statistical drift against a known-good baseline, and semantic profiling of each field. They fold into one verdict.

Structure brokenThe scraper is wrong. Repair it.
World changedThe scraper is fine — the supplier repriced. Alert; heal nothing.
StableNothing to do.

The middle verdict is the one that saves money. A system that cannot tell a genuine 28% repricing from a 28% breakage burns credits rewriting a scraper that was working perfectly, and overwrites correct extraction logic doing it.

When statistics abstain, the page decides

A shift across the whole catalogue is a real event. A shift confined to part of it is genuinely undecidable from the numbers — a promotion on some lines and a selector collision on some lines produce identical distributions. So drift refuses to guess, and hands the question to the markup.

1 · contract PASS 48 rows, 8 fields 2 · drift PASS Part of the catalogue moved coherently while the rest held still on price. Equally consistent with a partial promotion and with a selector collision, so this is not decided here — escalating to a DOM check. 3 · semantic PASS all fields in profile STABLESTRUCTURE BROKEN overruled by the DOM check price span.price → span.price [last] 2 nodes match span.price in a row; the correct value is the last
The gate

A fix has to prove itself

heal is called without --auto-approve. The preview it returns is scored against the same contract that caught the break, and only a passing preview is approved. A fix can never reach production on weaker evidence than the check that caught the failure.

  1. Detect and localiseWhich field failed — and on which rows, specifically.
  2. Diagnose against the DOMWhere did a known-good value move, and does its selector now collide?
  3. Heal, held at approvalThe proposal is returned as a preview, not shipped.
  4. Gate on the contractSame contract, same standard. Failing previews are rejected outright.
  5. Confirm on a full runApproval is necessary, never sufficient. Re-run and re-measure.

Against the live platform, a heal came back with eight of nine fields matching ground truth exactly and an empty object in the ninth on every row. A human reviewing two preview rows would have approved it. The contract rejected it.

Passing is not the same as proven

A live heal returned a one-row preview, and it passed cleanly. But the defect being repaired was a 15% null rate confined to out-of-stock tiles, and that one row was an in-stock product. It could not possibly have demonstrated the fix.

Seeing only clean rows bounds a null rate; it never proves one. With n clean observations the true rate is bounded at 1 − 0.051/n:

Clean preview rowsNull rate is at most
195.0%
277.6%
1025.9%
309.5%
595.0%

So a two-row preview cannot establish a 5% ceiling — not approximately, not at all. The gate now says so out loud, and confirms on a full run afterwards.

Live finding

The contract passed. The bug was untouched.

Running against a real distributor, the collector was dropping title, product_url and image_url on every out-of-stock tile — 46 of 300 rows, a 15% null rate, a clear failure. After the heal, the same contract passed at 1%.

Nothing had been repaired. The distributor had restocked. Two out-of-stock tiles remained instead of forty-six, and they were still just as broken. The population moved and dragged the statistic under the ceiling with it.

A null rate is a population statistic. It can be satisfied by the world changing rather than by the bug being fixed.

So a failure is localised the moment it is detected — a one-level decision stump over the low-cardinality fields, finding what the failing rows have in common — and confirmation re-measures that cohort rather than the average.

+ 0.1s localised title is null on 100% of the 46 rows where in_stock=False, and 0% of the rest + 3.3s approved preview satisfied the contract; fix is live + 51.4s confirm_failed 2 of 2 rows with in_stock=False are still missing title

A fix counts as confirmed only when the rows that were broken are present in the new run and are no longer broken. A cohort that has vanished demonstrates nothing.

Scoreboard

Seven failure modes, graded

anansi demo runs all seven offline and grades itself against outcomes fixed in advance. Three of them require not healing, which is the harder half.

ScenarioFamilyCorrect actionResult
baselinenothing; the catalogue is healthyno incident
rename_price_classbreakingre-anchor price, approvehealed
nest_stockbreakingdescend into the new node, approvehealed
list_price_firstsilentcatch the wrong-but-valid pricehealed
repricingreal eventreport the move, heal nothingno_action
drop_moqbreakingescalate — the field is gonecontract_stale
table_to_cardsbreakingreject the partial fix, page a humanescalated

282 tests. Python, FastAPI, numpy and scipy for the statistics, BeautifulSoup for the DOM diff, SQLite for incident post-mortems.

Test target

A catalogue that breaks on command

Meridian Components is a 48-part distributor catalogue with a deterministic mutation for each failure mode. The stylesheet deliberately styles the renamed classes too, so a breaking change looks identical to a human and is invisible to anything but a scraper.

All ten mutations, grouped by family →