Fetchwright · Lab notes

How it was built

Fetchwright began as two public experiments: two AI engineers building the same fetch layer against one frozen benchmark, and a relevance filter for the products it fetches. This page keeps the story — the problem, the bake-off, the judge, the stages and every result along the way.

The problem: every failed fetch is an empty slot

RoundupForge builds product roundups from public product and search pages — millions of them a month. The pages came through a rented scraping API. Over thirty days it delivered a usable page for 88 % of product requests — and for just 37.5 % of search requests. Quotas ran out mid-month, so the plan renewed early, again and again.

Before writing a line of fetching code, we replayed 84 days of real demand. More than half of all product fetches were re-fetches of a page another project had fetched hours earlier. A longer cache — with prices refreshed from the search result — removed them: −55 % product fetches, live since 20 September 2026.

The bake-off: two engineers, one contract

Two AI engineering agents build Fetchwright independently, behind the same interface. Neither may touch the other’s code — or the judge. The best parts of both get merged afterwards; the competition is how we find them.

Each implementation ships switched off by default. Production stays untouched until Fetchwright passes every gate — and costs less than the incumbent. Every attempt, failed ones included, is measured, so cost is counted, not estimated.

The judge: the benchmark is frozen first

The scoring rules, the measurement and the page sample are frozen before the first run. Changing any of them takes a new score version, so old and new results never get mixed. The incumbent runs through the same harness; nobody is graded on a curve.

Earned in stages, not bought up front

1. Lab · Stage 0 — develop on saved pages and direct connections; prove the fetcher before anything is bought.
2. Probe · Stage 1 — the smallest possible test in real conditions; anything that fails stops itself early.
3. Scale · Stage 2 — grow only what passed, then the alternating rounds and the four-hour sustained run.
4. Cost · the rule — Fetchwright takes over only where it is cheaper per usable page than the incumbent.

The payoff: 88 % lower scraping costs

StrongMocha started out paying for two rented scraping plans a month. Reusing pages already read cut that in half. Fetchwright then took over as the main source and, on current usage, cuts the rest by about another three quarters. All told, StrongMocha’s scraping costs are projected at 88 % below where they started — each month, and over a full year if usage stays as it is.

It is better, not just cheaper: since the 5 October update Fetchwright has delivered 99.5 % of more than 60,000 pages itself, in about three seconds each, with no hour below 97 %. The remaining 0.5 % went to the rented service, which stays on as a backup until around mid-October and is not renewed.

Experiment 2: keep the gold, drop the ash

A search for a wood stove also returns stove-pipe thermometers, gaskets and gloves. Experiment 2 asks whether JEV — a classification model that answers with a choice, a score or a yes/no, never with prose — can sort them at a fraction of the cost of a chat model. Dropping a good product damages an article; keeping junk only wastes a slot. So the gate is one percent.

The dataset: 70 real keywords and 1,031 keyword–product pairs, split before anyone looked — 28 keywords for development, 42 held out. Both agents label every pair blind and independently; where two model families agree, it becomes gold, and only the disagreements that change an outcome go to a human.

Field note: Grynlume

Grynlume is our self-hosted, cookieless web analytics — it also counts visits to this site. It is the first place JEV runs on live traffic, in an opt-in trial with three decisions: what kind of site sent a visitor (a choice), whether it is a person or a script (a yes/no — real browsers score 0.03–0.05, scripted tools 0.94–0.99), and, when traffic jumps or dips, the likely cause and its urgency (a score). grynlume.com

Status log

1. 7 Oct 2026Holds

   The first day and a half. More than 60,000 pages since the update: 99.5 % delivered by Fetchwright itself, about three seconds each, and not a single hour below 97 %. The rented backup took the other 0.5 %.

2. 5 Oct 2026Saved

   Scraping costs down 88 %. StrongMocha started with two rented plans a month. Reusing pages halved that; on current usage Fetchwright cuts the rest by about three quarters — 88 % less each month, and over a full year if usage stays as it is.

3. 5 Oct 2026Live

   Nearly every page by itself. Since the 5 October update Fetchwright adapts by itself when the shop starts pushing back. In the first hours it delivered 99.7 % of pages itself, in about three seconds each; 0.3 % still went to the rented backup.

4. 1 Oct 2026Resumed

   Astra is back at the bench. Bench B resumes work on its implementation, now on a newer OpenAI model, against the same frozen benchmark.

5. 1 Oct 2026Pass

   Three setups, three passes. The same 300-page test in three different setups: 97–99 % of search pages and 92 % of product pages usable on every one.

6. 1 Oct 2026Done

   The right place, always. Some addresses that every map puts in the US are placed abroad by the shop itself. Fetchwright now spots that on the first page and stops using them.

7. 1 Oct 2026Live

   Fetchwright is live. Every page RoundupForge reads now goes through Fetchwright first; the rented service only steps in as a backup. First hours: 99.7 % of pages delivered, the backup was needed six times.

8. 22 Sep 2026Live

   fetchwright.com opens. Cookieless analytics by Grynlume from the first visit.

9. 20 Sep 2026Review

   Benchmark handed to Astra for review. The scoring also checks price, rating and review dates, so a fetcher that saves data by losing it shows up.

10. 20 Sep 2026Done

    Fable completes stage 0. 42 live pages from a single connection, zero blocks, all on the first attempt; on the same 30 URLs it matches the rented API (28 of 30 each) — faster, with more complete data.

11. 1 Oct 2026Match

    As complete as the incumbent — or more. Each page is checked against the rented service in the background: titles identical, prices and review counts missing less often than there.

12. 20 Sep 2026Labelling

    Experiment 2 benchmark frozen. 1,031 pairs; blind labels from both agents pending.

13. 1 Oct 2026Pass

    Stage 1 complete for Bench A. Fable’s fetcher passes both quality gates in three different setups: 97–99 % of search pages, 92 % of product pages. Two of them can beat the incumbent on cost.

14. 1 Oct 2026Found

    The shop pushes back in its own way. After a few hundred pages, it began turning requests away in a pattern that had nothing to do with where they came from. Fetchwright learned to recognise that pattern and adapt.

15. 1 Oct 2026Found

    Maps disagree. The shop places some “US” addresses in the Netherlands or Japan, against every public map. Such pages are stopped after the first screen; the address is switched or retired.

16. 1 Oct 2026Live

    Bench A goes live. Fable’s fetcher becomes RoundupForge’s first source — 10 % at 08:22, all fetches from 10:47 — with the rented service as automatic backup and an off switch that works within 30 seconds. First hours: 99.7 % of pages delivered.

17. 1 Oct 2026Found

    Pages without a place. Some connections get a page with no delivery location at all — and no price. Found within an hour of going live through the page-by-page comparison; such pages are now fetched again elsewhere.

18. 1 Oct 2026Load test

    Four hours at eleven times the real load: 75 % usable overall, with three slowdowns of about 40 minutes each where the shop pushed back; 96–97 % in between. At RoundupForge’s real load no slowdown has appeared.

19. NextMid-October

    The rented service retires; Fetchwright runs entirely on its own. Bench B keeps building in parallel.

Last updated 7 October 2026

Back to Fetchwright