Webwright: Why AI Net Brokers Ought to Write Code, Not Click on

1. received’t maintain nonetheless

When you have constructed an internet agent not too long ago, you realize the failure sample. You give it a activity like “pull each itemizing from this listing right into a spreadsheet” and watch it inch ahead. It reads the web page. It predicts a click on. It waits for the brand new DOM, the web page construction the browser sees. It reads once more, predicts once more, waits once more.

Then, someplace round step 40, issues disintegrate. A modal pops up unexpectedly. The “subsequent web page” button strikes. The agent errors one aspect for an additional. Any one in every of these can derail the entire activity. The deeper drawback isn’t the unhealthy click on. It’s how the agent operates: have a look at the web page, determine on one motion, see what modified, then determine once more. It repeats this loop time and again, with no sturdy plan for full the duty from begin to end.

The sector has tried a couple of alternative ways to make this loop extra dependable. Some brokers, like OpenAI’s Operator and Anthropic’s Laptop Use, work from screenshots and work together with an internet site very similar to an individual would. Others, like WebVoyager, use the web page’s DOM to grasp what parts can be found and determine which one to work together with.

Benchmarks comparable to Mind2Web and WebArena made these programs simpler to check by giving brokers a normal set of actions—click on, sort, scroll, choose. And open-source instruments like browser-use, Skyvern, Stagehand, and LaVague have packaged these concepts into APIs that engineers can extra simply construct into actual functions.

These approaches make the loop extra dependable, however they don’t change the way it basically works: the agent nonetheless takes one motion at a time, waits to see what occurs, then decides what to do subsequent. And when the duty is over, it hasn’t constructed something reusable—it has solely accomplished a sequence of clicks.

Webwright, a browser-agent framework from Microsoft Analysis and the College of Hong Kong, takes a special method. Its tagline captures the thought: “A terminal is all you want for net brokers.”

As a substitute of asking the mannequin to determine the subsequent click on, Webwright has brokers writing and working code—utilizing bash and Playwright scripts to open browsers, examine pages, and perform the duty. The consequence isn’t only a lengthy sequence of browser actions. It’s a program engineers can examine, rerun, modify, and reuse.

This distinction issues most when the online is your knowledge supply: dashboards, product catalogs, search outcomes, inner instruments, JavaScript-heavy websites, and workflows you count on to run greater than as soon as. In these circumstances, the query isn’t simply whether or not an agent can end the duty. It’s whether or not it ought to hold clicking by means of the browser or write a reusable program to do the work.

We’ll begin with the 4 predominant approaches to constructing net brokers and the restrictions they nonetheless share. Then we’ll look inside Webwright: how its three core elements work, how a framework of roughly a thousand strains performs on benchmarks, and what the outcomes say about value and reliability. Lastly, we’ll put the method to work on three frequent scraping issues—paginated pages, JavaScript-rendered content material, and infinite-scroll feeds.

2. Why net brokers hold breaking

The shift to “write code” issues as a result of it addresses the underlying drawback, not simply the signs. Right this moment’s net brokers differ in how they perceive a web page—some have a look at screenshots, others learn the DOM—however most nonetheless work the identical means: take one browser motion, see what occurs, then determine on the subsequent one.

That works for brief duties. However the longer the duty runs, the extra possibilities there are for one unhealthy click on, a modified web page, or a misinterpret aspect to throw every thing off.

Imaginative and prescient brokers are simple to grasp: they have a look at the web page very similar to an individual does and determine the place to click on. That works effectively for a lot of browser duties, however scraping calls for extra consistency. A small format shift can transfer a button simply sufficient for the agent to click on the mistaken place—or nothing in any respect.

There’s additionally a price to repeatedly screenshots. Each new screenshot consumes tokens, and the agent has to hold sufficient context ahead to recollect what it already did. On an extended activity, that context turns into tougher and dearer to take care of.

DOM and accessibility-tree brokers keep away from a number of the issues that include screenshots. As a substitute of guessing the place a component relies on pixels, they’ll learn the construction of the web page and determine buttons, hyperlinks, kinds, and different parts immediately. WebVoyager, for instance, reported about 59% activity success throughout 50 real-world web sites, considerably higher than text-only baselines.

However this creates a special drawback: an excessive amount of web page knowledge. A fancy web page can produce an accessibility tree bigger than 50KB. Because the agent strikes by means of a activity, outdated web page state accumulates in its context despite the fact that a lot of it’s now not helpful. The references it makes use of to determine parts may change when a web page lazy-loads content material, rerenders, or navigates someplace new.

So higher entry to web page construction doesn’t essentially make lengthy browser duties dependable. On VisualWebArena, main vision-language brokers accomplished solely about 16% of duties, in contrast with roughly 89% for people.

Mounted motion APIs made net brokers simpler to construct and benchmark. Give the mannequin a small set of actions—click on, sort, scroll, choose—then let it observe the consequence and select once more. The draw back is that the agent can solely specific one small step at a time. It could actually’t naturally say, “hold clicking subsequent till there aren’t any pages left,” “retry if this aspect doesn’t seem,” or “gather these 1,000 rows and save them to a CSV.” Every of these needs to be damaged into many particular person actions, with one other mannequin name in between.

Frameworks like browser-use, Skyvern, Stagehand, and LaVague make browser brokers a lot simpler to construct and combine. That’s helpful, but it surely doesn’t clear up one essential drawback for recurring knowledge work: when the duty is completed, there typically isn’t something reusable left behind. The agent might have collected the info as soon as, however subsequent week it has to work by means of the browser once more.

Analysis had already pointed towards one other method. Within the ICML 2024 paper Executable Code Actions Elicit Higher LLM Brokers, the researchers behind CodeAct changed predefined, JSON-style actions with executable Python. They reported as much as 20% increased success charges whereas utilizing roughly 30% fewer steps.

The reason being easy: code lets the mannequin do greater than take one motion at a time. It could actually use loops, retailer variables, retry failures, write recordsdata, and examine errors—all inside a program it could run once more. Webwright brings that very same concept to browser automation.

3. Webwright’s distinctive level: make the workspace the state

Most browser brokers hold their progress within the browser session. Shut the tab, and that state is gone.

Webwright flips this round. The browser is momentary; the native workspace is what persists. The agent writes scripts, logs, screenshots, and output recordsdata as it really works, finally turning a profitable run right into a reusable software. This shift has a couple of sensible advantages.

• Extra strong interactions — Playwright selectors and wait circumstances are extra dependable than pixel coordinates or momentary aspect IDs.
• Higher composition — Loops and features can deal with tons of of repeated actions in a single program
• Seen state — Progress is seen in recordsdata and logs.
• Reusable output — As soon as the duty works, the code will be reproduced as a substitute of ranging from scratch.

The place the undertaking’s 4 said benefits come from:

• Sturdy, reusable interactions — the agent acts by means of queries and wait-for-condition checks (web page.locator(...), wait_for_selector(...)) as a substitute of pixel coordinates or frozen aspect IDs, so a script survives format shifts and re-renders.
• Environment friendly composition — loops, features, and variables let a single flip say “do that for each row,” work a one-action-at-a-time agent has to spell out step-by-step.
• Workspace as state — progress lives in recordsdata, not in a fragile session or a context window bloated with stale web page dumps.
• Minimal by design — the entire system leans on 4 libraries (httpx, pydantic, playwright, typer) with no hidden framework beneath, and nonetheless posts state-of-the-art numbers.

3.1 Let’s do fast comparisons between Webwright and different choices in some eventualities.

3.1.1 Demo 1 · When clicking isn’t exact sufficient

The duty: Use Chase’s IRA calculator to check a Conventional vs. Roth IRA for somebody who’s 30, retires at 65, saves $300 a month, earns a 3% return, and has tax charges of 13% right this moment and 24% in retirement.

The problem is that the calculator makes use of six interactive JavaScript sliders.

Consequence:

• Webwright units the values immediately in code by updating the DOM inputs and triggering the required occasions. The values are precise, the chart renders accurately, and the working resolution is saved as a reusable script.

• A imaginative and prescient agent has to govern the sliders visually. It will get shut, however not shut sufficient: the $300 contribution lands at $294.

3.1.2 Demo 2 · When the identical activity comes again

The duty: Search Google Flights for a spherical journey from Seattle to San Francisco, together with the dates, and return the ranked outcomes.

Consequence:

• Webwright completes the search like different browser brokers may. The essential distinction is what occurs afterward: it retains the working code.

• When the same flight search comes up later, the agent doesn’t have to determine each subject, date picker, and click on once more. It could actually reuse the earlier script, change the inputs, and run it once more.

That’s what Microsoft means by “your shopping historical past is code as a substitute of clicks.” A accomplished activity turns into a place to begin for the subsequent one, somewhat than a browser session that disappears when it ends.

3.2 How Webwright differs from different browser-agent repos

The options are helpful, however they nonetheless put the browser on the middle of the workflow. Stagehand combines Playwright with natural-language instructions. agent-browser offers brokers a CLI for taking small browser actions. browser-use repeatedly reads the web page, chooses an motion, and executes it.

Webwright takes a special method. As a substitute of selecting the subsequent browser motion, the mannequin can write a complete Python script. The browser is momentary; the code, logs, and outputs keep within the native workspace. And when the duty is solved, the agent leaves behind a program that may be run once more.

That’s the core concept: clicking completes the duty as soon as; code completes it and retains the answer.

3.3 Inside Webwright

So what does it take to construct an agent like this? Surprisingly little.

Most net brokers put a harness—the software program connecting the mannequin to the browser—between the 2. That harness normally offers the mannequin a hard and fast set of actions: click on this aspect, sort into this subject, scroll the web page, learn the DOM, take a screenshot.

Webwright takes a special method. As a substitute of giving the mannequin a menu of browser actions, it offers the mannequin a terminal and lets it determine what instructions to run.

That makes the system surprisingly small. The core harness is roughly 1,000 strains of code throughout three elements. The total repository is nearer to 1,500 strains when you embrace the command-line interface and assist for various mannequin suppliers. There isn’t any massive library of predefined browser actions. No customized DOM engine. The core system is simply three items:

• Runner (~150 strains) — Retains monitor of the duty and every thing that has occurred thus far: what the agent is attempting to do, the present state of its workspace, and the outcomes of earlier actions.
• Mannequin Endpoint (~550 strains) — Connects Webwright to the language mannequin. It offers backends for OpenAI, Anthropic, and OpenRouter.
• Setting (~300 strains) — Provides the mannequin a terminal related to Playwright working Chromium. That is the place instructions truly execute, browser interactions occur, and recordsdata created throughout the activity are saved.

The interplay between these items is an easy loop.

1. The Runner offers the mannequin the duty and the most recent context.
2. The mannequin decides what to do subsequent and returns a shell command.
3. The Setting runs that command and sends again what occurred—terminal output, logs, screenshots, or error messages.
4. Webwright provides these outcomes to the context and asks the mannequin what to do subsequent.

In brief, the loop seems like this:

perceive the present state → select a command → run it → see what occurred → repeat

The method continues till the mannequin believes the duty is full and a closing self-check agrees. Webwright doesn’t attempt to encode each potential browser interplay into the harness. It offers the mannequin a general-purpose interface—the terminal—and lets the mannequin determine use it.

The benchmarks assist the design. On On-line-Mind2Web, GPT-5.4 with Webwright scores 86.7%, the best amongst open-source AutoEval harnesses, whereas Claude Opus 4.7 reaches 84.7% and performs higher on the toughest duties.

The larger sign comes from Odysseys. GPT-5.4 utilizing coordinate-based browser management scores 33.5%. With Webwright, the identical mannequin reaches 60.1%—a 26.6-point achieve from altering the harness, not the mannequin.

Webwright’s undertaking web page lists 60.8%; I take advantage of the 60.1% reported in its GitHub comparability for consistency.

One other consequence helps the thesis: as soon as Webwright has constructed reusable instruments, the mannequin can get smaller. Microsoft stories that even a 9B open mannequin (Qwen-3.5-9B) performs effectively on On-line-Mind2Web as soon as 5 or extra instruments can be found. The software doesn’t simply save work—it lowers the mannequin functionality wanted subsequent time.

There are tradeoffs. These are LLM-judged AutoEval scores, and the headline Mind2Web consequence makes use of 100 of 300 duties. It’s also not low-cost: about $2.37 per activity with GPT-5.4 and $6.09 with Claude Opus 4.7. Webwright spends extra compute upfront to construct instruments which might be extra strong and reusable.

4. Experiments and Consequence

I examined Webwright on three more and more tough websites, utilizing a separate Claude Sonnet agent for every run. That is to see how strong Webwright is the place scraping normally breaks.

I used the Claude Code plugin somewhat than the standalone benchmark harness. It retains the identical core setup—terminal + Playwright—however Claude Code runs the agent loop. That removes the necessity for a separate API key or per-task API invoice, although not the compute value.

The tradeoff is token utilization. In Microsoft’s instance, the Codex-hosted talent used ~3.3M tokens versus ~424K for the standalone harness—about 8× extra, largely from cached context. The fee shifts into the host session somewhat than disappearing.

Setup took one command:

playwright set up firefox   # the Claude Code talent drives headless Firefox, ~110MB one-time

4.1 🔧 Check 1 — static pagination · books.toscrape.com

books.toscrape.com is the best one amongst 3 circumstances: 50 numbered catalogue pages, 20 books every, served as plain HTML. The duty was to extract each ebook title, value, score, availability, and URL, then craft a reusable CLI with --pages and --out.

Earlier than writing the scraper, the agent inspected the location and examined its boundaries: web page 50 had no subsequent hyperlink, whereas web page 51 returned a 404. It then pulled selectors from an actual product card and constructed a easy pagination loop.

for n in vary(1, pages + 1):
    url = CATALOGUE_URL_TEMPLATE.format(n=n)
    await web page.goto(url, wait_until="domcontentloaded")
    playing cards = web page.locator("article.product_pod")
    depend = await playing cards.depend()
    log(n, f"loaded catalogue web page {n}/{pages} ({url}) -> {depend} ebook playing cards discovered")

A click-based agent may deal with this website, however code was cleaner. One delicate challenge was relative ebook URLs, which change throughout pages. As a substitute of manually setting up them, the scraper used the browser-resolved href values.

The consequence was not simply scraped knowledge, however a standalone CLI software that would run once more with out an agent.

$ python final_script.py --help
utilization: final_script.py [-h] [--pages PAGES] [--out OUT]
Scrape all books listed on books.toscrape.com's catalogue pages.
  --pages PAGES  Variety of catalogue pages to traverse ... Default: 50.
  --out OUT      Output CSV file path ... Default: books.csv.

$ python final_script.py --pages 2 --out pattern.csv
-> 40 rows written to .../pattern.csv

Consequence: 1,000 books throughout 50 pages, zero empty fields, in ~37 seconds.

The verification step additionally caught a bug: the primary model by accident cleared its proof log when working --help. The agent discovered the facet impact, mounted it, examined the repair, and reran efficiently. Even on this easy website, the benefit was clear: a debuggable, reusable program as a substitute of a one-time click on hint.

4.2 🔧 Check 2 — JavaScript-rendered · quotes.toscrape.com/js

The second take a look at provides JavaScript. The quotes are not current within the uncooked HTML; they seem solely after the browser runs the web page’s JavaScript. The agent verified this primary: a direct HTTP request returned zero quote parts, whereas the rendered web page confirmed 10. A fundamental requests + BeautifulSoup scraper would silently return nothing.

Which means each web page should be rendered earlier than extraction. There may be one other catch: web page 11 nonetheless returns HTTP 200, so standing codes can not inform the scraper when to cease. As a substitute, this system checks the dwell DOM for the subsequent hyperlink and stops when it disappears on web page 10.

whereas True:
    url = BASE_URL if n == 1 else PAGE_URL_TEMPLATE.format(n=n)
    await web page.goto(url, wait_until="domcontentloaded")
    await web page.wait_for_selector(".quote", timeout=10000)   # watch for JS to inject the quotes
    ...                                                       # learn the ten rendered .quote playing cards
    has_next = await web page.locator("li.subsequent a").depend() > 0
    if not has_next or n >= pages:                           # cease on the DOM, not a standing code
        break
    n += 1

Once more, the output grew to become a reusable CLI with --pages and --out, in a position to run with out an agent.

The payoff — the crafted CLI. As on the books run, the working script grew to become a reusable scrape_quotes(pages, out) software with an argparse interface (--pages, default 10; --out, default quotes.csv) that re-runs standalone, no agent within the loop.

Consequence: 100 quotes throughout 10 pages in 8.9 seconds, with zero empty textual content or creator fields.

The run additionally uncovered a nasty assumption in my transient: I anticipated two pages to provide 40 rows, borrowing the 20-per-page depend from the books website. This website serves 10, so the proper consequence was 20. The agent returned the actual knowledge and flagged the mismatch somewhat than forcing the output to suit the spec.

4.3 🔧 Check 3 — infinite scroll · quotes.toscrape.com/scroll

The third take a look at removes pagination completely. Quotes load 10 at a time through AJAX because the web page scrolls, so there aren’t any web page URLs to iterate by means of. The scraper has to scroll, wait for brand spanking new content material, measure the web page, and determine when loading is completed.

The agent first confirmed the location’s conduct: has_next turns into false at web page 10, and web page 11 returns no quotes. It then constructed a scroll-until-stable loop that stops when no new content material seems.

for i in vary(1, max_scrolls + 1):
    await web page.consider("window.scrollTo(0, doc.physique.scrollHeight)")
    await web page.wait_for_timeout(1000)
    depend = await web page.locator(".quote").depend()
    if depend == prev_count:          # no new quotes arrived
        stable_iters += 1
        if stable_iters >= 2:        # cease on stability, not a hard and fast depend
            break
    else:
        stable_iters = 0
    prev_count = depend

The DOM depend grew 10 -> 20 -> ... -> 100, held at 100 for 2 scrolls, and stopped at iteration 11. --max-scrolls was solely a security backstop. Forcing --max-scrolls 5 returned precisely 60 rows, proving the cease situation answered to the web page, not a hidden fixed.

Consequence: 100 quotes in 11 iterations, taking ~17 seconds.

Throughout the three assessments, the worth of code turns into clearer because the websites get tougher. Static pagination is easy; JavaScript requires an actual browser; infinite scroll requires this system to motive about when to cease.

There may be additionally a helpful cross-check: Rungs 2 and three scrape the identical 100 quotes by means of two completely different interfaces—pagination and infinite scroll—and produce matching outcomes row for row. That provides us a completeness test we are able to truly diff.

There are tradeoffs. Setup required a ~110 MB Firefox obtain and a few Home windows surroundings fixes. The Claude Code plugin additionally makes use of headless Firefox and its personal screenshot capabilities somewhat than Webwright’s customary Chromium setup. For a one-off click on, this method is overkill. The payoff seems when duties contain repetition, dynamic content material, or outcomes it is advisable confirm and reuse.

5. Conclusion

General, these three assessments present that browser-based code era is greater than a solution to automate clicks. The ultimate result’s a reusable Playwright program that may be run once more with out the agent. Because the web sites grew to become extra complicated, the generated code additionally grew to become extra succesful. It moved from easy web page loops to rendering JavaScript and at last to reasoning about when an infinite scroll had completed.

The experiments additionally present the worth of verification. The agent discovered bugs in its personal code, questioned incorrect assumptions within the activity description, and confirmed that the extracted knowledge was full. The matching outcomes from the paginated and infinite-scroll variations of the quotes website present further confidence that nothing was missed. That is tough to realize with a one-time browser recording alone.

There are prices. Working Playwright requires a browser obtain and extra setup than easy HTTP scraping or click on recording. The generated packages are additionally longer and require some technical data to grasp. Nevertheless, these prices are outweighed when the duty must be repeated, maintained, or verified. General, the assessments counsel that browser-based code era is a sensible method for constructing dependable net scrapers that may adapt to completely different web site designs whereas producing code that’s reusable, clear, and simple to check.

Webwright’s contribution isn’t an even bigger mannequin or a greater immediate. It’s a easier concept:

Give the mannequin a terminal, let it program the browser, and hold the consequence as reusable code.

That adjustments the web-agent loop. As a substitute of fragile clicks and fixed replanning, the agent can write, run, debug, and reuse a program.

The thought extends past browsers. When a mannequin can code and its surroundings can execute that code, it could be higher to write this system that performs the duty than predict each motion one step at a time.

The perfect net brokers don’t simply click on. They write the software—and go away it behind.

6. Sources

Webwright (major)

The panorama

Diagrams: author-created (matplotlib). Household animations: author-created. Screenshots and generated CLIs