Ranking web pages for browser agents, in the browser

October 20, 2026 · Madaminov Sardor

Browser agents spend most of their tokens reading pages. At every step, an agent that works from page text turns the page into a list of elements and sends the whole list to its language model. spotlit is a 6.7 MB model that ranks that list against the agent's goal inside the browser, so the language model reads 50 elements instead of hundreds or thousands.

On Mind2Web's held-out test websites [1], the element a person actually clicked is in spotlit's top 50 on 76 to 81% of steps, and the agent reads 32 to 43 times fewer tokens. Inside Claude Code, seven multi-page tasks took 6.3 times fewer input tokens with spotlit, and both versions finished all seven. This post covers how it works, how it was measured, and where it falls short.

The cost of reading a page

An agent loop has three parts: look at the page, let the language model choose an action, run the action. Looking is the expensive part. On the Mind2Web development split, a page step has a median of 514 candidate elements, and one in ten has 1,431 or more. Wikipedia's article on Japan has more than 2,000 visible elements, and the first 2,000 alone come to about 36,000 tokens as a list.¹

The cost also compounds. Each page listing stays in the conversation, so every later step reads every earlier page again. In one of our runs, an agent looking up three mountains on Wikipedia read 1.25 million input tokens over nine model turns, nearly all of it page listings it had already acted on.

Two stages

Mind2Web's own baseline, MindAct [1], splits the work in two: a small ranker narrows the page to a short list, and the language model chooses only among those candidates. spotlit keeps that split and adds one requirement: the ranker should run where the page already is, in the browser, without a server. That rules out a model with hundreds of millions of parameters. spotlit has 6.4 million, in two stages.

Describing an element

Each element is first turned into one line of text: its tag, then its labels (aria-label, title, alt, placeholder, name, type, role and value, 60 characters each), then up to 24 words of its own text, then a | and up to 12 words of its parent's text. The parent text is what tells an Add to cart button which product it belongs to.

Links, buttons, form fields, list items, headings, table cells and anything with a role, a click handler, a tab index or a label are kept. Lines shorter than two words are dropped.

Stage 1: hashed byte n-grams

Stage 1 scores every element on the page, so it has to be cheap. It has no tokenizer. Text is lower-cased, anything that is not a letter or digit becomes a space, and the bytes are cut into overlapping pieces of 3, 4 and 5 bytes. Each piece is hashed with 64-bit FNV-1a into one of 6,144 buckets, with a sign (the hashing trick [2]), and each bucket owns a row of 320 numbers in an INT8 table. An element's vector is the signed sum of its rows, normalized. The goal gets a vector the same way, and the score is their cosine.

The table started as a general text-search embedding distilled from bge-small-en-v1.5, and was trained on Mind2Web's training websites, one page per update, with a softmax over every candidate on the page: three epochs, about 20 minutes on a laptop with MLX. Alone, it puts the clicked element in its top 50 on 55% of development steps. What matters more is that the element is in its top 400 on 87.6% of steps, because stage 2 only sees those 400.

Stage 2: a small cross-encoder

Stage 1 embeds the goal and the element separately, so it cannot see how a word in one relates to a word in the other. Stage 2 can. It is google/bert_uncased_L-2_H-128_A-2 [3], a two-layer BERT with 4.4 million parameters, used as a cross-encoder: it reads [CLS] goal [SEP] element [SEP] as one sequence and returns one score. It was trained on batches of 8 page steps, each with the clicked element and 31 others from stage 1's top 400, for three epochs, about 6 minutes on a laptop GPU. As INT8 ONNX it is 4.49 MB and runs on ONNX Runtime Web's WebAssembly backend.²

A common question is how a model limited to 512 tokens can handle a long page. spotlit uses 128, and the limit applies to one goal and one element, never to the page. A longer page means more pairs, not longer ones. On the development split an element's line is a median of 18 tokens and a goal with its last three actions a median of 48, so only 2.0% of pairs go over 128, and 1.0% of the pairs that hold the clicked element. Those are cut from the longer side first.

Because stage 2 only re-scores 400 pairs, ranking time does not grow much with the page. In Chrome with eight threads, stage 1 takes 35 to 45 ms for 500 elements and stage 2 about 185 ms. Without threads it takes about a second. A four-layer BERT was 6 points more accurate on development data but took 1.16 s for the same 400 pairs.

Results on Mind2Web

Mind2Web records people completing tasks on real websites; at each step, one element was clicked. The question is how often that element is in spotlit's top k. Only the training split was used for training. The development split uses websites held out from training, and the three test splits were run once, after the model files were frozen.

Table 1. The released files on the three test splits. Keyword search is BM25; it and stage 1 are measured at the top 50.

Cross-task is highest because its websites appear in training; the other two splits agree with development within three points. The top 50 is 43, 34 and 32 times fewer tokens than each step's cleaned HTML. For comparison, the benchmark authors' ranker [1], a DeBERTa model with 184 million parameters, reports 88.9, 85.3 and 85.7% on the same splits. spotlit gives up 8 to 10 points to be 29 times smaller and to run without a server.³

Inside Claude Code and Codex

Benchmark recall is not the same as helping an agent, so we connected spotlit to two. A small MCP server gives the agent a Chrome window and seven tools: open, elements, click, type, select, press and back. Its two modes differ only in elements, which returns either every visible element or spotlit's top 50 for a goal the agent writes. The prompt and the other tools are identical, and the agents' own browsers, shells and web search are turned off.

Table 2. Cost at API prices. The full list was faster: time is mostly model turns, and the agent with the short list took a few more.

We also replayed 30 random Mind2Web development steps, asking the same model once with every element and once with spotlit's top 50. Claude picked the right element on 15 steps with the full list and 13 with spotlit's, while reading 964,166 tokens against 76,740. Codex, on the 23 steps it answered before its plan's usage limit, picked 12 against 9. Thirty steps is too few to tell a difference of two from noise, but the pattern is consistent: a little accuracy for a large saving in tokens.

What broke along the way

Claude Code replaces any tool result over about 50,000 characters with a file path and a 2 KB preview. With the full list, the agent lost sight of large pages and clicked ids it could not see, until the server raised the limit with _meta["anthropic/maxResultSizeChars"].

Clicking the centre of a link that wraps onto two lines hits the heading around it. On two catalogue pages, 13 of 31 product links missed; aiming at the middle of one of the link's own lines fixed all of them.

Interactive Claude Code can send its first prompt before an MCP server has registered its tools, so the model starts with none and writes its tool calls as text. A start-up hook that waits for the server's tool list fixed it.

Limitations

• About one step in four loses the clicked element from the top 50. The agent has to notice and ask again, with a narrower goal or a longer list, or fall back to the full page.
• In our tests spotlit often lifted search fields and filters for goals that had little to do with them, and next-page links tended to rank lower than they should.
• It narrows the page; it does not decide. Asked for the cheapest backpack, it surfaces prices, filters and buttons, and comparing them is the language model's job. Reading-heavy tasks need longer lists.
• It was trained on English websites from 2023. Other languages and very different page structures are untested.
• The reference collector does not look inside iframes or shadow DOM, and it only sees what the page has loaded.

Using it

The demo runs spotlit in your browser on saved copies of a real bookshop and a Wikipedia article, or on any HTML you paste. To use it yourself, download the weights, one 5.0 MB file, and load them with spotlit.js. The full open-weight release, including the full-precision re-ranker for Python, is on Hugging Face.

import * as ort from "onnxruntime-web";   // 1.30.0, WebAssembly backend
import * as spotlit from "./spotlit.js";

const ranker = await spotlit.loadPacked(ort, "spotlit-v1.weights.gz");
const top50 = await ranker.rank(goal, spotlit.collectCandidates(document), {
  k: 50, history: lastThreeActions,
});

The repository's browser-agent folder has the MCP server and the side-by-side runs used above.

Summary

A browser agent does not need to read every element to act well. A 6.7 MB model in the browser keeps the element a person would click in the top 50 about three times in four on websites it has never seen, and an agent using that list read 6.3 times fewer tokens on real multi-page tasks while still finishing them. The price is recall: when spotlit misses, the agent has to ask again. Closing the gap to a 184-million-parameter ranker without leaving the browser is the next step.

Citation

If you use spotlit or this post, please cite it as:

@misc{spotlit2026,
  title  = {{spotlit: an in-browser element ranker for web agents}},
  author = {Madaminov, Sardor},
  year   = {2026},
  month  = {October},
  url    = {https://github.com/Sardor-M/spotlit}
}

References

1. Mind2Web: Towards a Generalist Agent for the WebDeng, Gu, Zheng, Chen, Stevens, Wang, Sun and Su. NeurIPS 2023, Datasets and Benchmarks Track · arXiv:2306.06070 · code and dataThe training and test data, and the MindAct ranker spotlit is compared with.
2. Feature Hashing for Large Scale Multitask LearningWeinberger, Dasgupta, Langford, Smola and Attenberg. ICML 2009 · arXiv:0902.2206Signed hashing into a fixed number of buckets, as stage 1 does.
3. Well-Read Students Learn Better: On the Importance of Pre-training Compact ModelsTurc, Chang, Lee and Toutanova. 2019 · arXiv:1908.08962 · bert_uncased_L-2_H-128_A-2The BERT-tiny checkpoint stage 2 is fine-tuned from.

1. Tokens are estimated at four characters per token. Back
2. In ONNX Runtime Web 1.30 the INT8 graph returns wrong scores on WebGPU, so the re-ranker runs on WebAssembly. Back
3. MindAct's figures are from the Mind2Web paper. Its element descriptions differ, and spotlit counts steps whose clicked element is missing from the page HTML (4 to 7%) as misses, so the comparison is approximate. Mind2Web is CC BY 4.0; its authors ask that the test data never be redistributed or used for training, and spotlit was trained on the training split only. Back