Your Website Has a New Visitor, and It Cannot Click
For the last thirty years, the web was built for two audiences: humans and search engine crawlers. Humans clicked buttons, scrolled past hero sections, and tolerated loading spinners. Crawlers indexed pages so humans could find them later. Both audiences, in the end, served the same goal: getting a person in front of your content.
That assumption is now wrong.
A growing share of traffic to every website is no longer human. It is Claude, ChatGPT, Cursor, Copilot, and the agents built on top of them, visiting your site to answer a question, write code, or complete a task on someone's behalf. These visitors do not scroll. They do not click. They read the raw response, extract what matters, and move on. If your site is not built for that kind of visitor, you are invisible to an entire category of users that is growing faster than any other.
This is the problem Vercel's Agent Readability Specification addresses, and it is the reason we spent the last few days rebuilding how Metera presents itself to the world.
Three Questions Every Agent Asks
When an AI agent lands on a website, it is silently asking three questions in sequence.
Can I find what exists here? Humans navigate through menus and search bars. Agents look for an index, which is a file that tells them, in plain language, what pages exist and what each one contains. Without that index, the agent has to crawl link by link, guessing at structure, burning time and tokens on pages that might not even be relevant.
Can I read this without fighting the page? A modern web page is mostly machinery, things like navigation bars, animation wrappers, cookie banners, and hydration scripts. The actual content might be five percent of the bytes that arrive. An agent has to strip all of that away just to get to the paragraph that answers the question. Every layer of unnecessary markup is a tax the agent pays before it can do anything useful.
Do I understand what this means? Headings, structured data, and clear terminology tell an agent what kind of content it is looking at and how the pieces relate to each other. A page that is just a wall of text with no hierarchy forces the agent to guess where one idea ends and another begins.
Discovery, structure, and context. Get these three right, and your site becomes something an agent can use confidently. Get them wrong, and the agent either gives up, hallucinates an answer based on partial information, or simply never surfaces your content at all.
The Standard Nobody Announced, Everyone Adopted
What makes this moment interesting is that there was no committee meeting, no formal RFC process, no industry consortium that decided how agent-readable sites should work. Instead, a handful of conventions emerged organically and were adopted, almost simultaneously, by the companies most exposed to agentic traffic.
llms.txt works like a sitemap, but written for language models instead of search engines, a single file at the root of a domain that lists the pages worth reading, in the order they matter. AGENTS.md (and its cousins CLAUDE.md, .cursorrules) gives coding agents a direct answer to "how do I install, configure, and use this," without having to infer it from scattered documentation pages. Markdown mirrors let an agent request Accept: text/markdown and receive the same content as the HTML page, minus the framework noise.
OpenAI, Anthropic, Stripe, Cloudflare, Mastercard, and Vercel itself all ship these files. Chrome's Lighthouse auditing tool added checks for them this year. None of this was mandated. It happened because the companies building agents and the companies being visited by agents converged, independently, on the same simple idea: if you want to be understood by something that reads markdown natively, give it markdown.
What We Changed at Metera
Metera already had a head start here, in a sense. Our entire product is built around the idea that an AI agent should be able to read a single document and immediately understand how to pay for an API, that is what our skill.md protocol does for agent wallets. An agent reads one URL, and it knows its wallet address, its spending limits, and how to call the payment proxy. No SDK, no configuration file, no human in the loop.
But the protocol only solved the problem for agents that were already inside Metera. The website itself, the front door, was still built the old way: a landing page optimized for a person scrolling on a Tuesday afternoon, with no equivalent front door for an agent trying to figure out what Metera even is.
So we extended the same philosophy outward. Metera now serves an llms.txt at its root that indexes every page worth reading, including the marketplace, the pricing structure, and the documentation for providers and for agents, each with a markdown counterpart so an agent never has to parse a rendered React tree to find a sentence. An AGENTS.md file gives any coding agent landing on the repository or the site a direct, copy-pasteable answer to "how do I register an API" or "how do I create an agent wallet," with working code examples instead of prose that assumes a human will translate it into commands.
We made sure robots.txt explicitly welcomes the crawlers that matter now, such as ClaudeBot, GPTBot, and CCBot, rather than leaving the question ambiguous. We added structured data to the homepage so an agent doesn't have to guess that Metera is a piece of software, what it costs, or when it was last updated. And we made sure that every page traces back to the sitemap, because a page an agent cannot find is a page that, for an enormous and growing slice of the internet, does not exist.
Why This Matters More for Infrastructure Than for Anything Else
There is a version of this argument that applies to every website, things like better discoverability, better citations, or a small bump in some abstract visibility metric. For a content site, that might be the whole story.
For Metera, it is something closer to the point.
Metera's actual users are not people browsing a marketplace with a mouse. They are AI agents that need to discover an API, understand its price, and pay for it autonomously, in the same motion. An agent that cannot read our marketplace cannot use our marketplace. An agent that has to guess at our pricing model cannot reason about whether a call is worth making. The agent-readable website is not a marketing nicety sitting on top of the product, for an x402 platform built for agents, it is the product, extended one layer outward, to the moment before the agent has even connected a wallet.
HTTP 402 sat unused for thirty years because nothing on either end of the connection knew what to do with it. We think the same quiet shift is happening again, at the layer just above payments: the websites that agents can actually read are the ones that will get used, recommended, and built on top of. The ones that cannot will simply not come up.
We'd rather be on the right side of that, starting now.
Metera is billing infrastructure for AI agents on Solana. Any API can charge agents automatically in USDC via the x402 protocol. No code changes, no KYC, no bank. metera.xyz or point your agent at metera.xyz/AGENTS.md.