Agent Readiness Study 2026: can an AI agent buy from a YC B2B startup?

I checked 295 YC B2B startup websites for what an AI agent needs to find, compare and buy. Almost none block AI crawlers. Most hide the price. By Saroj Jha, AAJ, 28 September 2026.

Download the data (CSV) — Check your own site with the Agent Readiness Quick Check

Key findings

• 99% let the main AI crawlers in. 293 of 297 robots.txt files allow GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot.
• 27% put a price where an agent can read it. 81 of 295 have a price in the raw HTML of their pricing page. Treat this as a ceiling.
• 39% offer no path but a demo or a sales call. 114 of 295 home pages. An agent cannot book its way through that.
• 20% show an agent a near-blank home page. 58 of 295 home pages have fewer than 150 words in the HTML before JavaScript runs.

The short answer

Almost nobody is blocking AI agents. 99% of the 297 robots.txt files I read let GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot in, and 99% of the sites served their home page to my identified crawler without a challenge. The door is open.

What an agent finds once it is inside is the problem. One home page in five (58 of 295) is close to blank until JavaScript runs. Only 27% put a price in the HTML of their pricing page, and 39% offer no next step except booking a demo or a call with a person.

Only 9% of the companies (27 of 299) pass all ten checks I could measure. The median company passes 57% of the measurable points.

Why I ran this

Buyers now hand part of their research to AI assistants: shortlist the tools that do X, compare their prices, tell me which have a free tier. When an agent does that work, the website is read by software first. I built the free Agent Readiness Quick Check and the agent-readiness-audit skill to score one site at a time. I wanted a baseline to score them against, from a public list anyone can re-run, not from a vendor survey.

So I took every company in Y Combinator's Winter, Spring and Summer 2025 batches that the public YC directory tags B2B, lists as active and gives a website for, and checked each website the way an agent would meet it: the raw HTML, before any JavaScript runs.

Access is solved

Blocking is rare and deliberate. The training crawlers are the ones named in robots.txt at all: CCBot by 4 sites, GPTBot and Google-Extended by 3, ClaudeBot and Applebot-Extended by 2. No site blocks OpenAI's, Anthropic's or Perplexity's search or user-fetch agents by name; one site blocks every crawler with a wildcard.

Only 52 sites have no robots.txt at all, which also means no block. Bot walls are just as rare: 295 of 299 sites served the home page without a challenge.

Reading is patchy

80% of home pages carry 150 or more words in the raw HTML. The rest depend on JavaScript to say anything, and an agent that does not run JavaScript sees a title and little else.

Structured data is a coin flip: 55% have any JSON-LD, 52% have Organization schema and 29% describe the product or an offer in schema. 73% have exactly one H1.

llms.txt is further along than I expected: 45% publish a real one (135 of 299), not an HTML page served at that address. I did not check which tool generated it.

Price is the gap

Only 33% have a pricing page an agent can reach from the home page or at /pricing, and 27% show a price in its HTML. Just 26% (77 of 295) are both readable and priced.

The sharpest miss is among the companies that want self-serve buyers. 41% of the sites with a sign-up or start-free path (42 of 102) show no price an agent can read. Their product is built to be bought without a salesperson, and their site cannot tell an agent what it costs.

Acting is mostly a human step

35% of home pages offer a self-serve path, and 39% offer only a demo or a sales call. On 76 home pages (26%) I found neither: no sign-up, no demo button, just content.

34% link to docs or an API from the home page, and 14% mention MCP, the protocol agents use to call tools. For a batch of AI-era companies, that is the number I expect to move fastest.

All ten checks

Partial score out of the 72 measurable points: median 57%, middle half between 38% and 74%.

Which AI agents robots.txt blocks

Out of 297 robots.txt files read (three sites did not answer).

What I would fix first

In order of cost, cheapest first where the payoff is similar. This is my judgement, not a finding of the study.

1. Put the price in the HTML. If you have plans, publish them on /pricing as text, not an image or a JavaScript widget. If you price on request, say so in words and give the starting point you are willing to state. An agent comparing five tools drops the one it cannot price.
2. Server-render the home and pricing pages. Most modern frameworks can pre-render these two pages. Check yours with a plain fetch of the page: if the answer is a title and an empty div, an agent sees the same.
3. Give an agent a next step that is not a calendar. A free trial, a sandbox, public docs or a clearly labelled self-serve plan. Keep the demo, but do not make it the only door.
4. Describe the company and the product in schema. Organization schema on the home page, and Product or SoftwareApplication with an Offer on the pricing page.
5. Last, and optional: llms.txt. A short markdown map of the pages that matter. Almost half of this batch already has one, but Google says Google Search, including AI Overviews and AI Mode, does not use it, so it is not a fix for Google. Do the four steps above first.

To score your own site on all fifteen checks, run the free Agent Readiness Quick Check, or give the agent-readiness-audit skill to your coding agent. If you want it done with you, that is the AI Visibility sprint.

How I ran it

Frame: every company in the YC Winter, Spring and Summer 2025 batches that the public YC directory tags B2B, lists as Active and gives a website for, read through the open yc-oss/api mirror; duplicate domains removed. That gave 300 companies.

Coverage: 299 sites reached, 295 home pages read, 4 returned an error status, and 1 was excluded because its robots.txt disallows the study's crawler.

Crawl: 28 September 2026. At most four URLs per company (robots.txt, home page, llms.txt, pricing page), each fetched once, with an identified user agent that links to this page. No forms, no logins, no browser impersonation.

Statistics: every rate is shown with its count and denominator, and a 95% Wilson interval. The denominator is 295 for home-page checks, 297 for robots.txt checks and 299 for checks that do not need the home page.

What this study cannot tell you

• I checked 16 of the automated readings by hand on 28 September 2026, and 13 were right. Both llms.txt files I opened were real, the blank home page was blank and the MCP mention was real. Two of ten price readings were wrong: a sample amount inside a demo conversation, and a table of other companies' model prices. One developer tool counted as demo-only actually starts from a Quickstart. So read the price rate and the demo-only rate as ceilings.
• The frame is YC's 2025 B2B batches: young, technical, venture-backed companies. It says nothing about established vendors or other markets.
• It is one crawl on one day. Websites change weekly, and this batch will change faster than most.
• The agent-readiness-audit skill scores fifteen checks. Ten can be measured by a script; the other five need judgement (for example, whether the positioning is clear enough to compare) and are not measured or scored here. The partial score is out of the 72 points those ten checks carry.
• A price in the HTML is not the same as a price an agent understands. The check finds a currency amount on the pricing page; it does not read the plan table.

Data and source

The dataset has one row per company, with every check, the pricing page used and the partial score. It is free to reuse with credit to AAJ, Agent Readiness Study 2026 (CC BY 4.0). Download the CSV. The company list comes from the public yc-oss/api mirror of the Y Combinator directory.