AHTML

AHTML is an open-source (MIT) snapshot format with TypeScript and Python toolkits that lets any website publish an agent-readable, token-efficient view of each page — typed entities plus typed actions with explicit cost, reversibility, auth, and side-effects — and auto-emit MCP, OpenAPI 3.1, JSON-LD, llms.txt, RSL, and Markdown from that single source, while browsers keep the same HTML. Agents can consume any site today, with or without adoption, via its universal extractor.

The contract layer of the agent web. One config in, every agent-readable protocol out — plus signed provenance, verified-agent auth, and priced actions.

At a glance — every number below is measured in this repo, not estimated:

• 5.6× fewer tokens than the HTML a browser loads on the flagship benchmark page (4.5–7.3× across the product / article / dashboard corpus, real OpenAI + Anthropic tokenizers) — WHY-AHTML.md, benchmark-results.md
• Fact-extraction accuracy rises from 91% on raw HTML to 100% on AHTML JSON in a real multi-model benchmark — 146 runs, 20 tasks, across gpt-4o-mini, claude-haiku-4.5, gemini-2.5-flash, and llama-3.3-70b — benchmark-results-llm.md
• 6/6 capability proofs executed live — MCP tools emitted, detached-JWS snapshot signed + verified, x402 402 payment flow built, RFC 9421 agent request verified, RSL 1.0 license emitted, Markdown negotiation — WHY-AHTML.md
• Sixteen npm packages under @ahtmljs at v1.1.0 plus the ahtml Python SDK — MIT, released with npm provenance, 700+ tests passing across both implementations
• Certifiable protocol — a language-agnostic conformance corpus covering every RFC-2119 MUST in the spec; TypeScript and Python both pass 100% through the same runner
• Rehearse before you pay — the SPEC §4.7 dry-run sandbox: agents simulate priced/irreversible actions (signed simulated: true responses, itemized cost, reversal path) before real money moves through x402

30-second quickstart:

npm install @ahtmljs/next @ahtmljs/schema

// app/ahtml/[[...path]]/route.ts
import { createAHTMLRoute } from '@ahtmljs/next/handler';
import { snapshot } from '@ahtmljs/schema';

export const { GET, HEAD } = createAHTMLRoute(async (segments, req) => {
  if (segments[0] !== 'products') return null;
  const p = await db.product.findUnique({ where: { slug: segments[1] } });
  return snapshot(req.url, 'product_detail')
    .ttl(60)
    .add({ id: `product:${p.slug}`, type: 'product', name: p.name,
           price: { amount: p.price, currency: 'USD' },
           stock: { status: p.qty > 0 ? 'in_stock' : 'out_of_stock' } })
    .action({ id: 'purchase', target: `product:${p.slug}`, category: 'transact',
              execute_url: '/api/checkout', auth: 'required',
              cost: { amount: p.price, currency: 'USD', category: 'purchase',
                      rails: ['x402'] },       // priced action (v0.9.5)
              reversible: { reversible: true, window: 'P30D' },
              side_effects: ['charge_card', 'decrement_stock'],
              confirmation: 'required' })
    .build();
});

Your Next.js app now serves MCP tools at /ahtml/mcp.json, OpenAPI at /ahtml/openapi.json, JSON-LD inline in HTML, a discovery manifest at /.well-known/ahtml.json, an llms.txt shim, RSL at /rsl.txt, a Markdown view over content negotiation, and a token-optimal agent snapshot at /ahtml/<route> — all from the same source.

Who is AHTML for? Two sides, two on-ramps

I'm a developer building an AI agent — do the sites I read need AHTML installed?

No. The consumer side works on any URL today, adopted or not:

npx @ahtmljs/cli extract https://any-website.com     # typed entities from JSON-LD/OpenGraph/microdata — no adoption needed
npx @ahtmljs/cli score https://any-website.com       # how agent-readable is this site? 0–100
npx @ahtmljs/cli mcp https://any-website.com         # turn ANY site into MCP tools for Claude/ChatGPT/Cursor

import { AHTMLClient } from '@ahtmljs/agent';
const page = await new AHTMLClient().fetchPage('https://any-website.com');
// AHTML snapshot when the site publishes one; extracted-from-HTML fallback when it doesn't.

pip install ahtml   # same consumer surface from Python — LangChain loader included

When the site does publish AHTML you additionally get: 5.6× fewer tokens, typed actions with cost/reversibility/auth contracts, signed snapshots you can verify, 304/diff refetches, and the dry-run sandbox before money moves.

I run a website (the supplier side) — what do I get for publishing AHTML?

One command wires it, and every protocol falls out of a single source:

npx @ahtmljs/cli init     # detects Next/Astro/SvelteKit/Vite/Hono, wires everything
npm run dev
npx @ahtmljs/cli doctor http://localhost:3000   # green = agents can use your site

What you get: your site becomes an MCP server (/ahtml/mcp.json) that Claude, ChatGPT, and Cursor can call; an OpenAPI 3.1 surface; JSON-LD; llms.txt; a token-optimal snapshot that makes agent answers about your content more accurate (91%→100% in our benchmark); typed, priced, policy-gated actions agents can execute safely (x402 payments, RFC 9421 verified agents); traffic insights showing which agents actually read you; a score badge for your README; and a listing in the AHTML Index so agents can find you.

What is AHTML?

AHTML is a web standard for AI agents: an agent-readable web layer that makes your existing site's content legible — and safely actionable — for LLM-powered assistants. If you are searching for a token-efficient HTML alternative, an llms.txt alternative with typed actions, MCP integration for an existing website, or a way to publish web content for LLMs without running a second server, that is exactly the problem AHTML solves.

The web that browsers see and the web that agents see are diverging. Browsers render pixels. Agents pay for tokens. A modern product page can ship 300 KB of nav, footer, tracking, and ad chrome — and an autonomous shopping agent pays for every byte. AHTML lets your site publish a typed semantic snapshot alongside its HTML: entities with stable IDs, actions with explicit cost / reversibility / auth / side-effects, freshness metadata, site-wide policy, cryptographic provenance via detached JWS (v0.8), and — as of v0.9.5 — verified-agent authentication (RFC 9421 signed requests) and priced actions (x402 machine payments + RSL 1.0 licensing).

The plugin auto-generates MCP tool manifests, OpenAPI 3.1 documents, JSON-LD fragments, the llms.txt discovery convention, an RSL 1.0 license file, and a Markdown view — all from the same source. Browsers see the same HTML. AHTML is additive — there is no migration.

And the whole toolchain works on any URL today, not just adopters: the @ahtmljs/cli and @ahtmljs/agent extract typed objects from ordinary HTML (schema.org + OpenGraph + microdata + data-attrs) when a site hasn't adopted AHTML yet.

The sixteen packages (plus Python)

Sixteen npm packages under the @ahtmljs scope, released together, plus the ahtml Python SDK. Grouped by what you're building:

Contract layer

Make your site agent-readable (site adapters)

Read the agent web (agent-side)

Tooling & infrastructure

Common install combos:

# site owner — make your site agent-readable (Next.js)
npm install @ahtmljs/next @ahtmljs/schema

# same, on the edge / Cloudflare Workers
npm install @ahtmljs/hono @ahtmljs/schema hono

# scaffold ANY supported framework in one command (Next/Vite/Hono/Astro/SvelteKit)
npx @ahtmljs/cli init

# agent author — read the agent web (with HTML fallback for non-adopters)
npm install @ahtmljs/agent @ahtmljs/schema

# agent author in Python
pip install ahtml

# audit / score / proxy any URL — no install
npx @ahtmljs/cli analyze https://example.com

Works on any URL today — the CLI

You don't need a site to adopt AHTML to get value. @ahtmljs/cli extracts typed objects from ordinary HTML and turns any site into MCP tools:

npx @ahtmljs/cli analyze  https://example.com   # bytes → tokens → savings %, entity counts, agent-readiness probe
npx @ahtmljs/cli score    https://example.com   # Lighthouse-for-agents: 0–100 score, A–F grade, copy-paste fix
npx @ahtmljs/cli doctor   https://example.com   # audit the AHTML discovery chain + verify signatures
npx @ahtmljs/cli extract  https://example.com   # schema.org + OpenGraph + microdata + data-attrs → snapshot
npx @ahtmljs/cli benchmark https://example.com  # HTML vs JSON-LD vs AHTML compact vs AHTML JSON table
npx @ahtmljs/cli mcp      https://example.com   # stdio MCP proxy — any URL becomes typed MCP tools in Claude/Cursor
npx @ahtmljs/cli llms     https://example.com   # crawl a site → spec-compliant llms.txt

ahtml mcp <url> is claude mcp add-compatible: it probes /.well-known/ahtml.json and proxies to the real endpoint for adopters, and auto-extracts from plain HTML for everyone else. Four universal MCP tools — fetch_page, list_pages, search, and invoke_action (adopters).

One config, every protocol

A single buildSnapshot function feeds every output below. No parallel implementations.

Every wire format is content-negotiated. All of them come from the same TypeScript object.

Why this exists — concrete numbers, not adjectives

The token-only benchmark was measured with the same tokenizers OpenAI and Anthropic use internally (gpt-tokenizer, @anthropic-ai/tokenizer) — no text.length / 4 guessing. The accuracy benchmark issues real API calls across gpt-4o-mini, claude-haiku-4.5, gemini-2.5-flash, and llama-3.3-70b. See benchmark-results-llm.md and examples/llm-benchmark/ to reproduce (~$0.10–0.50 in API spend), or run npx @ahtmljs/cli benchmark <url> for a one-command table on any live page.

For the head-to-head "why AHTML wins" story — token savings plus capability proofs executed live (MCP tools emitted, a snapshot signed and verified, an x402 402 built, an RFC 9421 agent request verified) — see WHY-AHTML.md, regenerated by examples/why-ahtml/ (npm run benchmark:why).

Comparison

Why AHTML vs llms.txt / MCP / raw HTML

The short version, sourced from the measured benchmarks and docs/compare.md:

AHTML is not a competitor to MCP or llms.txt — it emits both from one config. Use a hand-built MCP server for a purpose-built tool surface with no website behind it; use AHTML when you already have a website.

vs Firecrawl / ScrapingBee / Jina Reader / r.jina.ai / Spider / Browserbase / Diffbot / Cloudflare auto-markdown

These convert somebody else's HTML into LLM-friendly markdown by scraping it (Cloudflare now does it at the CDN with Accept: text/markdown). They are good tools. AHTML solves the inverse problem: let the site publish the agent-readable view itself, with typed actions, ETag, a signature, and a price. If you are the site owner, you should not need a third party to scrape your own pages — and AHTML also serves text/markdown when a client asks for it, so it's a superset, not a competitor.

vs Anthropic MCP SDK / OpenAI MCP SDK / FastMCP / mcp-framework / Smithery

These help you build an MCP server from scratch, as a separate process, with its own auth and deploy story. AHTML makes your existing Next.js, Vite, or Hono app emit MCP — same database, same auth, one deploy. The MCP tool manifest is generated from the same snapshot that already powers your agent-facing endpoint. And ahtml mcp <url> turns any site — adopter or not — into MCP tools for your Claude/Cursor session today.

vs schema.org JSON-LD / llms.txt

AHTML ingests schema.org for a free Level-0 snapshot on most Shopify/WordPress sites, emits llms.txt as a compatibility shim, and adds the typed action surface that both lack.

vs LangChain CheerioWebBaseLoader / Mozilla Readability / trafilatura / Unstructured.io

Those are HTML cleaners — they strip chrome and return text. AHTML returns a structured object with stable IDs, typed actions, and freshness — so the LLM does not have to guess what the page is about. The @ahtmljs/langchain loader is a drop-in replacement for CheerioWebBaseLoader when the upstream site emits AHTML.

A longer comparison — including WebMCP, NLWeb, RSL, x402, and Content Signals — is in docs/compare.md.

Install in 3 minutes

Next.js App Router shown; the same three steps apply to @ahtmljs/vite and @ahtmljs/hono.

1. Install.

npm install @ahtmljs/next @ahtmljs/schema

2. Declare snapshots.

// lib/ahtml.ts
import { snapshot } from '@ahtmljs/schema';

export async function buildSnapshot(segments: string[], req: Request) {
  if (segments[0] === 'products' && segments[1]) {
    const p = await db.product.findUnique({ where: { slug: segments[1] } });
    if (!p) return null;
    return snapshot(req.url, 'product_detail')
      .ttl(60)
      .add({
        id: `product:${p.slug}`,
        type: 'product',
        name: p.name,
        price: { amount: p.price, currency: p.currency },
        stock: { status: p.qty > 0 ? 'in_stock' : 'out_of_stock', quantity: p.qty },
      })
      .action({
        id: 'purchase',
        target: `product:${p.slug}`,
        category: 'transact',
        execute_url: '/api/checkout',
        auth: 'required',
        cost: { amount: p.price, currency: p.currency, category: 'purchase' },
        reversible: { reversible: true, window: 'P30D', policy: 'full_refund' },
        side_effects: ['charge_card', 'email_buyer', 'decrement_stock'],
        confirmation: 'required',
      })
      .build();
  }
  return null;
}

3. Wire the routes.

// app/ahtml/[[...path]]/route.ts
import { createAHTMLRoute } from '@ahtmljs/next/handler';
import { buildSnapshot } from '@/lib/ahtml';
export const { GET, HEAD } = createAHTMLRoute(buildSnapshot);

// app/.well-known/ahtml.json/route.ts
import { createWellKnownRoute } from '@ahtmljs/next/well-known';
export const { GET } = createWellKnownRoute();

// app/llms.txt/route.ts
import { createLlmsTxtRoute } from '@ahtmljs/next/llms-txt';
export const { GET } = createLlmsTxtRoute();

Done. Your site now emits MCP, OpenAPI, JSON-LD, llms.txt, the discovery manifest, and the typed snapshot — and serves text/markdown over content negotiation. Browsers see the same HTML they always did. To require verified agents on sensitive routes, add verifyAgents + agentKeys to the handler config; to price an action, add cost.rails: ['x402'] and gate its route with withPaymentGuard.

Reading the agent web — @ahtmljs/agent

For agent authors. Typed errors, request coalescing, retry-with-backoff, ETag-conditional fetch, dry-run, real-tokenizer cost estimate, streaming, and a universal fetchPage() that falls back to HTML extraction on non-adopters.

import { AHTMLClient } from '@ahtmljs/agent';

const client = new AHTMLClient({
  retry: { attempts: 3, baseMs: 200 },     // jittered exponential backoff
  timeout: 5_000,                          // per-attempt timeout
  coalesce: true,                          // dedupe concurrent in-flight fetches
  cache: 'memory',                         // or a pluggable CacheStore<T>
});

try {
  const snap = await client.fetch('https://shop.example.com/ahtml/products/widget');
  console.log(client.tokenize(snap, 'o200k_base'));  // real tokenizer cost
  const next = await client.fetch(snap.url, { since: snap.etag }); // diff replay

  // Universal read — returns a typed PageView even on plain-HTML sites.
  const page = await client.fetchPage('https://any-shop.example.com/widget');
  console.log(page.products, page.provenance); // 'authoritative' | 'extracted'
} catch (e) {
  // 13 stable error codes — type-narrowable at the call site
  if (e.code === 'RATE_LIMITED') retryLater(e.retryAfter);
  if (e.code === 'SIGNATURE_INVALID') refuse();
}

The error taxonomy (v0.6+) has 13 stable codes: SCHEMA_INVALID, DIFF_INVALID, COMPACT_PARSE, JSON_PARSE, ETAG_MISMATCH, NETWORK, HTTP_STATUS, AUTH_REQUIRED, POLICY_DENIED, RATE_LIMITED, TIMEOUT, CACHE_POISONED, SIGNATURE_INVALID. Extracted snapshots never carry actions (untrusted markup).

Compatibility & runtime

• Node 18+, dual ESM + CJS (as of v0.9.1) — both import and require('@ahtmljs/…') work; CI matrix is 18 / 20 / 22
• Edge-runtime safe: zero node:* imports in the @ahtmljs/schema hot path
• Tested on Cloudflare Workers, Vercel Edge, Bun, Deno, and AWS Lambda (via @ahtmljs/hono)
• OpenTelemetry tracing across the route handler and client (v0.9.0); did:web key resolution for zero out-of-band key distribution
• Specs: MCP 2025-11-25, OpenAPI 3.1, JSON Schema 2020-12, JWS (RFC 7515), HTTP Message Signatures (RFC 9421), x402/0.2, RSL 1.0, Content Signals, WebMCP (W3C WebML CG), llms.txt convention (Howard, 2024)
• License: MIT. Every release ships with npm provenance (sigstore attestation via GitHub Actions)
• 700+ tests passing across the monorepo — all sixteen packages, the Python SDK, plus the UX, conformance, and performance-budget suites, all green in CI

Compatible MCP clients that can consume /ahtml/mcp.json directly: Claude Desktop, Claude on the web, ChatGPT (Apps SDK + Connectors), Cursor, Continue, Cline, Aider, Microsoft Copilot (M365, GitHub), Gemini API + Vertex AI Agent Builder, Goose, Witsy, Zed AI, and any framework with MCP support (LangGraph, CrewAI, AutoGen).

Certified implementations

Any implementation can certify against the language-agnostic conformance corpus and publish a signed attestation — see docs/conformance.md.

FAQ

Longer, categorized version in docs/faq.md.

What is AHTML?

AHTML is an open-source (MIT) snapshot format and TypeScript toolkit that lets any website publish an agent-readable, token-efficient view of each page — typed entities plus typed actions with explicit cost, reversibility, auth, and side-effects — and auto-emit MCP, OpenAPI 3.1, JSON-LD, llms.txt, RSL, and Markdown from that single source, while browsers keep the same HTML.

Is AHTML a replacement for MCP?

No — AHTML emits MCP. MCP is the agent's tool-calling protocol; AHTML is the per-page contract that auto-generates an MCP manifest at /ahtml/mcp.json from your existing site, so you don't run a separate MCP server with parallel auth and deploys.

How is AHTML different from llms.txt?

llms.txt is unstructured markdown — useful as a sitemap for IDE agents, but it can't express typed entities or executable actions. AHTML auto-emits llms.txt as a compatibility shim and adds the typed contract; in the real-LLM benchmark llms.txt scored 89% on fact extraction vs 100% for AHTML JSON.

How many tokens does AHTML save vs raw HTML?

Measured with the real OpenAI and Anthropic tokenizers: 4.5–7.3× fewer tokens on the lean benchmark corpus (5.6× on the flagship page in WHY-AHTML.md). On production-bloat pages (200–500 KB of HTML) the ratio scales toward 50–100×, because the snapshot stays near ~2 KB.

Does AHTML make LLM agents more accurate?

Yes — in the multi-model benchmark (benchmark-results-llm.md, 146 runs, 20 fact-extraction tasks across gpt-4o-mini, claude-haiku-4.5, gemini-2.5-flash, llama-3.3-70b), accuracy rose from 91% on raw HTML to 100% on AHTML JSON.

Do I need to migrate my site to use AHTML?

No. AHTML is additive: it adds endpoints (/ahtml/*, /.well-known/ahtml.json, /llms.txt) next to your existing routes, and the HTML you serve to browsers is unchanged.

Which frameworks does AHTML support?

Next.js 14+/15 App Router (@ahtmljs/next), Astro (@ahtmljs/astro), SvelteKit (@ahtmljs/sveltekit), other Vite-based apps such as SolidStart (@ahtmljs/vite), and Hono on Node, Bun, Deno, Cloudflare Workers, and AWS Lambda (@ahtmljs/hono) — or use @ahtmljs/schema directly with hand-rolled routes in any framework. npx @ahtmljs/cli init detects your framework and wires everything for you.

Does AHTML work on sites that haven't adopted it?

Yes. @ahtmljs/cli and @ahtmljs/agent extract typed snapshots from ordinary HTML (schema.org + OpenGraph + microdata + data-attributes), and npx @ahtmljs/cli mcp <url> turns any URL into MCP tools today.

How do agents know an AHTML snapshot wasn't tampered with?

Snapshots can carry a detached JWS signature over the canonical JSON form (shipped in v0.8), verifiable against a did:web identity — and sites can require verified agents via RFC 9421 signed requests.

Can I charge AI agents for actions or license my content?

Yes. Actions declare typed cost with rails: ['x402'] and a standards-compliant HTTP 402 payment flow, and sites emit an RSL 1.0 license file plus Content Signals declarations (all shipped in v0.9.5).

Will AHTML slow down my site?

No. Snapshots are shaped at request time from data your route already has — no parsing or scraping — and ETag-conditional GET plus the ?since=<etag> diff endpoint keep repeat fetches cheap.

How much does AHTML cost?

Nothing — all sixteen packages (and the Python SDK) are MIT-licensed open-source libraries that run inside your own app. There is no SaaS, no per-request pricing, and no lock-in.

Roadmap

The 0.9.x series and the 1.0.0 cut are sequenced and dated in PLAN-NEXT-6.md. Headline:

Performance budgets are enforced in CI per release — the benchmark is a failing test, not a paragraph. See PLAN-NEXT-6.md for numeric limits per version.

Project structure

ahtml/
  README.md                 this file
  SPEC.md                   formal snapshot spec
  PLAN.md                   long-range phased build plan
  PLAN-NEXT-6.md            the 0.9.x series → 1.0.0 — dated, sequenced
  LANGUAGE.md               .ahtml source language preview (Phase 2)
  CHANGELOG.md              per-release notes
  llms.txt / llms-full.txt  ingestion files for AI assistants
  benchmark-results.md      token-only benchmark output
  benchmark-results-llm.md  real-LLM accuracy benchmark output
  WHY-AHTML.md              competitive benchmark — tokens + executed proofs

  ROADMAP.md                the post-1.0 "10x" series (1.1–1.4)

  packages/
    schema/                 @ahtmljs/schema      — the contract layer
    extract/                @ahtmljs/extract     — extractor pipeline + plugin API
    next/                   @ahtmljs/next        — Next.js adapter
    astro/                  @ahtmljs/astro       — Astro adapter
    sveltekit/              @ahtmljs/sveltekit   — SvelteKit adapter
    vite/                   @ahtmljs/vite        — Vite adapter
    hono/                   @ahtmljs/hono        — Hono / edge adapter
    agent/                  @ahtmljs/agent       — agent client SDK (+ dry-run sandbox)
    langchain/              @ahtmljs/langchain   — LangChain.js loader
    cli/                    @ahtmljs/cli         — init / analyze / score / doctor / badge / submit / conformance / mcp / llms
    kv/                     @ahtmljs/kv          — KV / cache / rate-limit backends
    webmcp/                 @ahtmljs/webmcp      — WebMCP browser tools
    insights/               @ahtmljs/insights    — agent-traffic analytics
    conformance/            @ahtmljs/conformance — corpus + certification runner
    index/                  @ahtmljs/index       — the AHTML Index (registry + crawler)
    badge/                  @ahtmljs/badge       — hosted score-badge service

  python/                   ahtml (PyPI)         — Python consumer SDK

  examples/
    landing/                dogfood Next.js site
    benchmark/              token-only benchmark
    ll