The TEN7 AI Readiness Index

Version 2.0 — specification

What this is

This document defines what a website has to do to score 100 on the TEN7 AI Readiness Index. It is written for two readers at once: the engineer building the scanner in this repository, and the consultant explaining a score to a client.

Every check has an ID, a weight, a detection method, and a scoring rubric. If a check is not described here, the scanner should not score it.

We built this because "is your site ready for AI" is currently an unanswerable question. Vendors assert things. Nobody measures. Our answer is a number you can reproduce, defend line by line, and re-run in six months to show what changed.

The Index measures one thing: can machines find, read, trust, and use this website? It does not measure design quality, marketing effectiveness, or whether the content is any good. A beautiful site can score 12. A plain one can score 94.

How to read a score

A result has three parts, and all three matter.

The score (0–100). A single weighted number across eight dimensions.

The gate status. Two checks are gates. If a site blocks AI crawlers in robots.txt or rejects them at the CDN, the total is capped at 40 no matter how good the rest is. A perfectly marked-up site that returns 403 to ClaudeBot is not ready — it is invisible. The cap makes that impossible to hide behind a good average.

The automated coverage. The percentage of applicable weight the scanner verified on its own. Some checks — log retention, a monthly prompt panel, whether schema is generated from fields or hardcoded in a template — cannot be seen from outside. Those are scored from attestation and flagged. A score of 82 with 91% automated coverage is a much stronger claim than 82 with 64%.

Report all three together. A bare number invites arguments we don't need to have.

Grade bands

When a gate fails, grade is Gated. The result also carries ungated_grade — the band the ungated score would have earned — so the report can say "Gated, and Strong at 81.2 once access is fixed" rather than leaving the consultant to supply the adjective.

The scoring model

Bands

Every check scores on a 0–4 band. The check's contribution is weight × (band ÷ 4).

Unless a check says otherwise, use the default coverage rubric against the sampled page set:

Coverage percentages are weighted, not raw page counts. Each money page (see Sampling, step 4) counts as 2 in both the numerator and the denominator; every other sampled page counts as 1. Ten money pages in a 250-URL sample therefore score as a 260-page sample. A failure on /apply costs twice what the same failure on a 2019 press release costs, which is the reason the client names those URLs in the first place.

Checks that are inherently site-wide and binary (a file exists, a header is set) define their own bands in the check entry.

Weights

Dimensions 1, 3 and 5 carry 62 of the 100 points. That is deliberate. Access, extraction, and entity identity determine whether anything else matters.

Dimension 8 carries only 3 points despite being the most commercially valuable work we do. Weight here reflects what a scanner can observe from outside a site, not what the work is worth. Track measurement maturity in the engagement, not in this number.

Applicability and N/A

Each check declares an applicability rule:

• always — scored on every site.
• conditional — scored only if a precondition holds. State the precondition in the check.
• vertical — scored only for a declared client vertical (higher ed, healthcare, nonprofit, government, research).

A conditional check states its precondition in one of two forms, and never in both:

• A named site fact — a single named predicate over collected evidence, such as sample_contains_tables or site_has_search. Predicates are code, not expressions in a config file, so each one is unit-tested and each one names itself in the N/A reason an auditor reads.
• A dependency on another check — written as requires: AR-1.6 ≥ 2. A dependency is satisfied when the named check scores band 2 or higher. Below band 2 the foundation is broken, and measuring what sits on top of it produces a finding nobody can act on: telling a client their malformed llms.txt is well cached is noise. A dependency that is itself N/A cascades N/A to everything that depends on it.

Five checks carry a dependency: AR-1.7 → AR-1.6, AR-4.4 → AR-4.3, AR-4.6 → AR-4.1 or AR-4.3, AR-7.2 → AR-7.1, AR-8.4 → AR-8.3.

Facts the scanner cannot observe

Six checks are conditional on a business fact no crawler can determine: AR-1.6 and AR-1.8 (licensing and access posture), AR-6.3 (YMYL content), AR-6.5 (research and policy publishing), AR-6.6 (original asset production), and AR-7.8 with AR-7.9 (named critical flows). AR-5.5 needs a declared vertical.

These come from an engagement profile supplied per client — never from inference. A fact not declared is N/A with the reason not_declared_in_engagement_profile. The scanner does not sniff a site for medical keywords and decide it is YMYL. A guess inside a number we ask a client to defend line by line is worse than a blank row, and a blank row is an easy conversation: we did not assess this because you have not told us yet.

A pre-sales scan run before any client contact will therefore leave those seven checks unassessed. The score renormalizes correctly around them.

A check ruled N/A is removed from the denominator. Recompute:

score = (Σ earned_points ÷ Σ applicable_weight) × 100

Never score an N/A check as zero. Penalizing a museum for having no Physician markup makes the Index useless.

Record every N/A decision with its reason in the result file. An auditor must be able to see what we chose not to measure.

Gates

Two checks are gates: AR-1.1 and AR-1.2.

If either scores 0 or 1, cap the final score at 40 and set gated: true. Report the uncapped score alongside it as ungated_score and ungated_grade, so the client can see what the site would be worth once access is fixed. That contrast is usually the most persuasive number in the whole report.

The cap is a floor-preserving minimum: total = min(ungated_score, 40). A site scoring 22 that also fails a gate stays at 22. Failing a gate never raises a score.

Verification mode

Every check declares how its band was determined:

• auto — the scanner determined the band from evidence it collected.
• auto-partial — the scanner produced a strong signal, but a human must confirm. Heuristics that can be fooled live here.
• manual — not observable from outside. Scored from a client attestation, or left null and excluded.

Automated coverage counts auto-partial at half weight, because a strong signal awaiting human confirmation is genuinely half-verified:

automated_coverage = (Σ auto_weight + 0.5 × Σ auto_partial_weight) ÷ Σ applicable_weight

The half is not arbitrary. Counting auto-partial at zero puts a ceiling of 0.83 on every possible run, because the always-applicable auto-partial checks are 15.75 points that never leave the denominator. Counting it in full reports 0.96 for a site nobody has confirmed anything about, and erases the distinction the mode exists to draw.

Never let a manual check silently default to 4. An unanswered attestation scores null and drops out of the denominator, and the result records it as unverified.

Attestation against an auto-partial check

Four auto-partial checks have a top band that asks something no scanner can see. This document says so in each case, and each has a different observable ceiling because their band tables differ:

Each of these four now has a band 3 that describes exactly what a scanner can see, so the attestation lifts 3 to 4 rather than jumping a gap. A check whose band table skips a value cannot express mostly right, and mostly right is the state most real sites are in.

An attestation may raise the ceiling on these four. It never sets a band and never lowers one: the scanner scores what it observed, and a supplied attestation permits that observed band to rise as far as the evidence supports. The result records which points came from a claim rather than a probe, and the report prints them as attested.

Without an attestation the check scores at its observable ceiling and the report says what would be needed to go higher. A client who did the work can prove it; a client who says nothing earns nothing.

Every other auto-partial check is fully observable. It carries that mode because the heuristic can be fooled — AR-1.8 compares a signed request against an unsigned one, AR-7.3 calls a live tool — not because it needs a client to vouch for it.

Checks that need a previous run

Four checks are defined against change over time: AR-2.5 (is lastmod a real signal or deploy noise), AR-3.6 (do anchor IDs survive a deploy), AR-6.4 (do dates move only when content moves), and AR-7.9 (do critical-flow selectors churn).

On a first run there is nothing to compare against. Score the observable half, cap the band at 3, set confidence: medium, and record stability_unconfirmed in the evidence. AR-6.4 already words its band 3 this way; the other three follow the same rule.

Supplying a previous run directory as a baseline lifts the cap and permits band 4.

Every ceiling — this one, the attestation ceiling above, and a check's own declaration that it could not see further — records the reason it applied. A client told a number was held below the evidence is owed the sentence explaining what would lift it. This makes the re-audit a first-class input to the score rather than a report-writing exercise, and it gives the client a visible, earnable reason the number rises next quarter.

How the audit runs

Sampling

Most checks score against a sampled page set, not the whole site. Build the sample like this:

1. Always include: the homepage, /robots.txt, /sitemap.xml (and every child sitemap), /llms.txt, /.well-known/ probes, and every URL in the site's primary navigation.
2. Stratify by content type. Use the sitemap index, URL patterns, or a declared content-type map. Take up to 10 URLs per type.
3. Add depth. Include at least 5 URLs three or more levels deep, to catch templates the homepage never exercises.
4. Add the money pages. The client names up to 10 URLs that matter most — programs, services, donate, apply, contact. These are weighted double in coverage math.
5. Cap the crawl. Default 250 URLs. Configurable. Record the actual count.

Minimum viable sample is 25 URLs. Mark the run low_confidence only when the sample is under 25 and more URLs were discovered than were sampled. A twelve-page nonprofit site that was crawled in full is a complete measurement, not a weak one.

The representation sub-sample

Four checks want an extra request against every sampled URL: AR-4.3 ({path}.md), AR-4.5 (Accept: text/markdown), AR-4.6 (cache behavior), and AR-1.9 (X-Robots-Tag on non-HTML). At the default 250-URL cap and one request per second, that is over 800 requests and roughly fifteen minutes of politeness delay stacked on top of the base crawl, against infrastructure that often belongs to a hospital.

Those four checks run against a representation sub-sample: 25 URLs by default, drawn from the main sample, always including every money page. The size is configurable and the selected URLs are recorded in the run metadata, so the sub-sample is reproducible across audits. Twenty-five pages resolves a rubric whose finest distinction is 75% against 95%.

Fetching

Fetch every sampled URL twice:

• Raw — plain HTTP GET, no JavaScript, with a declared TEN7 audit user-agent.
• Rendered — a headless browser with JavaScript enabled.

The difference between those two responses is the entire evidence base for AR-3.1 and AR-3.2, which together carry 9 points. Do not skip the second fetch.

Separately, probe a small set of URLs with each AI crawler user-agent to feed AR-1.2. Identify yourself honestly in a From header. We are auditing on the client's behalf, with permission, and we do not pretend to be somebody else's bot.

The agent classes

The Index groups the ten agents three ways. AR-1.1 evaluates all ten against robots.txt. AR-1.2 probes only the eight that actually issue requests.

Google-Extended and Applebot-Extended are not crawlers. They are opt-out tokens that Google and Apple honor in robots.txt; no request ever arrives carrying either string. Blocking them is a rights decision with no fetch behavior attached, which is why they count in AR-1.1 and cannot be probed in AR-1.2. A user-agent matrix that lists them is describing a crawler that does not exist.

Where a band anchor says major agents, it means the four answer-serving agents plus GPTBot and ClaudeBot.

Politeness

Respect Crawl-delay. Default to 1 request per second with 4 concurrent workers. Support an allowlist of client-approved hosts and refuse to run against anything else. Cache every response to disk so re-scoring never re-crawls.

Reproducibility

Store the raw evidence — headers, HTML, extracted JSON-LD, screenshots — in a run directory keyed by timestamp. A score you cannot reproduce next quarter is an opinion, not a measurement.

Result schema

The scanner writes one result.json per run. Everything else (Markdown report, HTML dashboard, remediation backlog) renders from this file.

{
  "schema_version": "1.0",
  "index_version": "1.0",
  "site": {
    "name": "Example University",
    "base_url": "https://example.edu",
    "vertical": "higher-ed",
    "cms": "drupal-10"
  },
  "run": {
    "id": "2026-09-01T20-14-33Z",
    "started_at": "2026-09-01T20:14:33Z",
    "finished_at": "2026-09-01T20:41:02Z",
    "scanner_version": "0.1.0",
    "urls_sampled": 187,
    "representation_sample": 25,
    "low_confidence": false,
    "baseline_run_id": null,
    "extraction_method": "readability-density",
    "extraction_version": "1"
  },
  "score": {
    "total": 61.4,
    "ungated_score": 61.4,
    "gated": false,
    "grade": "Adequate",
    "ungated_grade": "Adequate",
    "applicable_weight": 94.5,
    "earned_points": 58.0,
    "automated_coverage": 0.87
  },
  "dimensions": [
    {
      "id": 3,
      "name": "Rendering and extraction",
      "weight": 20.0,
      "applicable_weight": 20.0,
      "earned_points": 9.5,
      "score_pct": 47.5
    }
  ],
  "checks": [
    {
      "id": "AR-3.1",
      "title": "Primary content is present without JavaScript",
      "dimension": 3,
      "weight": 6.0,
      "applicability": "always",
      "applicable": true,
      "na_reason": null,
      "verification": "auto",
      "band": 1,
      "earned_points": 1.5,
      "confidence": "high",
      "summary": "Raw HTML contains 22% of rendered text on average across 187 URLs.",
      "evidence": [
        {
          "url": "https://example.edu/admissions",
          "raw_text_chars": 812,
          "rendered_text_chars": 9440,
          "coverage": 0.086
        }
      ],
      "remediation": {
        "severity": "critical",
        "effort": "large",
        "summary": "Server-render the decoupled front end.",
        "detail": "Program and admissions templates ship an empty shell..."
      }
    }
  ],
  "remediation_backlog": [
    {
      "rank": 1,
      "check": "AR-3.1",
      "points_available": 4.5,
      "severity": "critical",
      "effort": "large"
    }
  ]
}

Schema notes

• site.cms is best-effort, from the generator meta tag and response headers. It is nullable and nothing scores against it.
• A dimension with no applicable checks reports score_pct: null, not 0. Zero means measured and failed.
• Rounding happens once, at the end. All arithmetic runs unrounded; the values written to this file are rounded to one decimal for display. The per-check earned_points therefore will not sum exactly to score.earned_points, and neither is wrong. Do not "fix" this by rounding intermediates — that reintroduces the drift the single rounding step exists to prevent.
• extraction_method and extraction_version record how main-content text was isolated. When extraction improves, old scores stay interpretable because the method that produced them is on the record.

Remediation ranking

Sort the backlog by points available (weight − earned_points), then by effort ascending, then by check ID ascending. The last key is not cosmetic: two runs of the same evidence must produce the same backlog in the same order, or the re-audit comparison is unreadable.

A check appears in the backlog only if it is applicable, has a band, and has points available above zero. N/A checks, unattested manual checks, and checks already at band 4 are not remediation items.

Use three severities — critical, major, minor — and three effort sizes — small, medium, large. Effort is a property of the check, defined in this document. Severity is derived: gate failures and any check scoring 0 or 1 with weight ≥3.0 are critical; band ≤2 with weight ≥1.5 is major; everything else is minor.

Band 1 on a heavy check is a critical finding. A site whose program pages ship 22% of their content scores AR-3.1 at band 1, not 0, and calling that major alongside a missing caption on a table understates it to the person deciding what to fund.

The client-facing output is not the score. It is the ordered list of what to fix and what each fix is worth.

The checks

Each entry gives: weight, applicability, verification mode, effort, what the check measures, how to detect it, what evidence to keep, and the band rule when it differs from the default coverage rubric.

Dimension 1 — Retrievability and rights

Weight 20.0

Can crawlers and agents reach the site, and what has the organization said they may do with what they take? This dimension comes first because nothing downstream matters if the answer to the first half is no.

AR-1.1 — No blanket robots.txt blocks on AI crawler user-agents

• Weight: 5.0 · Gate · Applicability: always · Verification: auto · Effort: small
• Measures: Whether robots.txt disallows the crawlers that feed answer engines.
• Detect: Parse robots.txt. Evaluate each of GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Bytespider against the rule set. Resolve User-agent: * fallbacks correctly, including the longest-match group rule.
• Evidence: Raw robots.txt, the parsed rule tree, and a per-agent allow/deny verdict for / and for three sampled content URLs.
• Bands: anchored on which agents are blocked, not how many.
  • 4 — every listed agent may fetch content paths.
  • 3 — one training or directive-only agent blocked; every answer-serving agent allowed.
  • 2 — two or more training or directive-only agents blocked, up to and including all of them; every answer-serving agent allowed.
  • 1 — exactly one answer-serving agent blocked.
  • 0 — Disallow: / for *, or two or more answer-serving agents blocked.
• Why the gate turns on the answer-serving class: the Index measures whether machines can find, read and use a site. An answer engine blocked at robots.txt cannot answer a question about the client, which is the failure the gate exists to catch. A training block is a rights posture, and this document declines to judge rights postures — AR-1.4 scores any combination of yes and no at 4 for exactly that reason. A university that blocks GPTBot and welcomes ChatGPT-User is reachable, and capping its score at 40 would be measuring our approval rather than machine reach.
• Consequence: blocking every training crawler costs 2 of the 5 points and does not gate. Blocking a single answer-serving agent gates.
• Absent robots.txt: a 404 scores 4. Nothing is disallowed, so nothing is blocked. This is correct and frequently queried; a missing file is not a failure of this check.
• Note: A deliberate, documented block is a business decision, not a defect. Record it, score it honestly, and say in the report that the client chose it. The number should reflect machine reach, not our approval.
• Fix: Remove inherited blocks. Keep only directives the client can explain.

AR-1.2 — CDN and WAF do not silently reject AI crawlers

• Weight: 5.0 · Gate · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether the edge returns real content to AI user-agents, regardless of what robots.txt permits.
• Detect: Request 10 sampled URLs with each of the eight fetching AI user-agents — the answer-serving and training classes defined under Fetching. Google-Extended and Applebot-Extended are excluded; they issue no requests. Record status code, response size, and whether the body matches the baseline fetch. Flag 403, 429, 503, interstitial challenge markup, and bodies more than 40% smaller than baseline.
• Evidence: A user-agent × URL matrix of status codes, sizes, and challenge detection.
• Bands: 4 — all agents get 200 with full content. 3 — one agent rate-limited but eventually served. 2 — one agent blocked or challenged. 1 — several blocked. 0 — most blocked.
• Fix: Audit Cloudflare bot settings, Pantheon AGCDN rules, Fastly VCL, and origin WAF. Verify by re-running this probe, not by reading configuration.

AR-1.3 — No interactive challenges on read-only paths

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether anonymous GETs on browse, search, listing, and detail pages return content without a challenge.
• Detect: Look for Turnstile, reCAPTCHA, hCaptcha, and JS-challenge signatures in raw responses on non-form URLs. Detect meta-refresh interstitials and challenge cookies.
• Evidence: Per-URL challenge detection with the matching signature.
• Fix: Scope challenges to POST endpoints and authenticated routes.

AR-1.4 — Content Signals declared in robots.txt

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether the site states a machine-readable position on search, ai-input, and ai-train using Cloudflare's Content Signals Policy.
• Detect: Parse Content-Signal: lines. Validate syntax (key=yes|no, comma-delimited) and check for contradiction with Disallow rules.
• Bands: 4 — all three signals declared, syntactically valid, consistent with robots.txt. 3 — declared but one signal omitted. 2 — declared with a syntax error or an internal contradiction. 1 — a partial or malformed attempt. 0 — absent.
• Note: Any combination of yes/no scores 4. We are measuring that a deliberate position exists, not which position it is.
• Fix: Run the policy conversation with the client, then add the line.

AR-1.5 — Human-readable AI policy comment in robots.txt

• Weight: 1.0 · Applicability: always · Verification: auto-partial · Effort: small
• Measures: Whether a person reading robots.txt finds a plain-language statement of the organization's position.
• Detect: Extract # comment blocks. Require ≥120 characters of prose and at least two policy terms (AI, training, crawl, license, permission).
• Bands: 4 — substantive block present and consistent with the machine signals. 2 — a comment exists but is thin or boilerplate. 0 — none.
• Fix: Write four sentences above the signals. Journalists and counsel read this file.

AR-1.6 — RSL licensing document published and referenced

• Weight: 1.0 · Applicability: conditional — the client holds licensable archives (journals, museums, associations, research bodies) or has opted into a licensing posture · Verification: auto · Effort: medium
• Measures: Whether machine-readable licensing terms exist under RSL 1.0 and are discoverable.
• Detect: Fetch /license.xml and any path referenced from robots.txt. Validate against the RSL 1.0 schema. Check for an HTTP Link: header on content responses.
• Bands: 4 — valid document, referenced from both robots.txt and headers. 3 — valid, referenced from one. 2 — present but invalid. 1 — referenced but missing. 0 — absent.
• Fix: Decide terms with the client first. The XML is the easy half.

AR-1.7 — RSL terms propagated to feeds and schema

• Weight: 1.0 · Applicability: conditional — requires AR-1.6 to pass · Verification: auto · Effort: small
• Measures: Whether licensing travels with the content through every channel, not just the well-known file.
• Detect: Check RSS and Atom feeds for licensing elements. Check sampled JSON-LD for a license property on licensable types.
• Fix: Add the license reference to feed generation and to the schema field mapping.

AR-1.8 — Web Bot Auth verification configured

• Weight: 1.0 · Applicability: conditional — the client wants selective agent access rather than open or closed · Verification: auto-partial · Effort: large
• Measures: Whether the edge distinguishes cryptographically verified agents from unsigned scrapers.
• Detect: Probe /.well-known/http-message-signatures-directory if the site operates its own agents. Otherwise send a signed and an unsigned request and compare treatment.
• Bands: 4 — verified agents pass, unsigned agents claiming an AI UA are challenged. 2 — signature headers accepted but not acted on. 0 — no differentiation.
• Fix: Configure edge bot rules on Signature-Agent rather than user-agent strings.

AR-1.9 — Correct X-Robots-Tag on non-HTML resources

• Weight: 1.0 · Applicability: conditional — the site serves PDFs, documents, media, or JSON endpoints · Verification: auto · Effort: small
• Measures: Whether indexing directives reach resources that cannot carry a meta robots tag.
• Detect: HEAD every sampled PDF, document, media file, and API endpoint. Record X-Robots-Tag. Flag noindex on substantive content and absent directives on endpoints that should be excluded.
• Bands: 4 — every non-HTML type carries a deliberate, correct value. 2 — mixed or partial. 0 — absent everywhere, or substantive PDFs marked noindex.
• Fix: Set headers by path pattern at the CDN or web server.

AR-1.10 — No stray noarchive or nosnippet directives

• Weight: 1.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether the site suppresses the excerpting that answer engines depend on.
• Detect: Parse meta robots tags and X-Robots-Tag headers across the sample for noarchive, nosnippet, and max-snippet:0.
• Bands: 4 — none present. 2 — present on a minority of pages. 0 — present site-wide.
• Fix: Remove them. They are almost always left over from a legacy SEO module.

Dimension 2 — Crawl and index hygiene

Weight 12.0

AI crawlers visit less often than Googlebot and tolerate less noise. Budget wasted on faceted search never reaches the pages that answer questions.

AR-2.1 — Missing pages return 404 or 410

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Soft 404s — "not found" content served with a 200 status.
• Detect: Request 10 deliberately invalid URLs under real path prefixes. Flag 200 responses. Separately, flag sampled pages whose body matches a not-found template or falls below a minimum content threshold while returning 200.
• Evidence: Probe URLs with status codes and body fingerprints.
• Fix: Return real status codes. Use 410 for content that is permanently gone.

AR-2.2 — Redirects resolve in a single hop

• Weight: 1.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Redirect chain depth and loops.
• Detect: Follow redirects without auto-resolution and count hops per sampled URL and per internal link target.
• Bands: 4 — no chain exceeds one hop. 3 — a few two-hop chains. 2 — chains of three or more. 1 — widespread chains, meaning both more than 10% of the sample and at least three URLs. 0 — loops present.
• Fix: Flatten the redirect table so every source points at its final destination.

AR-2.3 — Every page emits a valid, self-consistent canonical

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Presence and correctness of rel=canonical.
• Detect: For each sampled page, extract the canonical and resolve it. Flag: missing, multiple, cross-host, pointing at a redirect, pointing at a non-200, or pointing at the homepage from a deep page.
• Fix: Emit a self-referencing canonical by default and correct the exceptions.

AR-2.4 — Faceted, calendar, and parameterized URLs are controlled

• Weight: 2.0 · Applicability: conditional — the site has search, filtering, or a calendar · Verification: auto-partial · Effort: medium
• Measures: Whether the crawlable URL space is finite and predictable.
• Detect: Crawl listing pages two levels deep with parameters followed. Measure URL growth rate and count distinct parameter combinations. Check whether those URLs are disallowed, noindex, or canonicalized to the unfiltered listing.
• Bands: 4 — parameter space bounded by robots rules, noindex, or canonicals. 3 — mostly controlled with gaps. 2 — partially controlled. 1 — minimal control. 0 — unbounded expansion observed.
• Fix: Combine disallow patterns, noindex on parameterized views, and canonicals back to the base listing.

AR-2.5 — Sitemap lastmod reflects real content changes

• Weight: 2.0 · Applicability: always · Verification: auto-partial · Effort: medium
• Measures: Whether lastmod is a freshness signal or deploy noise.
• Detect: Parse all lastmod values. Flag clustering — more than 60% of URLs sharing a single date, or timestamps identical to the minute across content types. Cross-check against on-page dateModified where present. Re-run across two audits to confirm.
• Bands: 4 — values are distributed and match on-page dates. 3 — mostly accurate with some clustering. 2 — heavy clustering. 1 — all identical. 0 — absent.
• First run: cap at 3 without a baseline run. See Checks that need a previous run.
• Fix: Wire lastmod to the node's changed timestamp, not to cron.

AR-2.6 — Sitemap index split by type, with media sitemaps

• Weight: 1.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether sitemap structure makes crawl coverage diagnosable.
• Detect: Fetch /sitemap.xml. Confirm it is an index with child sitemaps. Check for image and video sitemaps where the site has substantial media. Validate against the sitemap schema and the 50k URL / 50MB limits.
• Fix: Generate per-content-type children and submit the index to both search consoles.

AR-2.7 — Paginated series are coherently signaled

• Weight: 1.0 · Applicability: conditional — the site has paginated listings or multi-page articles · Verification: auto · Effort: small
• Measures: Whether page 2 and beyond are reachable and correctly related to page 1.
• Detect: Identify paginated series by URL pattern. Check for rel=next/rel=prev, a view-all page carrying the canonical, and whether deep pages are reachable by crawl.
• Fix: Prefer a view-all canonical where page weight allows.

AR-2.8 — IndexNow fires on publish and update

• Weight: 1.0 · Applicability: always · Verification: auto-partial · Effort: small
• Measures: Whether the site pushes change notifications instead of waiting to be crawled.
• Detect: Look for an IndexNow key file at the site root (a 32–128 character hex filename returning its own key). Confirm the key resolves. Whether submissions actually fire on publish requires attestation.
• Bands: 4 — key file present, valid, and submission confirmed. 3 — key file present and valid, submission unconfirmed. 2 — a key file is referenced but does not resolve or does not contain its own key. 0 — absent or not discoverable.
• Limitation: the key file's name is the key, and the key is not published. A scanner can only find it where robots.txt or a conventional path advertises it, so a zero here means not discoverable rather than absent. Score it honestly and say which.
• Fix: Hook submission into the publish workflow rather than a schedule.

Dimension 3 — Rendering and extraction

Weight 20.0

Most AI crawlers do not execute JavaScript. This dimension asks whether the facts exist in the server response, in a structure a parser can follow.

AR-3.1 — Primary content is present without JavaScript

• Weight: 6.0 · Applicability: always · Verification: auto · Effort: large
• Measures: The gap between the raw HTTP response and the hydrated DOM. This is the single heaviest check in the Index.
• Detect: For every sampled URL, extract main-content text from the raw fetch and from the rendered fetch. Compute coverage = raw_chars ÷ rendered_chars after normalizing whitespace and stripping nav, header, and footer regions. Average across the sample and report the per-template distribution.
• Evidence: Per-URL character counts, coverage ratio, and the first 500 characters of each version for spot-checking.
• Bands: 4 — mean coverage ≥0.95. 3 — 0.80–0.94. 2 — 0.50–0.79. 1 — 0.20–0.49. 0 — below 0.20.
• Edge cases: Coverage above 1.0 happens when JavaScript removes content that the raw response contained. Clamp the per-URL value to 1.0 for banding and keep the unclamped ratio in the evidence. A URL whose rendered text is empty is excluded from the mean and the exclusion is recorded — it is a fetch problem, not a coverage measurement.
• Note: Report the worst-performing template by name. "Your program pages ship 8% of their content" lands harder than a mean.
• Fix: Server-render or statically render. For decoupled Drupal this is the largest line item in most remediations.

AR-3.2 — Progressive disclosure content ships in the initial HTML

• Weight: 3.0 · Applicability: conditional — the site uses tabs, accordions, load-more, or infinite scroll · Verification: auto · Effort: medium
• Measures: Whether tabbed, collapsed, or lazily paged content exists before interaction.
• Detect: Identify disclosure widgets by ARIA roles (tab, tabpanel, region with aria-expanded), <details>, and common class patterns. For each, check whether the panel body is populated in the raw HTML. Detect infinite scroll by watching for XHR content growth on scroll in the rendered fetch, and check for a paginated fallback.
• Bands: 4 — every panel populated in raw HTML; infinite scroll has crawlable pagination. 3 — one pattern incomplete. 2 — panels populated on some templates only. 1 — most content loads on interaction. 0 — all disclosure content is fetched on demand.
• Fix: Render all panels and hide with CSS. Give infinite scroll a paginated fallback.

AR-3.3 — Semantic HTML structure

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether document structure carries meaning or only styling.
• Detect: Per page, require exactly one <main>, at least one <article> or <section> wrapping the primary content, and a <nav> for primary navigation. Compute a div-to-semantic-element ratio in the content region and flag pages above 12:1.
• Fix: Replace structural divs with article, section, nav, aside, figure, and dl.

AR-3.4 — One h1 and a non-skipping heading hierarchy

• Weight: 3.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether headings segment the page into topics a model can follow.
• Detect: Count h1 elements. Walk the heading tree and flag skipped levels, empty headings, and headings used purely for styling (no following content).
• Bands: Default coverage rubric, where a page passes only if it has exactly one non-empty h1 and no skipped levels.
• Fix: Move the site name out of h1. Fix skips in the templates, not the content.

AR-3.5 — Real data tables with header cells

• Weight: 2.0 · Applicability: conditional — the sample contains tables · Verification: auto · Effort: medium
• Measures: Whether tabular facts — tuition, hours, comparisons, fees — are machine-parseable.
• Detect: For each <table>, require <th> with scope, a <caption>, and <thead>/<tbody>. Separately flag layout tables (no th, presentational attributes, single row or column).
• Bands: 4 — every data table complete, no layout tables. 3 — minor omissions such as missing captions. 2 — header cells present without scope. 1 — tables present with no header cells. 0 — layout tables carrying data.
• Fix: Fix the tables that carry facts first. Convert layout tables to CSS.

AR-3.6 — Stable, human-readable anchor IDs on section headings

• Weight: 2.0 · Applicability: always · Verification: auto-partial · Effort: medium
• Measures: Fragment-level citability — whether a model can deep-link a passage rather than a page.
• Detect: For headings below h1, check for an id. Classify each as slug-like (derived from heading text) or unstable (section-3, block-a7f2c1, framework hashes). Compare IDs across two runs to detect churn.
• Bands: 4 — ≥95% of subheadings carry slug-like IDs that are stable across runs. 3 — 75–94%. 2 — IDs present but a majority are generated or unstable. 1 — sparse. 0 — none.
• First run: cap at 3 without a baseline run. Stability cannot be asserted from one observation. See Checks that need a previous run.
• Fix: Derive IDs from heading text, not render order.

AR-3.7 — Alt text on content images, empty alt on decorative

• Weight: 1.0 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether images have a textual representation.
• Detect: For every <img>, record presence of alt, its length, and whether the image sits inside a content region or a decorative one. Flag missing alt, filename-as-alt, and long alt on obviously decorative images.
• Fix: Audit existing content, then make the field required in the media entity so the problem stops recurring.

AR-3.8 — Key facts exist as HTML text, not only in images or PDFs

• Weight: 1.0 · Applicability: always · Verification: auto-partial · Effort: large
• Measures: Whether high-value facts are trapped in a scan, an infographic, or a PDF.
• Detect: Identify pages whose visible text is thin relative to their images or that link to a PDF as the primary content. Extract PDF text and look for numbers, dates, and prices that appear nowhere in the site's HTML. Requires review before scoring.
• Bands: 4 — no high-value facts found only in binaries. 2 — some found only in PDFs. 0 — core facts (tuition, hours, deadlines, contact) exist only in binaries.
• Fix: Publish an HTML equivalent. Keep the PDF as a download, not as the source of truth.

Dimension 4 — Alternate representations

Weight 8.0

Clean, low-noise versions of the content alongside the HTML.

Read this before you sell it: evidence that llms.txt is actually consumed by the major models remains thin, and Google has said it does not use it. We include the dimension because it is cheap, it forces good content hygiene, and being early costs nothing. We weight it at 8 points, not 20, for the same reason. Do not let a report imply this is a ranking lever.

AR-4.1 — /llms.txt exists and is generated from the content model

• Weight: 3.0 · Applicability: always · Verification: auto-partial · Effort: medium
• Measures: Presence, validity, and freshness of /llms.txt.
• Detect: Fetch /llms.txt. Validate against the llmstxt.org structure: an # H1 title, an optional blockquote summary, and ## sections of Markdown links. Resolve a sample of the linked URLs. Compare listed URLs against the sitemap to estimate coverage and staleness. Detect the Drupal llms_txt module by response headers or known output patterns.
• Bands: 4 — valid, links resolve, coverage tracks the sitemap, generation confirmed. 3 — valid and current but hand-maintained. 2 — present with broken links or clear staleness. 1 — present but malformed. 0 — absent.
• Fix: On Drupal, use the llms_txt module with token-driven menus. A hand-written file is stale within a quarter.

AR-4.2 — /llms-full.txt where full-text inclusion is appropriate

• Weight: 1.0 · Applicability: conditional — the client permits full-text inclusion; N/A if Content Signals declare ai-train=no · Verification: auto · Effort: small
• Measures: Presence of the expanded variant.
• Detect: Fetch /llms-full.txt. Confirm it inlines content rather than linking. Cross-check against AR-1.4 for contradiction.
• Bands: 4 — present and consistent with the licensing posture. 2 — present but contradicts declared signals. 0 — absent where appropriate.
• Fix: Gate this on the licensing decision, not on convenience.

AR-4.3 — Markdown companion for every canonical page

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether a clean .md representation exists alongside each page.
• Detect: For each sampled content URL, request {path}.md. Require a 200 with text/markdown or text/plain, content that materially matches the page body, and no navigation chrome.
• Fix: Generate from the same source as the HTML. Strip nav, promos, and cookie banners.

AR-4.4 — link rel=alternate advertises the Markdown version

• Weight: 1.0 · Applicability: conditional — requires AR-4.3 to pass · Verification: auto · Effort: small
• Measures: Discoverability of the alternate representation.
• Detect: Parse <link rel="alternate" type="text/markdown"> from the head and resolve the href.
• Fix: Add it to the head template. A file nobody can find is not much use.

AR-4.5 — Accept: text/markdown content negotiation

• Weight: 0.5 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether the canonical URL serves Markdown on request, with correct cache headers.
• Detect: Request sampled URLs with Accept: text/markdown. Confirm the response type and that Vary: Accept is set. Re-request through the CDN to confirm the variants are cached separately.
• Bands: 4 — negotiation works and Vary: Accept is correct. 2 — negotiation works but Vary is missing, which risks serving Markdown to browsers. 0 — not supported.
• Fix: Getting Vary wrong is worse than not doing this at all. Verify at the edge.

AR-4.6 — Alternate representations are edge-cached

• Weight: 0.5 · Applicability: conditional — requires AR-4.1 or AR-4.3 · Verification: auto · Effort: small
• Measures: Whether generated representations hit the CDN rather than origin.
• Detect: Request /llms.txt and sampled .md URLs twice. Read cache-status headers and compare response times.
• Fix: Set TTLs and wire invalidation to the same cache tags as the source content.

Dimension 5 — Structured data and the entity graph

Weight 22.0

The heaviest dimension. An Organization block and some FAQ markup is commodity work. A linked, externally anchored entity graph is not.

AR-5.1 — JSON-LD is generated from mapped fields, not hardcoded

• Weight: 2.0 · Applicability: always · Verification: auto-partial · Effort: medium
• Measures: Whether schema follows the content or rots on the next content type change.
• Detect: Heuristics only. Compare JSON-LD values against visible field values on the same page — a mapped implementation matches. Flag identical literal values repeated across pages of the same type where the visible content differs. Detect Schema.org Metatag output patterns on Drupal. Confirm with the build team.
• Bands: 4 — values track content across every sampled page of a type, and the build team confirms the mapping. 3 — values track content on every sampled page, generation unconfirmed. 2 — mixed, with some literals. 0 — clear hardcoding, or values contradict the visible page.
• Fix: Map Drupal fields to schema properties through configuration so editors maintain it.

AR-5.2 — A single linked entity graph with @id references

• Weight: 4.0 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether schema forms one coherent graph or dozens of disconnected assertions.
• Detect: Extract all JSON-LD across the sample. Build a graph of nodes and @id references. Measure: what fraction of entity nodes carry an @id — value objects such as ListItem, PostalAddress, ContactPoint, EntryPoint and BreadcrumbList are excluded, because they have no identity and nothing should ever reference them; whether Organization and WebSite have stable canonical @id URIs reused site-wide; whether WebPage nodes reference WebSite and WebSite references Organization; and how many duplicate definitions of the same entity exist.
• Evidence: The resolved graph, orphan node count, duplicate entity count.
• Bands: 4 — canonical @ids reused, chain intact, no duplicates. 3 — chain intact with minor duplication. 2 — @ids present but inconsistently reused. 1 — @ids rare. 0 — isolated blobs with no references.
• Fix: Define canonical @id URIs once and reference them everywhere.

AR-5.3 — sameAs links to external authority records

• Weight: 3.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Entity disambiguation — whether a model can be certain which organization this is.
• Detect: Extract sameAs from Organization and Person nodes. Classify each URL by authority: tier 1 (Wikidata, ROR, NPI, IRS EIN, ORCID), tier 2 (Wikipedia, Crunchbase, LinkedIn, Candid), tier 3 (social profiles). Resolve each URL and confirm it returns 200 and references the organization back where possible.
• Resolution is opt-in. Those hosts are not the client's, they are not on the allowlist, and requesting them tells a third party which site is being audited. Classification from the markup alone separates a controlled identifier from a Facebook page; without resolution the check stops at band 3 and says why.
• Bands: 4 — at least one tier-1 identifier plus two others, all resolving. 3 — tier-1 present, some links stale. 2 — tier-2 and tier-3 only. 1 — social profiles only. 0 — no sameAs.
• Fix: Claim the Wikidata item. For universities, ROR is free and takes a day.

AR-5.4 — BreadcrumbList on every page below the homepage

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether a page declares where it sits in the information architecture.
• Detect: Validate BreadcrumbList structure and confirm the trail matches the URL path or menu position. Flag trails that are a single item or that contradict the path.
• Fix: Generate from the menu, not by hand.

AR-5.5 — Vertical-specific schema types

• Weight: 4.0 · Applicability: vertical — scored against the client's declared vertical only · Verification: auto · Effort: large
• Measures: Whether the site models the things it actually contains, using the right types.
• Detect: From the declared vertical, load the expected type set and check coverage against matching content types:
  • Higher ed: Course, EducationalOccupationalProgram, CollegeOrUniversity, EducationEvent
  • Healthcare: MedicalOrganization, Physician, MedicalCondition, MedicalWebPage
  • Nonprofit: NGO, DonateAction, Grant, FundingScheme
  • Government: GovernmentOrganization, GovernmentService, Legislation
  • Research: Dataset, DataCatalog, ScholarlyArticle
• Bands: Fraction of expected types present and valid on the content that warrants them: 4 — ≥90%. 3 — 70–89%. 2 — 40–69%. 1 — 1–39%. 0 — none.
• Note: Dataset and DataCatalog are badly underused. Research organizations routinely hold catalogd data with no machine-readable description at all.
• Fix: Pick the applicable set deliberately. Do not implement types the content does not support.

AR-5.6 — FAQPage and QAPage on genuine question-and-answer content

• Weight: 1.0 · Applicability: conditional — the site has real Q&A content · Verification: auto · Effort: small
• Measures: Correct markup on genuine Q&A — and the absence of fabricated FAQ blocks.
• Detect: Validate FAQPage and QAPage structure. Cross-check that every marked-up question appears in the visible page text. Flag pages where FAQ markup exists with no corresponding visible content.
• Bands: 4 — real Q&A marked up, nothing fabricated. 2 — partial coverage. 0 — absent where warranted. Cap at 1 if fabricated FAQ blocks are detected, regardless of coverage.
• Fix: Mark up what exists. Do not invent questions to earn the markup — it is a spam pattern and it degrades the page for people.

AR-5.7 — HowTo on procedural content

• Weight: 1.0 · Applicability: conditional — the site has procedural content · Verification: auto · Effort: small
• Measures: Whether step sequences are explicit rather than inferred from prose.
• Detect: Identify procedural pages by ordered lists with step-like language in headings. Check for HowTo with ordered HowToStep.
• Fix: Application walkthroughs, enrollment steps, and permit processes are exactly what people ask assistants about.

AR-5.8 — speakable on summaries and ledes

• Weight: 1.0 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether the page flags its own best short answer.
• Detect: Extract speakable with cssSelector or xpath. Resolve the selector against the DOM and confirm it matches 1–3 elements containing 40–400 characters. Flag selectors that match the whole body or nothing.
• Bands: 4 — resolves to a genuine summary passage. 2 — present but resolves too broadly. 0 — absent.
• Fix: Point it at the lede, not the article.

AR-5.9 — isAccessibleForFree, about, and mentions with entity references

• Weight: 1.0 · Applicability: always · Verification: auto · Effort: medium
• Measures: Access status and topical entity linkage.
• Detect: Check for isAccessibleForFree on substantive content. Check about and mentions, and classify their values as @id references or bare strings.
• Bands: 4 — access declared and subject entities referenced by identifier. 2 — properties present with string values only. 0 — absent.
• Fix: Entity references are how a model decides a page is about a subject rather than merely containing the word.

AR-5.10 — Product, Offer, and Service with real prices

• Weight: 1.0 · Applicability: conditional — the site sells, charges, or offers something · Verification: auto · Effort: medium
• Measures: Whether commercial facts are marked up and current.
• Detect: Validate Offer structure. Compare price, priceCurrency, and availability against the visible page. Flag any mismatch and any priceValidUntil in the past.
• Bands: 4 — accurate and matching the page. 2 — present but drifting from visible values. 0 — absent, or demonstrably stale.
• Note: Stale pricing in schema is worse than none. A model will quote it with confidence.
• Fix: Source the schema value from the same field the page renders.

AR-5.11 — SearchAction on the WebSite node

• Weight: 1.0 · Applicability: conditional — the site has search · Verification: auto · Effort: small
• Measures: Whether site search is exposed as a callable pattern.
• Detect: Extract potentialAction / SearchAction from the WebSite node. Execute the URL template with a test term and confirm real results come back.
• Bands: 4 — declared and the template returns results. 2 — declared but the template fails. 0 — absent.
• Fix: Small piece of the agent-interface story, and it costs almost nothing.

AR-5.12 — Schema validation runs in CI

• Weight: 1.0 · Applicability: always · Verification: manual · Effort: medium
• Measures: Whether broken markup fails the build instead of reaching production.
• Detect: Not observable from outside. Attested by the build team, ideally with a link to the pipeline definition.
• Bands: 4 — validation gates merges on all covered content types. 2 — validation runs but does not block. 0 — none.
• Fix: This is the check that separates an engineering firm from an SEO boutique. Schema rots silently otherwise.

Dimension 6 — Authorship, provenance, freshness

Weight 10.0

Trust signals, expressed as structured data rather than as vibes.

AR-6.1 — Author entity pages with credentials

• Weight: 2.0 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether authors resolve to real, credentialed entities.
• Detect: Follow author references to their target pages. Require Person markup with a canonical @id, plus at least two of jobTitle, affiliation, knowsAbout, sameAs. Resolve sameAs (ORCID, LinkedIn, institutional profile).
• Bands: 4 — every recurring author has a resolvable entity page with verified identifiers. 3 — most do. 2 — author pages exist without markup or credentials. 1 — author names only. 0 — no author entities.
• Fix: A byline string asserts nothing. A resolvable entity with credentials does.

AR-6.2 — Bylines linked to author entities

• Weight: 1.5 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether substantive pages are signed, and whether the schema author is an @id reference rather than a repeated string.
• Detect: For each substantive page, check for a visible byline and an author property. Classify the value as @id reference or literal.
• Bands: 4 — visible byline plus @id reference on ≥95%. 3 — 75–94%. 2 — author present as a bare string. 1 — sparse. 0 — unsigned.
• Fix: Where the author is a department, name the department and model it as an entity.

AR-6.3 — reviewedBy on YMYL content

• Weight: 1.0 · Applicability: conditional — the site publishes medical, legal, or financial content · Verification: auto · Effort: medium
• Measures: Whether expert review reaches the markup.
• Detect: On YMYL pages, check for reviewedBy referencing a credentialed Person, plus lastReviewed where the type supports it, plus a visible reviewer statement.
• Fix: Most healthcare and legal clients already run review workflows. The reviewer's identity simply never reaches the page.

AR-6.4 — datePublished and dateModified are accurate

• Weight: 2.5 · Applicability: always · Verification: auto-partial · Effort: medium
• Measures: Whether the freshness signal is honest.
• Detect: Extract both dates from schema and visible markup. Flag: missing dates, dateModified before datePublished, future dates, and clustering that suggests a deploy stamp. Compare against sitemap lastmod (AR-2.5) and against the previous audit run to confirm dates only move when content moves.
• Bands: 4 — dates present, internally consistent, and stable across a deploy with no content change. 3 — present and consistent, stability unconfirmed. 2 — present but clustered or contradicting the sitemap. 1 — one date only. 0 — absent.
• Fix: Wire both to the node's own timestamps. A deploy must not move them.

AR-6.5 — citation markup on primary-source references

• Weight: 0.5 · Applicability: conditional — the site publishes research or policy content · Verification: auto · Effort: medium
• Measures: Whether sourcing is machine-readable.
• Detect: Check for citation on research and policy pages. Resolve DOIs where present. Compare against the count of outbound links to primary sources in the body.
• Fix: Credibility that a parser cannot see does not count.

AR-6.6 — C2PA Content Credentials on original assets

• Weight: 0.5 · Applicability: conditional — the client produces original photography, primary research, or documents where authenticity matters · Verification: auto · Effort: large
• Measures: Whether provenance survives into the delivered file.
• Detect: Download sampled original images and PDFs and read C2PA manifests. Confirm the manifest survives the image derivative pipeline — this is where it usually breaks.
• Bands: 4 — valid manifests on originals and derivatives. 2 — present on originals, stripped by the pipeline. 0 — absent.
• Fix: Scope this deliberately. It needs signing infrastructure.

AR-6.7 — Editorial policy page referenced via publishingPrinciples

• Weight: 0.5 · Applicability: always · Verification: auto · Effort: small
• Measures: Whether the site documents how content is produced, reviewed, corrected, and funded.
• Detect: Look for publishingPrinciples on the Organization node and resolve it. For publishers, also check correctionsPolicy and diversityPolicy.
• Fix: Write the page that describes the process the client already follows.

AR-6.8 — About, Contact, and organizational detail are complete

• Weight: 1.5 · Applicability: always · Verification: auto · Effort: medium
• Measures: Whether a model can resolve who this organization actually is.
• Detect: Locate About and Contact pages. Score presence of: founding date, leadership names, postal address, service area, contactPoint entries differentiated by purpose, and public legal identifiers (EIN, ROR, DUNS). Check both the visible page and the Organization node.
• Bands: Fraction of those seven facts present in both HTML and schema: 4 — ≥6. 3 — 5. 2 — 3–4. 1 — 1–2. 0 — none.
• Fix: Thin About pages are a persistent weakness for nonprofits and institutions. Do not leave the key facts in an annual-report PDF.

Dimension 7 — Agent interfaces

Weight 5.0

The move from "a site agents can read" to "a site agents can use." Low weight, high differentiation — most sites score zero here, and that is currently fine. Treat this dimension as an opportunity map rather than a deficiency list.

AR-7.1 — NLWeb endpoint over the site's schema.org data

• Weight: 0.5 · Applicability: always · Verification: auto · Effort: large
• Detect: Probe known NLWeb paths and /.well-known/ entries. If found, send a representative question and check for a schema.org-formatted JSON response.
• Bands: 4 — endpoint answers correctly and is grounded in site content. 2 — endpoint responds but answers poorly. 0 — absent.

AR-7.2 — NLWeb exposed as an MCP server with an ask method

• Weight: 0.5 · Applicability: conditional — requires AR-7.1 · Verification: auto · Effort: medium
• Detect: Attempt an MCP handshake. Confirm the ask method is discoverable and returns correct answers.

AR-7.3 — Domain MCP server over real content APIs

• Weight: 1.0 · Applicability: always · Verification: auto-partial · Effort: large
• Measures: Whether the client's structured data — course catalog, provider directory, grant database, program finder — is callable as documented tools.
• Detect: Discover from /.well-known/. Enumerate tools, check each has a description and typed parameters, and call one read-only tool against live data.
• Bands: 4 — documented, typed tools returning live data, with auth and rate limiting. 3 — tools present but thinly documented. 2 — a generic query endpoint rather than narrow tools. 0 — none.
• Note: The most defensible item in the whole Index. A well-described tool with three parameters beats a generic endpoint.

AR-7.4 — OpenAPI specification for public APIs

• Weight: 0.25 · Applicability: conditional — the site has a public or semi-public API · Verification: auto · Effort: small
• Detect: Look for the spec at conventional paths and from /.well-known/. Validate it, and confirm it is versioned and linked from the site.

AR-7.5 — /.well-known/ discovery entries for agent capabilities

• Weight: 0.25 · Applicability: conditional — the site exposes any agent capability · Verification: auto · Effort: small
• Detect: Enumerate /.well-known/ for MCP, NLWeb, OpenAPI, and http-message-signatures-directory. Validate each returns parseable metadata.

AR-7.6 — Form fields carry label, name, and autocomplete

• Weight: 1.0 · Applicability: conditional — the site has public forms · Verification: auto · Effort: medium
• Measures: Whether an agent can tell what each field is for.
• Detect: For every public form field: a <label for> or equivalent accessible name, a meaningful name attribute (reject field_1, input3), and a valid autocomplete token where one applies (given-name, email, postal-code, tel). A token applies where the field type implies a purpose — email, tel, url, password. A search box or a free-text field has no sensible token and is not marked down for lacking one.
• Bands: Fraction of fields passing all three: 4 — ≥95%. 3 — 75–94%. 2 — 40–74%. 1 — 1–39%. 0 — none.
• Fix: Same change satisfies WCAG. Easy joint justification.

AR-7.7 — No captchas on browse, search, or filter interactions

• Weight: 0.5 · Applicability: always · Verification: auto · Effort: small
• Detect: Submit search and filter forms and check for challenge responses. Complements AR-1.3, from the application side rather than the edge.

AR-7.8 — Critical flows complete without JS-only interactions

• Weight: 0.5 · Applicability: conditional — the client names critical flows · Verification: auto-partial · Effort: large
• Measures: Whether register, donate, apply, or find-a-clinician can be completed through standard form submissions and real URL transitions.
• Detect: For each named flow, walk it with JavaScript disabled. Flag div-based controls with click handlers, wizards with no addressable step URLs, and modal-only paths.
• Bands: 4 — every flow completes with real URLs at each step. 2 — flows start but cannot complete. 0 — flows are JS-only.
• Fix: Run this as a joint engineering and UX audit.

AR-7.9 — Stable selectors on critical-flow elements

• Weight: 0.5 · Applicability: conditional — requires AR-7.8 flows to be named · Verification: auto · Effort: medium
• Detect: Collect selectors for the key elements in each flow. Classify as stable (id, data-*, semantic) or fragile (hashed utility classes, build-generated IDs). Diff across two runs to detect churn.
• Bands: 4 — stable and unchanged across runs. 2 — mixed. 0 — hashed classes only.
• First run: cap at 2 without a baseline run, since band 4 is defined entirely by cross-run stability. See Checks that need a previous run.
• Fix: Treat these selectors as a contract that survives redeploys.

Dimension 8 — Measurement and instrumentation

Weight 3.0

The proof layer. Commercially, this is the most valuable work in the whole checklist — and almost none of it is visible from outside a site, which is why it carries three points. Do not read a low score here as unimportant. Read it as unmeasurable by scanner.

AR-8.1 — GA4 channel group for AI assistant referrers

• Weight: 0.5 · Applicability: always · Verification: manual · Effort: small
• Measures: Whether the client can see traffic from chatgpt.com, perplexity.ai, claude.ai, copilot.microsoft.com, and gemini.google.com. By default it lands in Direct and disappears.
• Detect: Analytics presence is detectable from the page. The channel group is not — attest it, ideally with a screenshot.
• Bands: 4 — group live with a saved report. 2 — analytics present, no grouping. 0 — no analytics.

AR-8.2 — Server-side tagging captures stripped referrers

• Weight: 0.5 · Applicability: always · Verification: manual · Effort: medium
• Detect: A server-side endpoint may be inferable from network requests. Reconciliation against client-side numbers must be attested.

AR-8.3 — Access log retention with a queryable store

• Weight: 0.5 · Applicability: always · Verification: manual · Effort: medium
• Measures: Whether crawler logs survive long enough to show a before-and-after window. Many hosts discard them in days.
• Bands: 4 — ≥12 months retained and queryable by user-agent, path, status, date. 2 — retained but not queryable. 0 — discarded.

AR-8.4 — Scheduled AI crawler activity report

• Weight: 0.5 · Applicability: conditional — requires AR-8.3 · Verification: manual · Effort: medium
• Measures: Whether crawler behavior is monitored continuously rather than audited once. A crawler that stops appearing usually means somebody reintroduced a block.

AR-8.5 — Fixed prompt panel run monthly across models

• Weight: 0.5 · Applicability: always · Verification: manual · Effort: medium
• Measures: Whether answer-share is tracked on a stable panel of brand, category, and comparison prompts.
• Bands: 4 — fixed panel, baseline stored, monthly runs comparable. 2 — ad-hoc checks. 0 — none.
• Note: The panel has to stay fixed. Tuning prompts between runs destroys the trend line.

AR-8.6 — Extraction-fidelity baseline captured

• Weight: 0.25 · Applicability: always · Verification: manual · Effort: medium
• Measures: Whether a before-and-after fidelity score exists — feed the page to a model, ask it the client's key questions, score the accuracy.
• Note: The scanner can generate this itself in a later version. Until then, attest it. The before-and-after delta is the best single-slide value demonstration we have.

AR-8.7 — Search Console and Bing Webmaster Tools verified

• Weight: 0.25 · Applicability: always · Verification: auto-partial · Effort: small
• Detect: Look for google-site-verification and msvalidate.01 meta tags, or DNS TXT verification records. Whether anyone reads the alerts must be attested.
• Bands: 4 — both verified, sitemaps submitted, alerts routed to a person. 3 — both verified; submission and alert routing unconfirmed. 2 — one verified. 0 — neither.
• Note: Bing matters more than its search share suggests. Its index feeds several AI products, and clients frequently have Search Console with no Bing property at all.

What 100 means

A site scoring 100 has, at minimum:

• Every AI crawler reaching real content at the edge and in robots.txt, with a deliberate, documented licensing position.
• A clean, finite URL space with honest freshness signals.
• Every substantive fact present in the server response, in semantic markup, with stable anchors a model can cite.
• Generated llms.txt and Markdown companions, cached at the edge.
• One linked entity graph, anchored to external authorities, validated in CI so it cannot rot.
• Credentialed authors, honest timestamps, machine-readable sourcing.
• Documented agent endpoints and flows an agent can actually complete.
• Instrumentation that proves all of it, running on a schedule.

Almost no site will score 100, and that is the point. The Index is a target to move toward, not a bar to clear. A client who moves from 41 to 72 in one engagement has a number they can take to their board — which is worth considerably more to them than a perfect score they will never reach.

What the Index deliberately does not measure

We keep this list public because leaving it out would be dishonest.

• Whether the content is any good. Structure, not substance.
• Whether the models actually cite the site. That is answer-share monitoring (AR-8.5), a different instrument on a different cadence.
• Traditional SEO performance. Rankings, backlinks, and Core Web Vitals are real, and they are not this.
• Accessibility conformance. Several checks overlap with WCAG, happily. This is not a WCAG audit and must not be sold as one.
• Security posture. We probe bot handling at the edge. We do not test the site's defenses.

Versioning and change control

The Index version is independent of the scanner version. Both appear in every result file.

Bump the minor version for new checks, clarified detection, or corrected band anchors. Bump the major version for any weight change, gate change, or scoring-model change — anything that makes two scores non-comparable.

Never silently re-weight. A client's trend line is the most valuable thing the Index produces, and one quiet weight change destroys it. When a major bump lands, re-score the last run under both versions and show the difference.

Changelog

2.0 — 2026-09-01

A major bump, and the versioning rule applies in full. The one-time exemption taken at 1.1 is spent. Any site scored under 1.x must be re-scored under 2.0 before its number is compared to anything.

1. AR-1.1's bands are anchored on agent class rather than agent count, and the gate now turns on the answer-serving class. Blocking every training crawler costs 2 of 5 points and does not gate; blocking one answer-serving agent does. Under 1.x the table could not tell PerplexityBot from CCBot — both scored band 3 — which meant a site invisible to an answer engine and a site excluded from a corpus were graded identically. This changes which sites are gated, which is why the version is 2.0.
2. critical severity widens to band 0 or 1 on checks weighing 3.0 or more. A 6-point check at band 1 is not a major finding sitting alongside a missing table caption. Ordering and points are unchanged; only the label moves.
3. AR-2.8, AR-5.1 and AR-8.7 gain a band 3, and their unattested ceilings rise from 2 to 3 to match. Their tables previously skipped 3, so with a ceiling at 2 the whole observable range was {0, 2} and there was no way to say mostly right.

1.2 — 2026-09-01

Refinements found while implementing all 68 checks. None changes a weight or a gate. Two change how a band is reached and are recorded here so a re-score is explicable.

1. @id coverage in AR-5.2 counts entity nodes only. A ListItem inside a BreadcrumbList, a PostalAddress on an Organization and a SearchAction's EntryPoint are value objects: they have no identity and nothing should reference them. Counting them penalized a site for marking breadcrumbs up correctly.
2. AR-5.3 resolution is opt-in. Band 4 requires sameAs links to resolve, which means requesting Wikidata, ROR and LinkedIn — hosts the client never allowlisted, and hosts that learn which site is being audited. The scanner classifies by authority tier from the markup alone and declares an observed ceiling of band 3 until told to fetch.
3. AR-7.6 expects an autocomplete token only where the field type implies one (email, tel, url, password). A search box has no sensible token and must not be marked down for lacking one.
4. "Widespread" in AR-2.2 needs a count as well as a share. One three-hop chain on a two-page site is a finding, not a systemic failure.
5. Every band ceiling now states its reason in the result and in the report: awaiting a baseline run, awaiting confirmation from the client, limited by the band attested, or needing review by a person. A client told a number was held down is owed the reason.
6. AR-2.8's detection method is weaker than this document implied. The IndexNow key file is named after the key, and the key is not published, so a third party can only find it where the site advertises it. A negative result is not discoverable, which is weaker than absent, and the check says so.
7. AR-6.6 is not implemented. Reading C2PA manifests means parsing image and PDF bytes. The check reports that it had no evidence and leaves the denominator rather than scoring a site zero for a measurement nobody took.

1.1 — 2026-09-01

Twelve gaps found while building the first scanner against 1.0. Several of these would ordinarily force a major bump, because they change how a score is computed. They are landing as a minor bump because no site has ever been scored under 1.0 — the scanner did not exist yet, so there is no trend line to break. This exemption applies once. Every change after this one follows the rule above.

1. Automated coverage now counts auto-partial at half weight. Under the previous wording the metric could not exceed 0.83, while this document's own example result showed 0.87 — arithmetically unreachable.
2. Attestations may lift an auto-partial band ceiling, never set or lower a band. Four checks (AR-2.8, AR-4.1, AR-5.1, AR-8.7) had a top band this document itself says requires attestation, and no mechanism to supply one.
3. Cross-run checks cap at band 3 on a first run (AR-7.9 at 2) and lift with a supplied baseline. AR-2.5, AR-3.6 and AR-7.9 previously required a second audit with no definition of what run one does.
4. The ten AI agents are sorted into three classes. AR-1.1's bands referenced categories the document never defined, and AR-1.2 asked us to probe two user-agents that no crawler ever sends.
5. Money-page double weighting is defined as counting 2 in both sides of the coverage rubric. Previously asserted and never operationalized.
6. A gated run grades as Gated and gains an ungated_grade field. The bands table offered two labels for the same run.
7. "Requires AR-X.Y to pass" means band ≥ 2, with N/A cascading to dependents. Previously undefined against a 0–4 scale.
8. Business-fact applicability comes from a declared engagement profile, never inference. Undeclared scores N/A with a reason.
9. A 25-URL representation sub-sample bounds AR-1.9, AR-4.3, AR-4.5 and AR-4.6, which together would otherwise triple the request count against a client's origin.
10. confidence is an enum: high, medium, low.
11. AR-3.1 coverage clamps to 1.0 and excludes empty renders; a missing robots.txt scores AR-1.1 band 4; score_pct is null where nothing applies; backlog ties break on check ID; site.cms is best-effort and nullable.
12. Rounding happens once, and the file records that per-check points will not sum exactly to the total.

68 checks · 100.0 points

Maintained by TEN7. Corrections are welcome as a pull request.