Agent-ready sites

Pondoknusa publishes the files and headers Is Agentic looks for. registerHttpMiddleware() and prepareHttpServer() turn this on. You do not add routes by hand.

Set agent.enabled: false in config/agent.ts to opt out.

What the framework serves

Path	Role
/robots.txt	Allows major AI crawlers and sets Content-Signal
/sitemap.xml	Indexable GET routes
/sitemap.md	Markdown sitemap
/llms.txt	Site guide with a When to use section
/llms-full.txt	Guide plus the markdown sitemap
/openapi.json	OpenAPI 3.0 stub from registered routes
/.well-known/api-catalog	RFC 9727 linkset that points at the OpenAPI document

Existing app routes win. If you already register /robots.txt, Pondoknusa leaves it alone.

HTML pages also get:

• Accept: text/markdown negotiation and Vary: Accept (applied in the HTTP kernel, so it still runs when routes register before middleware)
• JSON-LD (WebSite plus Person / Organization / SoftwareApplication)
• JSON-LD (WebSite plus Person / Organization / SoftwareApplication)
• Canonical and Open Graph tags when the view omits them
• Link headers for llms.txt, the API catalog, markdown, and OpenAPI

Errors return RFC 9457 application/problem+json (with the existing message / status fields) or a short markdown body. Rate-limit middleware adds IETF RateLimit / RateLimit-Policy headers and Retry-After on 429.

Streamed HTML is not buffered. JSON-LD injection runs on string HTML bodies (including Response.ssr()).

Configure identity

pondoknusa new writes config/agent.ts:

import type { AgentConfig } from '@pondoknusa/core';

export default {
  enabled: true,
  identity: 'Organization',
  description: 'A Pondoknusa website.',
  whenToUse: [
    "Read this site's public pages, sitemap, and OpenAPI description.",
  ],
  contentSignal: {
    search: true,
    aiTrain: true,
    aiInput: true,
  },
} satisfies AgentConfig;

Name, URL, and locale come from config/app.ts. Set identity to Person for a personal site.

Optional keys:

Key	Notes
sameAs	Profile URLs for JSON-LD
contactEmail	JSON-LD ContactPoint
paths.robots	Custom path, or false to skip
paths.sitemap / sitemapMarkdown / llms / llmsFull / openapi / apiCatalog	Same pattern

Check a site

npx is-agentic your-domain.example

The public pages are at https://is-agentic.com/. Essential checks need a reachable origin: server-rendered HTML, real 404s, markdown negotiation, /openapi.json, and crawlers that are not blocked.

pondoknusa doctor warns when public/robots.txt disallows GPTBot, ClaudeBot, and similar crawlers.

Cloudflare and other WAFs

The framework cannot override a proxy that challenges or blocks agents.

On Cloudflare:

1. Do not enable Block AI Crawlers / AI Crawl Control if you want Is Agentic access checks to pass.
2. Do not add managed robots.txt rules that Disallow: / for GPTBot, ClaudeBot, ChatGPT-User, PerplexityBot, Google-Extended, or Applebot-Extended.
3. Bot Fight Mode can still challenge non-browser clients. Test curl from a datacenter IP.
4. Edge cache must honour Vary: Accept. If a cache rule stores HTML without that vary token, agents can receive HTML for Accept: text/markdown.

See Cloudflare Module 6.

What you still write

Pondoknusa does not invent trust pages, payment APIs, or MCP servers. Add /privacy and /terms when they apply. Point whenToUse at the real jobs this site can do. Publish a richer OpenAPI document when the stub is not enough (pondoknusa make:openapi).

Do not advertise an HTTP MCP catalog unless the app actually serves MCP on that host.