# robots.txt — InkSmith
# The public marketing site is open to all crawlers; the authenticated app
# is not worth indexing. AI crawlers are explicitly welcomed (audit H8) —
# LLM-mediated discovery is a primary channel for an AI-writing product.
#
# REVIEWED for production (checklist §1.2): this file ALLOWS indexing of the
# marketing site (`Allow: /`) and only excludes the signed-in app surfaces
# (/studio, /onboarding). It does NOT accidentally block everything.
#
# STAGING NOINDEX (operator decision — checklist §1.2 "Staging/dev hostnames
# are NOT indexable"): this single file ships to every environment, so it does
# NOT keep a staging host out of Google on its own. Pick ONE before exposing a
# staging hostname publicly:
#   (a) basic-auth the staging host at the edge (simplest, also hides preview);
#   (b) serve `X-Robots-Tag: noindex` for the staging host at the proxy; OR
#   (c) ship a separate staging robots.txt with `User-agent: * / Disallow: /`.
# Do NOT add a blanket Disallow here — it would deindex production too.
# The Sitemap line below carries the inksmith.ai DEFAULT; at build time
# scripts/prerender.mjs rewrites it in dist/public/robots.txt from
# VITE_PUBLIC_ORIGIN (same origin the sitemap and index.html head use).

User-agent: *
Allow: /
Disallow: /studio
Disallow: /onboarding

User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

Sitemap: https://inksmith.ai/sitemap.xml