Move skills to .agent directory
Some checks failed
Build / build (push) Successful in 2m0s
OpenWiki Update / update (push) Has been cancelled

This commit is contained in:
Tim Krampitz
2026-10-04 22:13:19 +02:00
parent 8bccf66e84
commit ffb017568b
186 changed files with 0 additions and 0 deletions

View File

@@ -0,0 +1,31 @@
---
name: agent-ready-robots-txt
description: >
Sub-skill de agent-ready-cloudflare: Implement robots.txt
---
# Implement robots.txt
Publish a valid robots.txt at your site root per
[RFC 9309](https://www.rfc-editor.org/rfc/rfc9309).
## Requirements
- Serve `/robots.txt` as `text/plain` with HTTP 200
- Include `User-agent` directives with `Allow`/`Disallow` rules
- Reference your sitemap if one exists: `Sitemap: https://example.com/sitemap.xml`
## Cloudflare
[AI Crawl Control](https://developers.cloudflare.com/ai-crawl-control/)
can manage your robots.txt from the dashboard, including AI-specific bot rules.
## Validate
```
POST https://isitagentready.com/api/scan
Content-Type: application/json
{"url": "https://YOUR-SITE.com"}
```
Check that `checks.discoverability.robotsTxt.status` is `"pass"`.