Endpoint · lookupGETx402.agentharbor.agency /ai-crawlers ?domain=agentharbor.agency
Given a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot…
Read more
Given a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and more, each allowed or blocked, plus Googlebot, listed sitemaps, whether /llms.txt is published and page-level noindex/nosnippet, with fixes. Static check, no browser. Not charged if robots.txt cannot be read (no response, or a bot block).Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.01 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://x402.agentharbor.agency/ai-crawlers?domain=agentharbor.agency
MCP server on this host: io.github.yandyman/agent-harbor-x402
Seen
first 34 days ago · last 17 hr ago
38 free handshakes
Payment
Pays through
x402 · MPP tempo
Offered on?
Base USDC, Tempo USDC.e
Listed on?
Base
Paid calls
No paid calls yet.
Delivery is unknown until someone pays. A handshake only shows it asks for payment.
Paid for on the same host1 of 10 endpoints at x402.agentharbor.agency
| Endpoint | Grade | Paid calls | Last result | Last seen |
|---|---|---|---|---|
| GET /domain-verify?domain=mailchimp.com | UNRATED | 1 | answered, not checked | 17 hr ago |
Verify it yourself
npx teppi-check https://x402.agentharbor.agency/ai-crawlers?domain=agentharbor.agencycurl -s https://api.teppi.xyz/v1/trust/cap_01M4Z1HNEEKQFR86GP0ETM2VCN