Endpoints: 28,729MCP servers: 18,413Payout addresses: 2,071Paid calls: 1,537Letters: 14Defects: 1,323counted 3 min ago
teppi

Endpoint · extractionPOSTx402.forgemesh.io/robots-txt-ai-check

Fetches a domain's robots.txt and parses its Content-Signal and AI-preference directives (search, ai-input, ai-train) into structured JSON plus a plain-English summary of what's…
Read moreFetches a domain's robots.txt and parses its Content-Signal and AI-preference directives (search, ai-input, ai-train) into structured JSON plus a plain-English summary of what's allowed — the "can this domain be crawled by an AI agent" check. Run before scraping content or building a RAG index, ahead of the industry's move toward stricter bot-gating defaults later this year.
UNRATEDDelisted

Overview

Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.005 per call
Paid calls
none yet
delivery unknown until someone pays

More info

Seller?
0x84a1827f…a4a57a08
URL
https://x402.forgemesh.io/robots-txt-ai-check
Seen
first 34 days ago · last 34 days ago
13 free handshakes

Payment

Pays through
x402 · MPP evm
Offered on?
Base USDC
Listed on?
Base

About

?Description
Fetches a domain's robots.txt and parses its Content-Signal and AI-preference directives (search, ai-input, ai-train) into structured JSON plus a plain-English summary of what's allowed — the "can this domain be crawled by an AI agent" check. Run before scraping content or building a RAG index, ahead of the industry's move toward stricter bot-gating defaults later this year.
URL
https://x402.forgemesh.io/robots-txt-ai-check
Method
POST
Class
extraction
?Checkable to
L1
Endpoint id
cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG
?Badge

Reads live. UNRATED until paid calls earn a letter.

Markdown
[![Teppi](https://api.teppi.xyz/badge/cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG.svg)](https://teppi.xyz/endpoint/cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG)
HTML
<a href="https://teppi.xyz/endpoint/cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG"><img src="https://api.teppi.xyz/badge/cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG.svg" alt="Teppi delivery record"></a>
Image
https://api.teppi.xyz/badge/cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG.svg
Verify it yourselfnpx teppi-check https://x402.forgemesh.io/robots-txt-ai-checkcurl -s https://api.teppi.xyz/v1/trust/cap_01M1A5Q9KEYCEBYHK9S8QVJ8AG