Endpoint · extractionGETautomatom-x402.netlify.app /robots
Fetches and parses a site's robots.txt into user-agent groups (allow/disallow rules), crawl-delay and sitemap URLs.
Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.01 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://automatom-x402.netlify.app/robots
MCP server on this host: app.netlify.automatom-x402/web-data-toolkit
Seen
first 11 days ago · last 4 days ago
8 free handshakes
Payment
Pays through
— never asked
Offered on?
— never asked
Listed on?
Base
About
- ?Description
- Fetches and parses a site's robots.txt into user-agent groups (allow/disallow rules), crawl-delay and sitemap URLs.
- URL
- https://automatom-x402.netlify.app/robots
- Method
- GET
- Class
- extraction
- ?Checkable to
- L1
- Endpoint id
- cap_01M6WFY33SR5T41SK4YP0C414N
- ?Badge
Reads live. UNRATED until paid calls earn a letter.
- Markdown
- [](https://teppi.xyz/endpoint/cap_01M6WFY33SR5T41SK4YP0C414N)
- HTML
- <a href="https://teppi.xyz/endpoint/cap_01M6WFY33SR5T41SK4YP0C414N"><img src="https://api.teppi.xyz/badge/cap_01M6WFY33SR5T41SK4YP0C414N.svg" alt="Teppi delivery record"></a>
- Image
- https://api.teppi.xyz/badge/cap_01M6WFY33SR5T41SK4YP0C414N.svg
Verify it yourself
npx teppi-check https://automatom-x402.netlify.app/robotscurl -s https://api.teppi.xyz/v1/trust/cap_01M6WFY33SR5T41SK4YP0C414N