Endpoint · extractionGETsaylorinnovations.com /api/web/robots-check ?path=%2Fprivate%2Fpublic%2Fa.html&robots=User-agent%3A+*%0ADisallow%3A+%2Fprivate%2F%0AAllow%3A+%2Fprivate%2Fpublic%0A%0AUser-agent%3A+GPTBot%0ADisallow%3A+%2F%0ASitemap%3A+https%3A%2F%2Fexample.com%2Fsitemap.xml&user_agent=Googlebot
Check whether a crawler may fetch a path under a robots.txt per RFC 9309 - matched group and rule, crawl delay and sitemaps, for Googlebot, GPTBot or any agent
Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.01 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://saylorinnovations.com/api/web/robots-check?path=%2Fprivate%2Fpublic%2Fa.html&robots=User-agent%3A+*%0ADisallow%3A+%2Fprivate%2F%0AAllow%3A+%2Fprivate%2Fpublic%0A%0AUser-agent%3A+GPTBot%0ADisallow%3A+%2F%0ASitemap%3A+https%3A%2F%2Fexample.com%2Fsitemap.xml&user_agent=Googlebot
MCP server on this host: com.saylorinnovations/data
Seen
first 28 days ago · last 10 days ago
64 free handshakes
Payment
Pays through
x402
Offered on?
Polygon, Arbitrum, Base USDC, Solana
Listed on?
Polygon, Arbitrum, Base, Solana
Buy it through Teppi
- ?Pay on delivery
- 0.012000 USDC (0.010000 to the seller, 0.002000 fee)
- Where
- https://api.teppi.xyz/v1/via/cap_01M5EFT2CB0JDK6ESYACGN8DK4
- ?Ask the price
curl -si https://api.teppi.xyz/v1/via/cap_01M5EFT2CB0JDK6ESYACGN8DK4
About
- ?Description
- Check whether a crawler may fetch a path under a robots.txt per RFC 9309 - matched group and rule, crawl delay and sitemaps, for Googlebot, GPTBot or any agent
- URL
- https://saylorinnovations.com/api/web/robots-check?path=%2Fprivate%2Fpublic%2Fa.html&robots=User-agent%3A+*%0ADisallow%3A+%2Fprivate%2F%0AAllow%3A+%2Fprivate%2Fpublic%0A%0AUser-agent%3A+GPTBot%0ADisallow%3A+%2F%0ASitemap%3A+https%3A%2F%2Fexample.com%2Fsitemap.xml&user_agent=Googlebot
- Method
- GET
- Class
- extraction
- ?Checkable to
- L1
- Endpoint id
- cap_01M5EFT2CB0JDK6ESYACGN8DK4
- ?Badge
Reads live. UNRATED until paid calls earn a letter.
- Markdown
- [](https://teppi.xyz/endpoint/cap_01M5EFT2CB0JDK6ESYACGN8DK4)
- HTML
- <a href="https://teppi.xyz/endpoint/cap_01M5EFT2CB0JDK6ESYACGN8DK4"><img src="https://api.teppi.xyz/badge/cap_01M5EFT2CB0JDK6ESYACGN8DK4.svg" alt="Teppi delivery record"></a>
- Image
- https://api.teppi.xyz/badge/cap_01M5EFT2CB0JDK6ESYACGN8DK4.svg
Verify it yourself
npx teppi-check https://saylorinnovations.com/api/web/robots-check?path=%2Fprivate%2Fpublic%2Fa.html&robots=User-agent%3A+*%0ADisallow%3A+%2Fprivate%2F%0AAllow%3A+%2Fprivate%2Fpublic%0A%0AUser-agent%3A+GPTBot%0ADisallow%3A+%2F%0ASitemap%3A+https%3A%2F%2Fexample.com%2Fsitemap.xml&user_agent=Googlebotcurl -s https://api.teppi.xyz/v1/trust/cap_01M5EFT2CB0JDK6ESYACGN8DK4