Endpoint · searchGETfreetier-agent-api.pages.dev /api/robots-check ?ua=MyAgent&url=https%3A%2F%2Fexample.com%2Fsome%2Fpage
Check whether a URL may be crawled per the site's robots.txt for a given user-agent.
Read more
Check whether a URL may be crawled per the site's robots.txt for a given user-agent. Returns allowed, matched rule, crawl-delay, sitemaps.Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.003 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://freetier-agent-api.pages.dev/api/robots-check?ua=MyAgent&url=https%3A%2F%2Fexample.com%2Fsome%2Fpage
MCP server on this host: dev.pages.freetier-agent-api/freetier-agent-api
Seen
first 17 days ago · last 4 days ago
22 free handshakes
Payment
Pays through
x402
Offered on?
Base USDC
Listed on?
Base
Handshakescomputed 1 day ago
0.399 from 19 free handshakes over 30 days · UNRATED ?
| Component | Weight | Measured | Lower bound ? | Adds | Short ? | Uncertain ? | |
|---|---|---|---|---|---|---|---|
| livenesscosts the most it answered at all |
0.60 | 0.79 | 0.331 | 0.199 | −0.128 | −0.273 | |
| latency p95 it answered as fast as its class |
0.20 | 0.00 | 0.000 | 0.000 | −0.200 | 0 | |
| price stability the price stayed where it was listed |
0.20 | 1.00 | 1.000 | 0.200 | 0 | 0 | |
| Composite | 1.00 | 0.399 | gap to 1.000 = 0.601 · 0.328 short · 0.273 uncertain | 0.399 | −0.328 | −0.273 |
Not measured: correctness · nobody paid; honesty · nobody paid; schema conformance · no answer had a published shape to check
BURN_INFREE_TIER_ONLY
Verify it yourself
npx teppi-check https://freetier-agent-api.pages.dev/api/robots-check?ua=MyAgent&url=https%3A%2F%2Fexample.com%2Fsome%2Fpagecurl -s https://api.teppi.xyz/v1/trust/cap_01M6AT47ANFEK4JYBF22D1FGR8