Endpoint · extractionGETnetintel.dev /robots-txt/analyze ?domain=example.com
Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked…
Read more
Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked for any bot — so agents can respect crawl policies and locate sitemaps before scraping.Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.01 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://netintel.dev/robots-txt/analyze?domain=example.com
Seen
first 34 days ago · last 14 hr ago
64 free handshakes
Payment
Pays through
x402
Offered on?
Base USDC, Solana
Listed on?
Base, Solana
Handshakescomputed 2 days ago
0.789 from 58 free handshakes over 30 days · UNRATED ?
| Component | Weight | Measured | Lower bound ? | Adds | Short ? | Uncertain ? | |
|---|---|---|---|---|---|---|---|
| latency p95costs the most it answered as fast as its class |
0.20 | 0.36 | 0.262 | 0.052 | −0.128 | −0.019 | |
| liveness it answered at all |
0.60 | 1.00 | 0.894 | 0.536 | 0 | −0.064 | |
| price stability the price stayed where it was listed |
0.20 | 1.00 | 1.000 | 0.200 | 0 | 0 | |
| Composite | 1.00 | 0.789 | gap to 1.000 = 0.211 · 0.128 short · 0.083 uncertain | 0.789 | −0.128 | −0.083 |
Not measured: correctness · nobody paid; honesty · nobody paid; schema conformance · no answer had a published shape to check
BURN_INFREE_TIER_ONLY
Verify it yourself
npx teppi-check https://netintel.dev/robots-txt/analyze?domain=example.comcurl -s https://api.teppi.xyz/v1/trust/cap_01M1A59M5GWWXMXJK609Y01EHD