Endpoint · extractionGETnetintel.dev /robots-txt/analyze ?domain=example.com
Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked…
Read more
Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked for any bot — so agents can respect crawl policies and locate sitemaps before scraping.Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.01 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://netintel.dev/robots-txt/analyze?domain=example.com
Seen
first 34 days ago · last 13 hr ago
64 free handshakes
Payment
Pays through
x402
Offered on?
Base USDC, Solana
Listed on?
Base, Solana
Score, from free handshakes onlyweights 2026.09.6
UN
RATED
RATED
0.789 liveness verified, no letter
To a letter · 30 paid calls
0 of 30
Held down by latency p95 · gap to 1.000 is 0.211: 0.083 uncertain ? + 0.128 short ? · 0.011 to B
| Component | Weight | Measured | Lower bound ? | Adds | Short ? | Uncertain ? | |
|---|---|---|---|---|---|---|---|
| latency p95costs the most it answered as fast as its class |
0.20 | 0.36 | 0.262 | 0.052 | −0.128 | −0.019 | |
| liveness it answered at all |
0.60 | 1.00 | 0.894 | 0.536 | 0 | −0.064 | |
| price stability the price stayed where it was listed |
0.20 | 1.00 | 1.000 | 0.200 | 0 | 0 | |
| Composite | 1.00 | 0.789 | gap to 1.000 = 0.211 · 0.128 short · 0.083 uncertain | 0.789 | −0.128 | −0.083 |
Not measured: correctness · nobody paid; honesty · nobody paid; schema conformance · no answer had a published shape to check
The multiplication, written out
0.200 × 0.262 latency p95
+ 0.600 × 0.894 liveness
+ 0.200 × 1.000 price stability
= 0.789 → UNRATED: 58 of 30 paid calls weights 2026.09.6 · 58 samples · seed cap_01M1A59M5GWWXMXJK609Y01EHD|2026-09-02T05:29:18.190Z|2026.09.6
Verify it yourself
npx teppi-check https://netintel.dev/robots-txt/analyze?domain=example.comcurl -s https://api.teppi.xyz/v1/trust/cap_01M1A59M5GWWXMXJK609Y01EHD