Endpoint · extractionGETnetintel.dev /robots-txt/analyze ?domain=example.com
Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked…
Read more
Fetch and parse a domain's robots.txt file — returns all crawl rules by user-agent, sitemap URLs, crawl delay settings, and checks whether a specific path is allowed or blocked for any bot — so agents can respect crawl policies and locate sitemaps before scraping.Overview
Grade?
UNRATED
0 of 30 paid calls toward a letter
Price
$0.01 per call
Paid calls
none yet
delivery unknown until someone pays
More info
Seller?
URL
https://netintel.dev/robots-txt/analyze?domain=example.com
Seen
first 2 months ago · last 4 days ago
113 free handshakes
Payment
Pays through
x402
Offered on?
Base USDC, Solana
Listed on?
Base, Solana
Score, from free handshakes onlyweights 2026.09.6
UN
RATED
RATED
0.801 liveness verified, no letter
To a letter · 30 paid calls
0 of 30
Held back by sample size, not by the seller · gap to 1.000 is 0.199: 0.139 uncertain ? + 0.061 short ? · 0.099 to A
| Component | Weight | Measured | Lower bound ? | Adds | Short ? | Uncertain ? | |
|---|---|---|---|---|---|---|---|
| livenesscosts the most it answered at all |
0.60 | 1.00 | 0.795 | 0.477 | 0 | −0.123 | |
| latency p95 it answered as fast as its class |
0.20 | 0.70 | 0.618 | 0.124 | −0.061 | −0.016 | |
| price stability the price stayed where it was listed |
0.20 | 1.00 | 1.000 | 0.200 | 0 | 0 | |
| Composite | 1.00 | 0.801 | gap to 1.000 = 0.199 · 0.061 short · 0.139 uncertain | 0.801 | −0.061 | −0.139 |
Not measured: correctness · nobody paid; honesty · nobody paid; schema conformance · no answer had a published shape to check
The multiplication, written out
0.600 × 0.795 liveness
+ 0.200 × 0.618 latency p95
+ 0.200 × 1.000 price stability
= 0.801 → UNRATED: 30 of 30 paid calls weights 2026.09.6 · 30 samples · seed cap_01M1A59M5GWWXMXJK609Y01EHD|2026-10-16T03:01:24.827Z|2026.09.6
Verify it yourself
npx teppi-check https://netintel.dev/robots-txt/analyze?domain=example.comcurl -s https://api.teppi.xyz/v1/trust/cap_01M1A59M5GWWXMXJK609Y01EHD