robots-txt-parse

Fetch and parse a website's robots.txt.

Fetch and parse a website's robots.txt. Returns rules per user-agent, sitemaps, crawl delays, and whether all crawlers are blocked.

UNRATEDListed

Overview

Grade?

UNRATED

0 of 30 paid calls toward a letter

Price

$0.0216 per call

Paid calls

none yet

delivery unknown until someone pays

More info

Seller?

0x66d7c2f9…54ea83bc

URL

https://api.strale.io/x402/robots-txt-parse

MCP server on this host: io.github.strale-io/strale

Seen

first 34 days ago · last 2 days ago

63 free handshakes

Payment

Pays through

x402

Offered on?

Base USDC

Listed on?

Base

Score, from free handshakes onlyweights 2026.09.6

UN
RATED

0.693 liveness verified, no letter

To a letter · 30 paid calls

0 of 30

Held back by sample size, not by the seller · gap to 1.000 is 0.307: 0.232 uncertain ? + 0.075 short ? · 0.107 to B

How the composite of 0.693 is made up, part by part
Component	Weight	Measured	Lower bound ?		Adds	Short ?	Uncertain ?
latency p95costs the most it answered as fast as its class	0.20	0.68	0.000		0.000	−0.064	−0.136
liveness it answered at all	0.60	0.98	0.822		0.493	−0.011	−0.096
price stability the price stayed where it was listed	0.20	1.00	1.000		0.200	0	0
Composite	1.00		0.693	gap to 1.000 = 0.307 · 0.075 short · 0.232 uncertain	0.693	−0.075	−0.232

Not measured: correctness · nobody paid; honesty · nobody paid; schema conformance · no answer had a published shape to check

The multiplication, written out

  0.200 × 0.000   latency p95    
+ 0.600 × 0.822   liveness       
+ 0.200 × 1.000   price stability
= 0.693           → UNRATED: 57 of 30 paid calls      weights 2026.09.6 · 57 samples · seed cap_01M1A5Q7PQRBQGFXPN8K3KTWD5|2026-08-31T15:09:17.658Z|2026.09.6

Verify it yourselfnpx teppi-check https://api.strale.io/x402/robots-txt-parsecurl -s https://api.teppi.xyz/v1/trust/cap_01M1A5Q7PQRBQGFXPN8K3KTWD5