robots?url=https%3A%2F%2Fexample.com

Fetch and parse robots.txt (+ llms.txt if present) into structured JSON, including which AI crawlers are blocked.

UNRATEDListed

Overview

Grade?

UNRATED

0 of 30 paid calls toward a letter

Price

$0.005 per call

Paid calls

none yet

delivery unknown until someone pays

More info

Seller?

0x55ae5d17…8dba4e20

URL

https://toolvend.dev/parse/robots?url=https%3A%2F%2Fexample.com

MCP server on this host: dev.toolvend/dns-whois-domain-tools

Seen

first 26 days ago · last 13 hr ago

48 free handshakes

Payment

Pays through

x402

Offered on?

Base, Solana USDC

Listed on?

Base, Solana

Buy it through Teppi

?Pay on delivery: 0.007000 USDC (0.005000 to the seller, 0.002000 fee)
Where: https://api.teppi.xyz/v1/via/cap_01M1ZRPQWWV926J5BS6ERRWDFM
?Ask the price: curl -si https://api.teppi.xyz/v1/via/cap_01M1ZRPQWWV926J5BS6ERRWDFM

About

?Description

Fetch and parse robots.txt (+ llms.txt if present) into structured JSON, including which AI crawlers are blocked.

URL

https://toolvend.dev/parse/robots?url=https%3A%2F%2Fexample.com

Method

GET

Class

extraction

?Checkable to

Endpoint id

cap_01M1ZRPQWWV926J5BS6ERRWDFM

?Badge

Reads live. UNRATED until paid calls earn a letter.

Markdown: [![Teppi](https://api.teppi.xyz/badge/cap_01M1ZRPQWWV926J5BS6ERRWDFM.svg)](https://teppi.xyz/endpoint/cap_01M1ZRPQWWV926J5BS6ERRWDFM)
HTML: <a href="https://teppi.xyz/endpoint/cap_01M1ZRPQWWV926J5BS6ERRWDFM"><img src="https://api.teppi.xyz/badge/cap_01M1ZRPQWWV926J5BS6ERRWDFM.svg" alt="Teppi delivery record"></a>
Image: https://api.teppi.xyz/badge/cap_01M1ZRPQWWV926J5BS6ERRWDFM.svg

Verify it yourselfnpx teppi-check https://toolvend.dev/parse/robots?url=https%3A%2F%2Fexample.comcurl -s https://api.teppi.xyz/v1/trust/cap_01M1ZRPQWWV926J5BS6ERRWDFM