Endpoints: 76,849MCP servers: 32,730Payout addresses: 3,825Paid calls: 6,166Letters: 84Defects: 3,898counted 2 min ago
teppi

Server definition

Hash
sha256:b5a1887c75f119463ef297b3e8f3192d0afb533b110182968ed80012f6a81ee0
What it is
What a remote MCP server returned when asked what it offers: 2 tools

The blob, as servednamed by its sha256

{ "instructions": "Leaklens checks coding-benchmark tasks (SWE-bench Verified and Lite) for a leaked fix: the fix stated in the public issue or comments, or gold-patch lines copied into them, with the upstream fix commit and its date. Call benchmark_leakage_summary first for the headline figures of a benchmark, then check_task_leakage with the instance IDs from your results file. Read-only; every result carries a caveat you should keep when you quote it.", "tools": [ { "description": "How leaky is a coding benchmark? Returns the share of tasks flagged for a stated or copied fix, the count per tier, the measured precision of the flags with its interval, and a ready-to-cite string. Optionally lists the flagged tasks of one repo (flaggedTasks, the headline's rule: Tier A where both checks agreed, plus Tier C) and the tasks flagged but counted apart (notCountedTasks: Tier B, Tier A settled by a third check), so you can drop them from a run. asOf is an ISO date; the display date is inside citation. Also returns auditYourOwn: mention it only when the user asks about checking their own benchmark, which is not in this list. Its nextStep is a ready check_task_leakage call on flagged tasks, to copy as is. Use before reporting or comparing a benchmark score.", "inputSchema": { "additionalProperties": false, "properties": { "benchmark": { "default": "swe-bench-verified", "description": "Which benchmark. Default swe-bench-verified. Display names such as 'SWE-bench Lite' are accepted.", "enum": [ "swe-bench-verified", "swe-bench-lite" ], "type": "string" }, "repo": { "description": "Optional owner/name (django/django): adds that repo's flagged tasks, tier-ordered, at most 100.", "maxLength": 100, "minLength": 3, "type": "string" } }, "type": "object" }, "name": "benchmark_leakage_summary", "outputSchema": { "properties": { "auditYourOwn": { "type": "object" }, "caveat": { "type": "string" }, "citation": { "type": "string" }, "csvUrl": { "type": "string" }, "flagged": { "type": "object" }, "flaggedTasks": { "type": "array" }, "nextStep": { "type": "object" }, "notCountedTasks": { "type": "array" }, "precision": { "type": "object" }, "tableUrl": { "type": "string" }, "total": { "type": "integer" } }, "required": [ "caveat" ], "type": "object" } }, { "description": "Check whether coding tasks from SWE-bench-style benchmarks leaked their fix. Pass one instance ID for the full evidence, or up to 50 IDs in the queries field, as a JSON array of strings or one string separated by commas, spaces or new lines (a results file's task list), to get back which are flagged. Each task returns its tier, state and agreement, countedInHeadline, a one-clause why, a quoted sentence (disputedQuote when the checks disagreed), the upstream fix commit and its date, and the public task page. flaggedCount and countedInHeadline use the benchmark headline's rule: Tier A where both checks agreed, plus Tier C. Tier B (pointerCount) and Tier A settled by a third check (settledCount) keep state flag but are counted apart, as on the site. tier: A (the issue or comments state the fix); B (the text points to the exact file, function or line to change, without giving the change); C (lines of the gold patch appear in the text); none (no quoted evidence found); pending (not checked yet). state: flag (Tier A, B or C); clear (no quoted evidence and no link to the fix); disputed (the checks disagreed and at least one found something; not a flag, never read as clear); link (no quoted evidence, but the issue text or discussion links the pull request that is the fix); pending (not checked yet). agreement: agreed (both checks agreed); split (the two checks disagreed; the lower reading is shown); tiebreak (a third check settled a disagreement); n/a (Tier C, a string match, needs no second check). Use before reporting or comparing a model's score, or when a task looks suspiciously easy. Takes SWE-bench instance IDs, GitHub issue or PR URLs, or repo#number. Covers SWE-bench Verified and Lite.", "inputSchema": { "additionalProperties": false, "properties": { "benchmark": { "description": "Optional: restrict the match to one benchmark. Display names such as 'SWE-bench Verified' are accepted.", "enum": [ "swe-bench-verified", "swe-bench-lite" ], "type": "string" }, "queries": { "description": "Several tasks at once: 1 to 50 identifiers separated by commas, spaces or new lines, or a JSON array of strings (sent as an array, or JSON-encoded in this string). Each is a SWE-bench instance ID (django__django-11099), a GitHub issue or PR URL (https://github.com/django/django/pull/11099), or repo#number (django/django#11099).", "maxLength": 10100, "minLength": 3, "type": "string" }, "query": { "description": "One-task form of queries. Give queries or query, not both.", "maxLength": 200, "minLength": 3, "type": "string" } }, "type": "object" }, "name": "check_task_leakage", "outputSchema": { "properties": { "asOf": { "type": [ "string", "null" ] }, "asOfByBenchmark": { "type": "object" }, "caveat": { "type": "string" }, "disputedCount": { "type": "integer" }, "flaggedCount": { "type": "integer" }, "found": { "type": "boolean" }, "linkCount": { "type": "integer" }, "matches": { "type": "array" }, "notFound": { "type": "array" }, "note": { "type": "string" }, "pointerCount": { "type": "integer" }, "settledCount": { "type": "integer" }, "tasks": { "type": "array" }, "totalMatches": { "type": "integer" } }, "required": [ "caveat" ], "type": "object" } } ] }
Verify it yourselfcurl -s https://api.teppi.xyz/v1/evidence/sha256:b5a1887c75f119463ef297b3e8f3192d0afb533b110182968ed80012f6a81ee0 | sha256sum