Free tool
Score your tool schemas against 4,749 public servers
Paste the tools/list JSON your
server returns. You get your undescribed-parameter rate, your description collisions and
your schema token cost, each next to the same figure measured across 4,749 servers in our survey.
It runs entirely in this tab. Your schema is never uploaded, and there is no request to remove — how it works.
The whole JSON-RPC response, the result object,
or just the array of tools — any of the three.
What it measures
Five things, all of them properties of a schema sitting still. None of them needs a model in the loop, and none of them can tell you how often a model actually gets it wrong — that needs traffic, which is the other half of this.
Undescribed parameters
21.5% → 24.7%
A parameter with no description is a parameter the model guesses at. It has the name and the type, and that is the whole brief. A description that only repeats the name is counted too — it passes every linter, and it takes the survey's rate from 21.5% to 24.7%.
Description collisions
22.9%
Tools whose description contains no word that its siblings do not also use. Clear English that still cannot tell a model which tool to pick. Measured over names too, because that is what a model falls back to when the descriptions stop discriminating.
Tools that act on the same thing
25.1% of servers
Two tools that operate on the same object can both be correct for one request, however differently they are worded. This is the pair a request routes to by accident, and a distinctive description does not prevent it.
Schema token cost
~1,251 tokens
Your whole tool list is sent on every connection, whether or not a single tool gets called.
Undeclared closed sets
857 parameters
A parameter that names its valid values in prose and then leaves the schema an open string.
Parameters are counted at the top level
— inputSchema.properties, not nested object properties.
That is what the survey counted, so it is what a comparison can be made against.
Collisions are measured within your paste. A tool's description is distinctive relative to the other tools you gave it, so pasting half your server will flatter it.
Acting on the same thing is not the same as being worded alike. Distinctive share asks whether two descriptions look alike. Input overlap asks whether a
request could route to either tool, which is the question that predicts a wrong pick — and a
pair can be distinctively worded and still share every object it acts on. Input nouns are a
description's content words, minus the opening verb and minus anything after returns or a colon, plus your parameter names. No grammar model behind it, so adjectives survive as
nouns: that dilutes an overlap rather than inventing one.
Names are measured the same way. Authors
report that when descriptions stop discriminating, a model falls back to pattern-matching on
the tool name — so the name is the backstop, and a server whose descriptions collide and
whose names are clear is in a different position from one where both have gone. A prefix
every tool shares (gmail_) is stripped first: it says which
server a tool belongs to and discriminates nothing inside it.
Tokens are bytes ÷ 4, the same estimate the survey uses. Close enough to compare servers, not exact for a bill. Size is measured re-serialised, so pretty-printing your paste does not change the score.
Nothing here grades correctness. A zero-distinctive description can be perfectly good writing — three of the four examples in the survey are — and a schema that scores clean can still fail constantly for reasons no schema shows.