Writing
Page 4 of 5
Older posts on measuring MCP servers — retries, empty results, schema cost, and what the four outcomes of a tool call actually tell you.
Page 4 of 5
-
Measurement 6 minWhat does your MCP tool list cost per connection?
The median public MCP server spends about 1,250 tokens of schema before anyone asks a question. The 90th percentile spends 7,992. One spends 286,000.
-
Tool design 7 minHow many tools should an MCP server have?
Across 4,749 public servers, the share of tools with nothing to tell them apart rises from 0.5% to 32.4% as the tool count grows. That is a factor of sixty-three.
-
Tool design 5 min21.5% of MCP parameters have no description at all
55,383 of 257,287 parameters across 4,749 servers ship with nothing but a name and a type. Only 0.4% of tools do. That asymmetry is the whole finding.
-
Tool design 8 minWe read the schemas of 4,749 public MCP servers
Not what models do with your tools — what they are handed first. 82,549 tools, 257,287 parameters, one tool in six with nothing to tell it from its neighbour.
-
Measurement 5 minEmpty results are failures the protocol calls success
An empty array is a 200. The model got nothing it could use, the user got a shrug, and no error was recorded anywhere. These are the failures nobody reports.
-
Tool design 5 minYour tool schemas are billed on every session, used or not
Descriptions and JSON Schema go into the context window at the start of every conversation. Twelve tools can cost 3,000 tokens before anyone asks a question.
-
Measurement 4 minDead tools — registered, described, and never once called
Every server has at least one. It costs schema tokens on every session, competes for the model's attention, and returns nothing, because nobody has ever reached it.
-
Engineering 5 minTelling a crash from an error return is harder than it looks
The MCP SDK catches whatever your handler throws and turns it into an error result. From outside the handler, a bug and a deliberate error look identical.
-
Tool design 5 minbad_args is a message about your schema, not about the model
When arguments fail validation, the instinct is that the model got it wrong. Usually the schema asked for something the model had no way to know.
-
Privacy 5 minWhy we hash arguments instead of storing them
Twelve hex characters of a SHA-256 over the arguments, keys sorted. Enough to tell whether two calls were the same, and not enough for anything else.
-
Engineering 5 minWhy the SDK runs inside your server, not in front of it
A proxy is easier to build and the wrong thing to ship. Directory-listed servers cannot change their URL, and OAuth breaks when traffic is redirected.
-
Engineering 4 minPercentiles cannot be summed, so we bucket latency instead
You cannot average two days of p95 and get the week's p95. Four counters can be added across any range and still tell you the truth.