Inferock Bench

What is Inferock Bench?

Inferock Bench is a local diagnostic proxy that sits between your application and AI providers like OpenAI, Anthropic, Gemini, or OpenRouter. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for. Instead of trusting the provider's own billing records, Inferock Bench gives you a neutral, per-call audit trail you control.

Who it's for

• Engineering leads managing significant AI/LLM spend who need visibility into whether billed amounts match actual usage.
• FinOps-minded developers tracking token consumption across multiple providers and wanting to catch billing discrepancies early.
• Compliance-conscious teams that require independent evidence of API call failures, retries, and cost exposure for internal audits.

Key features

Local proxy measurement

Inferock Bench routes your API traffic through a localhost proxy, capturing provider-reported usage, pricing evidence, request/response metadata, status, timing, and retry signals. It only measures calls it actually sees, so the data is grounded in your real traffic.

Receipt generation

The tool renders receipts through the shipped @inferock/measure grading code and The Inferock Standard. Each receipt separates provider spend, bill-bounded money loss, time loss, and invoice-check exposure, so you're not comparing apples to oranges.

Coverage transparency

Every surface is marked as watched-clean, signal, or not-openable. This means unopened checks are visible rather than silently assumed clean, giving you an honest picture of what was and wasn't verified.

Provider key safety

Your provider keys are never sent to Inferock by the local benchmark. The proxy attaches them only to provider requests, and receipts stay local unless you choose to share them.

What stands out

"Providers shouldn't get to grade their own bills."

That's the core philosophy here. Inferock Bench doesn't just log calls — it actively surfaces delivery failures that affect your bill, like billed-empty output, refusals, truncation, token-recount mismatches, duplicate request IDs, and cache-discount-at-risk evidence. It also flags provider-fault retries, which are often silently absorbed into your costs. The tool is honest about its limits: it cannot audit traffic that bypassed the proxy, and it doesn't declare every mismatch an overcharge. Instead, it preserves the evidence so billing-integrity questions can be answered with data, not guesswork.

Worth checking out if…

You're spending meaningfully on AI APIs and suspect your bill doesn't match reality. Inferock Bench is especially useful if you need to answer questions like "was I billed for a failed API call?" or want to measure Claude or GPT token usage locally. It's also a strong fit if you're evaluating multiple providers and want a consistent, independent yardstick for cost and failure patterns. The tool is extensible by design, though only four provider planes are measured today: OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints. If you're comfortable with a local proxy setup and want billing transparency you can verify yourself, this is worth a look.