feat(metrics): Add prefill KV compute metric excluding cached tokens

[ziliangpeng]

Summary

This PR adds a new metric vllm:request_prefill_kv_computed_tokens that tracks the number of KV tokens computed during prefill phase, excluding cached tokens.

Motivation

Currently, vLLM tracks total prompt tokens (vllm:request_prompt_tokens) but doesn't have per-request visibility into how many KV tokens were actually computed vs served from cache (local prefix cache or remote KV cache like LMCache). This metric helps:

• Understand cache effectiveness on a per-request basis
• Better estimate actual compute costs vs total prompt size
• Debug and optimize caching strategies
• Monitor workload characteristics more accurately

Changes

• Added num_cached_tokens field to FinishedRequestStats dataclass
• Updated update_from_finished_request() to accept num_cached_tokens parameter
• Added new histogram metric vllm:request_prefill_kv_computed_tokens in metrics loggers
• Metric calculation: num_prompt_tokens - max(num_cached_tokens, 0)
• Added comprehensive unit tests

Testing

• Added unit tests in tests/v1/metrics/test_stats.py:
  • Test with prefix cache hits
  • Test without cache
  • Test edge cases (negative values, all tokens cached)
• Verified in production workloads showing expected cache effectiveness

The metric correctly includes cache hits from both local prefix cache and remote KV stores (KV connector, LMCache).

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 noreply@anthropic.com

[gemini-code-assist]

Code Review

This pull request introduces a new metric, vllm:request_prefill_kv_computed_tokens, to track the number of KV tokens computed during the prefill phase, excluding any tokens served from the cache. The changes are well-implemented, adding the num_cached_tokens field to FinishedRequestStats and plumbing it through from the output processor. A new histogram is added to the Prometheus logger to record this metric, correctly calculating it as the difference between prompt tokens and cached tokens. The inclusion of comprehensive unit tests covering various scenarios, including edge cases, ensures the reliability of this new feature. The code is clear, follows existing patterns, and improves the observability of cache effectiveness. Overall, this is a solid contribution.

[mergify]

Hi @ziliangpeng, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.