Description
Return an exact unpadded key/value-cache byte count.
Inference budgets depend on the number of stored values, sequence lengths, routing assignments and the chosen accounting basis. Logical cache bytes and padded token counts differ from actual device allocation, latency and billed usage. Caller-supplied prices, cache fractions and capacity assumptions must match the intended system.
Return an exact unpadded key/value-cache byte count.
When to use this tool
Use Artificial Intelligence: Kv Cache Token Byte Count when reconciling the stated cache, padding, routing or price basis.
How to use the tool and how it works
- Enter tokens, layers, kvHeads, headDimension, bytesPerValue. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read result. Expand result to inspect the returned fields.
Return an exact unpadded key/value-cache byte count. Require whole tokens 0..1e9 and layers/kvHeads/headDimension/bytesPerValue 1..1e6; cacheBytes = 2*tokens*layers*heads*dimension*bytes, rendered as decimal text.1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| tokens | Tokens | Required. Safe integer. Calculation range: 0 through 1000000000, inclusive. |
| layers | Layers | Required. Safe integer. |
| kvHeads | Kv Heads | Required. Safe integer. |
| headDimension | Head Dimension | Required. Safe integer. |
| bytesPerValue | Bytes Per Value | Required. Safe integer. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| result | Object containing the returned operation-specific members listed below. | Required. Object. |
| result.cacheBytes | Exact decimal-integer text: 2*tokens*layers*kvHeads*headDimension*bytesPerValue; factor two counts keys and values. | Calculated result member: string. |
{
"input": {
"tokens": 10,
"layers": 2,
"kvHeads": 4,
"headDimension": 8,
"bytesPerValue": 2
},
"output": {
"result": {
"cacheBytes": "2560"
}
}
}Limitations and assumptions
- The factor two counts keys and values; this excludes batch multiplicity unless already represented, padding, metadata, scales and runtime allocation overhead.
- Outputs are nested inside result. Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use actual model/runtime layouts, allocator measurements and current provider billing records for real memory, performance or cost planning.
References
Similar or alternative tools
- Kv Cache Head Padding Budget
Return useful and padding cache bytes.
- Inference Batch Sequence Waste Term
Return ragged-batch token totals and padding waste.