Public Tools · AI Infrastructure

AI Model Fit

Estimate whether an inference model is likely to fit within a given accelerator or unified-memory budget. Planning aid only. Not a benchmark, vendor commitment, or compatibility guarantee.

Model weights

Parameters (billions)

Precision (bits/param)

Workload

Context length (tokens)

Concurrent sequences

KV cache architecture (optional for weights-only; required for full fit)

Transformer layers

KV heads

Head dimension

KV bytes / element

Target system

Target memory (GiB)

Runtime overhead (%)

Desired headroom (%)

How this is calculated

Methodology

Weight bytes = parameterCountBillions * 1,000,000,000 * bitsPerParameter / 8.
Weight GiB = weight bytes / 2^30.
When full KV architecture is supplied: KV bytes/token/sequence = 2 * layers * kvHeads * headDimension * bytesPerElement (2 accounts for K + V).
Total KV GiB = (bytes/token/sequence) * contextLength * concurrency / 2^30.
Base memory = weightGiB + kvMemoryTotalGiB. If KV inputs are missing, KV = 0 and status is incomplete.
Runtime overhead = baseMemoryGiB * runtimeOverheadFraction. Estimated total = base + overhead.
Classification: incomplete (KV architecture missing); no-fit (estimatedTotal > target); marginal (estimatedTotal > safeMemoryCeiling); likely-fit (else).

This is a vendor-neutral planning aid. Actual runtime memory varies with allocator behavior, mixed-precision tensors, activation/workspace buffers, runtime reservations, OS/shared-memory on unified-memory systems, and framework/backend implementation. It does not model CPU offload, tensor parallelism, pipeline parallelism, or heterogeneous memory placement.