Integer arithmetic

Both PTQ methods produce true INT8 or INT4 models: weights stored as 8-bit integers, and INT8 or INT4 execution at inference time. The difference is how activations are handled, and that drives the trade-off between them.

ptq_static (default)

Weights and activations are quantized ahead of time, using representative calibration images.

from r3alai.quant import Quantizer

# ptq_static is the default method; it needs a few representative calibration samples
result = Quantizer().quantize(
    "model.onnx",
    calibration_data=["img1.jpg", "img2.jpg", "img3.jpg"],  # a handful of representative inputs
    output_dir="./out",
    max_calib_samples=100,
)
print(result.path)

Pro: the smallest file and the best inference speed, including on convolution-heavy backbones.
Pro: the default and the production path when calibration images match your deployment distribution.
Con: needs representative calibration images; badly chosen images cost accuracy.

ptq_dynamic

The no-data path. No calibration images, no setup.

from r3alai.quant import QuantConfig, Quantizer

result = Quantizer(QuantConfig(method="ptq_dynamic")).quantize("model.onnx", output_dir="./out")
print(result.path)

Pro: no calibration data needed, zero setup, and a smaller file.
Pro: fits transformer or MatMul-heavy models, since it quantizes MatMul and Gemm (fully connected) layers.
Con: on a convolution-heavy vision backbone it finds nothing to quantize and reports an unsupported config, so reach for ptq_static with calibration data there.

QAT (when PTQ is not enough)

When PTQ loses too much accuracy, especially at low bit widths, quantization-aware training recovers it by fine-tuning the model with quantization simulated in the loop. It costs a training run and needs representative images. See QAT explained.

Expected improvements

Typical outcomes when static INT8 or INT4 fits your model and runtime:

Metric	Typical change vs float ONNX
File size	~3-4x smaller
CPU latency	often 1.2-2x faster (model-dependent)
Output fidelity	cosine similarity often >= 0.999

Always benchmark on your model. Speedups depend on architecture, input resolution, and whether you run on CPU or GPU.

Which one?

Have representative sample images?  →  ptq_static   (default: smallest, fastest)
No calibration data (MatMul-heavy)? →  ptq_dynamic  (no data)
Accuracy too low after PTQ?         →  QAT

Get Started

Core Concepts

Guides

Integer arithmetic

ptq_static (default)

ptq_dynamic

QAT (when PTQ is not enough)

Expected improvements

Which one?

​ptq_static (default)

​ptq_dynamic

​QAT (when PTQ is not enough)

​Expected improvements

​Which one?

ptq_static (default)

ptq_dynamic

QAT (when PTQ is not enough)

Expected improvements

Which one?