QAT explained

Quantization-Aware Training (QAT) fine-tunes a model with quantization simulated during training. The model learns to cope with lower precision, recovering accuracy that post-training quantization (PTQ) often loses, especially at low bit widths. Unlike PTQ, QAT does train, so it costs a training run on GPU and needs representative images. The training runs on the R3AL platform; the SDK uploads your model and images and downloads the quantized result.

QAT uses MSE on the model output to update weights and quantization thresholds to restore accuracy.

Minimal call

from r3alai.quant import QuantConfig, Quantizer

result = Quantizer(QuantConfig(mode="qat")).train_qat(
    "model.onnx",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./qat_out",
)
print(result.path)

Defaults: epochs=1, qat_wbit=8, qat_abit=8.

What happens on the platform

QAT is a black box you drive with a few parameters. End to end:

You provide the model and images

Upload your ONNX model (or a checkpoint the SDK exports first) and a representative set of calibration images.

The engine fine-tunes with quantization in the loop

R3AL’s engine trains the model on GPU with lower precision simulated during the forward pass, and automatically protects accuracy-sensitive layers.

You get a standard quantized ONNX bundle

The engine exports a standard quantized ONNX model plus r3alai_manifest.json, downloaded into your output_dir.

Key fields

`epochs`

How many times the model trains over your calibration data.

Value	Use case
`1`	Smoke test to verify the pipeline works
`3-10`	Typical production fine-tuning
`10+`	Aggressive low-bit quantization

`qat_wbit`: weight bits

Precision of the model’s weights.

8: safe default
4: aggressive, roughly 4x smaller weights, needs QAT to recover accuracy
3: very aggressive, only with QAT and enough epochs

Set it on the config: QuantConfig(qat_wbit=...).

`qat_abit`: activation bits

Precision of the intermediate tensors (layer outputs). Activations typically have different value ranges than weights, so you can set qat_abit independently:

qat_wbit=4, qat_abit=8: aggressive weight compression, safer activations
qat_wbit=8, qat_abit=8: conservative, minimal accuracy risk (recommended starting point)

Set it on the config: QuantConfig(qat_abit=...).

Full example

from r3alai.quant import QuantConfig, Quantizer

config = QuantConfig(
    mode="qat",
    qat_wbit=8,
    qat_abit=8,
    qat_calib_batches=16,
)
result = Quantizer(config).train_qat(
    "model.onnx",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./qat_out",
    epochs=3,
    learning_rate=0.0001,
)
print(result.path)

Calibration images for QAT

QAT uses calibration_data for both calibration and training, unlike PTQ where calibration is only a measurement pass. Calibration runs first and sets each layer’s starting clip threshold; training then tunes those thresholds together with the weights. qat_calib_method chooses how the starting value is derived: percentile (the default, at quantile qat_calib_q), max, or entropy.

config = QuantConfig(
    mode="qat",
    qat_calib_method="percentile",   # or "max", "entropy"
    qat_calib_q=0.9999,              # a fraction, unlike the PTQ calibration_percentile
    qat_calib_batches=32,
)

Use images that match your production distribution. Random unrelated photos teach the model the wrong ranges and hurt accuracy.

See Calibration for building a good image set.

Get Started

Core Concepts

Guides

Minimal call

What happens on the platform

Key fields

`epochs`

`qat_wbit`: weight bits

`qat_abit`: activation bits

Full example

Calibration images for QAT

​Minimal call

​What happens on the platform

​Key fields

​epochs

​qat_wbit: weight bits

​qat_abit: activation bits

​Full example

​Calibration images for QAT

Minimal call

What happens on the platform

Key fields

`epochs`

`qat_wbit`: weight bits

`qat_abit`: activation bits

Full example

Calibration images for QAT