> ## Documentation Index
> Fetch the complete documentation index at: https://docs.r3al.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Calibration

> When and why calibration images are needed.

By default, standard preprocessing is applied: the image is decoded as RGB, resized to the graph's input height and width, scaled to \[0,1], and transposed from HWC to CHW. Channel order is **not** changed, so a model trained on BGR input (the OpenCV default) needs its channels swapped before it reaches us. For custom
preprocessing, provide input arrays with the correct dimensions instead. For an example, see [Providing calibration data](#providing-calibration-data).

Calibration images tell the engine what inputs your model typically sees, so it can produce a well-tuned quantized model. Two paradigms use them, for different reasons.

## Which methods need calibration

| Method            | Calibration | Why                                                                                                       |
| ----------------- | ----------- | --------------------------------------------------------------------------------------------------------- |
| PTQ `ptq_dynamic` | No          | No representative data required                                                                           |
| PTQ `ptq_static`  | **Yes**     | The engine calibrates on your samples, then produces the quantized model; faster inference                |
| QAT               | **Yes**     | Calibrates and also trains the model on your images; faster inference, further decrease the accuracy drop |

## PTQ static INT8 or INT4: measurement only

Static INT8 or INT4 needs to know the range of values your activations take, which depends on the inputs the model sees. You provide representative images; the engine calibrates on your samples, then produces the quantized model.

<Note>
  **PTQ calibration is not training.** It is a set of forward passes that complete quickly, with no gradient updates.
</Note>

```python theme={null}
from r3alai.quant import QuantConfig, Quantizer

result = Quantizer(QuantConfig(method="ptq_static")).quantize(
    "model.onnx",
    calibration_data=[
        "/data/images/sample1.jpg",
        "/data/images/sample2.jpg",
    ],
    max_calib_samples=100,
)
```

## Choosing the clipping threshold

Calibration measures the range of values each activation takes, and INT8 or INT4 spends its 256 levels on that range. The question every calibration method answers is where the range should stop: values beyond the threshold are clipped, values inside it get finer resolution. One rare outlier batch can stretch the range far past where the real signal lives, and every level spent covering that gap is a level not spent on the values your model actually sees.

`calibration_method` picks how the threshold is chosen. It applies to `ptq_static` only.

| `calibration_method` | Threshold                                                                                                             | Cost                              | Use when                                                               |
| -------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ---------------------------------------------------------------------- |
| `minmax` (default)   | The largest absolute value observed. Nothing is clipped                                                               | Cheapest                          | The activations have no long tails, or you have few calibration images |
| `percentile`         | The `calibration_percentile` percentile of the observed values (99.99 or 99.999)                                      | One extra pass over the histogram | Activations have outliers you are willing to clip                      |
| `entropy`            | The threshold whose quantized distribution loses the least information (minimum KL divergence) against full precision | Slowest, seconds per model        | You want the threshold picked for you rather than tuned by hand        |

```python theme={null}
from r3alai.quant import QuantConfig, Quantizer

config = QuantConfig(
    method="ptq_static",
    calibration_method="percentile",
    calibration_percentile=99.99,
)

result = Quantizer(config).quantize(
    "model.onnx",
    calibration_data=image_paths,
    max_calib_samples=256,
    output_dir="./out",
)
```

<Note>
  `calibration_percentile` is a percentile in percent (99.99), not a fraction (0.9999). Passing a fraction raises `UnsupportedConfigError` instead of silently clipping away most of your distribution. The QAT equivalent, `qat_calib_q`, is a fraction.
</Note>

Entropy calibration follows the scheme TensorRT uses: histogram the observed magnitudes into `calibration_num_bins` bins (2048 by default), then score each candidate threshold by the KL divergence between the full-precision distribution and what INT8 or INT4 can represent below it. It usually lands close to a well-chosen percentile, which is why `percentile` is the cheaper way to the same place when you know your data.

`calibration_symmetric` forces the range to be symmetric around zero. Leave it off unless you know you need it: activations are quantized to unsigned INT8 or INT4 with a zero point, so a one-sided range (a post-ReLU tensor spanning `[0, alpha]`) keeps all 256 or 16 levels, while a symmetric range spends half of them below zero where the tensor never goes.

Which method was used, with its parameters and the number of samples it saw, is recorded in the deliverable's manifest.

### How many images

All three methods need representative data, and the histogram methods need the most: `percentile` and `entropy` estimate a whole distribution per tensor rather than a running maximum. Aim for 256 to 512 diverse samples for those, and note that `max_calib_samples` caps how many are used (it defaults to 100). The SDK warns if a histogram method is calibrating on fewer than 256 samples.

## QAT: calibration + training

QAT uses the same `calibration_data`, but the engine both calibrates on it and trains the model on it over `epochs`:

```python theme={null}
from r3alai.quant import QuantConfig, Quantizer

result = Quantizer(QuantConfig(mode="qat")).train_qat(
    "model.onnx",
    calibration_data=["/data/images/img1.jpg", "/data/images/img2.jpg"],
    epochs=3,
)
```

Calibration sets each layer's initial clip threshold before training starts, and training then tunes those thresholds along with the weights. The same three options are available through `qat_calib_method`: `percentile` (the default, at quantile `qat_calib_q`), `max`, and `entropy` (over a `qat_calib_num_bins` histogram).

```python theme={null}
config = QuantConfig(mode="qat", qat_calib_method="entropy", qat_calib_batches=32)
```

<Warning>
  For QAT, calibration images also serve as training data. Use more images and more epochs than a PTQ smoke test.
</Warning>

## Providing calibration data <a id="providing-calibration-data" />

Pass image file paths (the SDK uploads them with your model) or preprocessed arrays:

```python theme={null}
import numpy as np
from r3alai.quant import QuantConfig, Quantizer

calib = [preprocess(p) for p in image_paths]

Quantizer(QuantConfig(method="ptq_static")).quantize(
    "model.onnx",
    calibration_data=calib,
)
```

## Building a good calibration set <a id="building-a-good-calibration-set" />

<Steps>
  <Step title="Match production preprocessing">
    Same resolution, normalization, and letterboxing as your inference pipeline.
  </Step>

  <Step title="Cover the deployment distribution">
    256-1024 images spanning lighting, object sizes, scene types, and edge cases. Fewer works for `minmax`, but the histogram methods get noisy below roughly 256.
  </Step>

  <Step title="Validate afterwards">
    Compare quantized vs original outputs. See [Benchmarking](/guides/benchmarking). Especially
  </Step>
</Steps>