Skip to main content
Since 2.0 the SDK is a thin client. You install r3alai locally, but the quantization itself runs on the R3AL platform (GPU infrastructure at platform.r3al.ai). Your model is uploaded, the job runs on our hardware, and the quantized model is downloaded back to you.
The free plan includes 3 runs. Each quantize or QAT job consumes one run; local benchmark() and export are free. A fourth run raises PlanLimitError (HTTP 402); upgrade your plan on the platform.

​
1. Create an account and an API key

1

Sign up

Create an account at platform.r3al.ai and verify your email.
2

Mint a key

Open the SDK / API keys page and create a key. It looks like r3l_live_.... Copy it now: the secret is shown only once.

​
2. Install

The vision extra pulls in onnxruntime and pillow so you can run and benchmark the quantized model locally. See Installation for other extras.

​
3. Authenticate

Point the SDK at your key, either in code or through the environment:
Or store it once with the CLI, which verifies it before writing it 0600 — after that neither the SDK nor the CLI needs an environment variable:

​
4. Quantize on the platform

The default method is ptq_static: the best size and latency wins across vision models, including convolution-heavy backbones. It needs a few representative calibration samples (sample inputs drawn from your own data). One call uploads your model and calibration data, runs the job on R3AL GPUs, streams live progress, and downloads the quantized model into output_dir.
No calibration data on hand? Use method="ptq_dynamic", which needs none. It only quantizes MatMul and Gemm (fully connected) layers, so it suits transformer or MatMul-heavy models and is a poor fit for convolution-heavy vision backbones (it would find nothing to quantize). See Pick the right method.
Only have framework weights? Pass the checkpoint and the SDK exports it to ONNX locally before upload: Quantizer(QuantConfig(method="ptq_static")).quantize("checkpoint.pt", source="pytorch", input_shape=[1, 3, 224, 224], calibration_data=["img1.jpg", "img2.jpg"], output_dir="./out"). Local export needs the matching extra, for example r3alai[export-torch].

​
5. Watch the job live

While quantize() blocks, it prints progress on your terminal. You can also follow the same job on the platform’s Jobs page, which shows its status and progress bar in real time and keeps a record of every run.

​
6. Measure it locally

Latency numbers only matter on the hardware you deploy to, so benchmark() runs on your machine, not the platform:
The benchmark runs on CPU or GPU, prints live progress, and reports the hardware it ran on. For more details and information on method options, see Benchmarking.

​
Calibration and the no-data alternative

ptq_static (the default) gives the smallest file and the best latency win, including on convolution-heavy backbones. The trade-off is that it needs representative calibration samples, and badly chosen samples cost accuracy. A small accuracy drop versus full precision is normal even with good calibration, so always validate. If you have no calibration data, ptq_dynamic needs none:
It only quantizes MatMul and Gemm layers, so it fits transformer or MatMul-heavy models. On a convolution-heavy vision model it finds nothing to quantize and reports an unsupported config, so use ptq_static with calibration data instead. Read Integer arithmetic for the dynamic-vs-static trade-off, and Calibration for building a good calibration set.

​
Next steps

Using the platform

Accounts, API keys, the Jobs page, and your plan.

Pick the right method

Dynamic vs static INT8 or INT4, and when to use QAT.

QAT explained

Recover accuracy with quantization-aware training.

End-to-end workflow

Export, quantize on the platform, validate locally.