Introduction

r3alai is the R3AL.AI vision quantization SDK. As of 2.0 it is a thin client: you install r3alai locally, but your model runs on R3AL’s optimization infrastructure (GPU, at platform.r3al.ai). The SDK uploads your model, runs the job on our hardware, streams progress, and downloads the quantized deliverable back to you.

your_model.onnx  →  upload  →  R3AL platform (GPU)  →  download  →  quantized ONNX + manifest

No vendor lock-in on the output: the deliverable is a standard ONNX file plus a manifest. Classification, detection, segmentation, pose: if it is a vision model you can export to ONNX, it works.

Platform + SDK

The SDK (local)

pip install r3alai. Authenticate, submit jobs, and run local benchmarks and inference on downloaded bundles.

The platform (remote)

Your account, API keys, and the Jobs page where every quantization runs and is tracked.

quantize() and train_qat() run on the platform (and consume plan quota). benchmark() and load() run locally, on your own hardware, which is exactly where latency numbers matter. The free plan includes 3 runs.

Two paradigms

PTQ: Post-Training Quantization

No training. Model plus calibration data in, quantized model out. Two methods: ptq_static (default) and ptq_dynamic.

QAT: Quantization-Aware Training

A short fine-tuning pass that recovers accuracy at low bit widths. Model plus a small training set in, quantized model out.

Methods at a glance

Method	What it needs	What you get	Trade-off
`ptq_static` (PTQ, default)	Representative calibration images	Smallest file, best latency, covers convolution-heavy backbones	Poorly chosen images cost accuracy; always validate
`ptq_dynamic` (PTQ)	Nothing extra, no calibration data	No-data path, smaller file	Quantizes MatMul and Gemm layers only, so it suits MatMul-heavy models, not convolution-heavy backbones
QAT	Calibration/training images + GPU time	Recovers accuracy PTQ loses, especially at low bits (3-4 bit)	Costs a training run; needs representative images

Start with ptq_static, the default: it gives the biggest size and speed wins across vision models and needs only a few representative calibration samples. Use ptq_dynamic when you have no calibration data and a MatMul-heavy model. Reach for QAT only when PTQ accuracy is not good enough, especially at aggressive bit widths. Always benchmark on your model before shipping.

One PTQ call

import r3alai.platform as platform
from r3alai.quant import Quantizer

platform.login("r3l_live_...")   # or set R3AL_API_KEY

result = Quantizer().quantize(
    "your_model.onnx",
    calibration_data=["img1.jpg", "img2.jpg", "img3.jpg"],  # a handful of representative inputs
    output_dir="./out",
)
print(result.path)

The SDK uploads the model and calibration data, the platform quantizes it with the default method (ptq_static), and the deliverable is downloaded next to its manifest.

One QAT call

from r3alai.quant import QuantConfig, Quantizer

result = Quantizer(QuantConfig(mode="qat")).train_qat(
    "your_model.onnx",
    calibration_data=["img1.jpg", "img2.jpg"],
    output_dir="./qat_out",
    epochs=1,
)
print(result.path)

QAT is a separate paradigm, not a PTQ method: it fine-tunes the model before quantizing.

How it fits together

Authenticate

Create an API key on the platform, then platform.login(...) or set R3AL_API_KEY.

Export to ONNX (if needed)

Already have .onnx? Skip ahead. Otherwise pass a checkpoint and the SDK exports it locally before upload (source=, input_shape=).

Choose PTQ or QAT

PTQ for speed and simplicity. QAT when you need better accuracy after aggressive quantization.

Quantize on the platform

One call uploads, runs the job on R3AL GPUs, and downloads the result. Watch it live on the Jobs page.

Validate locally

Benchmark latency, size, and output fidelity on your own hardware before you ship.

Get Started

Core Concepts

Guides

Platform + SDK

The SDK (local)

The platform (remote)

Two paradigms

PTQ: Post-Training Quantization

QAT: Quantization-Aware Training

Methods at a glance

One PTQ call

One QAT call

How it fits together

​Platform + SDK

The SDK (local)

The platform (remote)

​Two paradigms

PTQ: Post-Training Quantization

QAT: Quantization-Aware Training

​Methods at a glance

​One PTQ call

​One QAT call

​How it fits together

Platform + SDK

Two paradigms

Methods at a glance

One PTQ call

One QAT call

How it fits together