> ## Documentation Index > Fetch the complete documentation index at: https://docs.r3al.ai/llms.txt > Use this file to discover all available pages before exploring further. # Python SDK > Quantizer, QuantConfig, PlatformClient, Job, and authentication for r3alai 2.0. The 2.0 SDK is a thin client. `Quantizer` is the high-level facade most users need; `PlatformClient` and `Job` are the lower-level primitives it is built on. `quantize()` and `train_qat()` run on the R3AL platform and consume plan quota. `benchmark()` and `load()` run **locally**, on your own hardware. ## Authenticate ```python theme={null} import r3alai.platform as platform platform.login("r3l_live_...") # set the key for this process ``` Or set `R3AL_API_KEY` in the environment. To override the platform host: ```python theme={null} platform.configure(base_url="https://platform.r3al.ai", api_key="r3l_live_...") ``` Resolution order for the key: an explicit argument, then `login(...)`/`configure(...)`, then `R3AL_API_KEY`, then a key stored by [`r3al login`](/cli/commands) at `~/.config/r3al/credentials.json`. The environment deliberately beats the stored file, so exporting a key for one process still wins. ## Quantizer The facade for the common flow: upload, run on the platform, download. ```python theme={null} from r3alai.quant import QuantConfig, Quantizer # PTQ (ptq_static is the default method; it needs calibration data) result = Quantizer().quantize( "model.onnx", calibration_data=["img1.jpg", "img2.jpg", "img3.jpg"], # a handful of representative inputs output_dir="./out", ) # QAT qat = Quantizer(QuantConfig(mode="qat")) result = qat.train_qat( "model.onnx", calibration_data=["img1.jpg", "img2.jpg"], output_dir="./qat_out", epochs=3, ) ``` ### Quantizer(config) Configuration shared by every call on this instance. Omit it for the defaults: `mode="ptq"`, `method="ptq_static"`, `bits=8`, `calibration_method="minmax"`. Invalid combinations raise `UnsupportedConfigError` at construction time. Methods: [`quantize()`](#quantize-runs-on-the-platform), [`train_qat()`](#train-qat-runs-on-the-platform), [`benchmark()`](#benchmark-local), [`load()`](#load-local). ### QuantConfig The Pydantic config shared by PTQ and QAT. Validate one without a network call using `validate_config_local`. #### Shared The only supported value. Anything else raises `UnsupportedConfigError`. `ptq` (post-training, no training loop) or `qat` (quantization-aware training). `mode="ptq"` is required by `quantize()`; `mode="qat"` is required by `train_qat()`. PTQ method, ignored when `mode="qat"`: `ptq_static` (best accuracy, needs `calibration_data`) or `ptq_dynamic` (no calibration, but only quantizes MatMul/Gemm layers — a poor fit for Conv-heavy backbones). See [Methods](/concepts/methods). Bit width for PTQ: `8` (INT8) or `4` (INT4). Any other value raises `UnsupportedBitsError`. INT4 halves the model again but has far less headroom for calibration error — pair it with `calibration_method="percentile"` or `"entropy"` and a larger sample set, and check accuracy with [`benchmark()`](#benchmark-local) before shipping. In `mode="qat"` this field is ignored and overwritten with `qat_wbit`, which accepts 3-8. Execution device for the engine: `auto`, `cuda`, or `cpu`. Device string hint, e.g. `cuda:0`. Deliverable format. ONNX is the only supported value. #### PTQ calibration (`method="ptq_static"`) How the activation clipping threshold is chosen: `minmax` (observed absolute max, nothing clipped), `percentile` (clip at `calibration_percentile`), or `entropy` (KL-divergence search over a histogram, the TensorRT scheme). Aliases: `max`/`min_max`/`min-max` → `minmax`, `kl`/`kl_divergence` → `entropy`. Setting anything but `minmax` with `method="ptq_dynamic"` raises `UnsupportedConfigError` — dynamic quantization never runs a calibration pass. See [Calibration](/concepts/calibration). Clipping percentile for `calibration_method="percentile"`, **in percent** and in the range (50, 100]. Passing a fraction (`0.9999`) is rejected with a message telling you the percent form. Histogram bins over the observed absolute-value range, used by `percentile` and `entropy`. Range \[128, 8192]. More bins means a finer threshold search and a slower calibration pass. The `entropy` search then merges candidates down to `2^(bits-1)` magnitude levels — 128 at INT8, 8 at INT4 — so `bits` changes which threshold wins, not just the output grid. Force the calibrated activation range symmetric around 0. Off by default so a one-sided range (e.g. post-ReLU `[0, alpha]`) keeps the whole grid — 256 levels at INT8, 16 at INT4 — instead of spending half of it below zero. Applies to every calibration method. `percentile` and `entropy` fit a histogram per activation tensor. `quantize()` emits a Python warning when either is calibrating on fewer than 256 samples (counting the `max_calib_samples` cap); with fewer, `minmax` is the safer choice. #### QAT (`mode="qat"`) Weight bit width, 3-8. Also becomes the config's `bits`. Activation bit width, 3-8. Bit width for the stem convolution. `None` leaves the first layer in float32, which is the usual mitigation when low-bit QAT loses accuracy at the input. Whether activations are fake-quantized during training. `False` trains weight-only QAT. Default training epochs (minimum 1). `train_qat(epochs=...)` overrides it. `1` is a smoke test, not a converged model. Batches used to initialize the activation clip thresholds (`act_alpha`) before training. Minimum 1. Also bounds the training set: the pipeline trains on at most `epochs × qat_calib_batches` batches. How each activation clip threshold is initialized: `percentile` (quantile `qat_calib_q` of observed absolute values), `max` (observed absolute max), or `entropy` (KL-divergence search over a `qat_calib_num_bins` histogram). Aliases: `minmax`/`min_max`/`min-max` → `max`, `kl`/`kl_divergence` → `entropy`. Quantile for `qat_calib_method="percentile"`, as a **fraction** in (0, 1] — `0.9999` is the 99.99th percentile. Note this is the opposite convention to the PTQ field `calibration_percentile`, which is in percent. Histogram bins for `qat_calib_method="entropy"`, range \[128, 8192]. Ignored by the other QAT calibration methods, and only sent with the job when the method is `entropy`. #### Derived properties `True` for `ptq_static` and for every QAT config. The inverse of `requires_calibration`. `True` for both PTQ methods (the engine always quantizes ONNX). ### quantize() (runs on the platform) Uploads the model (exporting it to ONNX locally first when needed), runs the job on R3AL infrastructure, waits with live progress, and downloads the deliverable bundle into `output_dir`. ```python theme={null} result = Quantizer().quantize( model="model.onnx", # .onnx, a checkpoint, or an nn.Module calibration_data=["img1.jpg", "img2.jpg", "img3.jpg"], # required for ptq_static (the default) output_dir="./out", # bundle is downloaded here verbose=True, # print live job progress (default) domain="object-detection", # optional manifest tag max_calib_samples=256, # engine option ) result.path # local path to the downloaded bundle ``` #### Signature An `.onnx` path, a checkpoint in any format the export registry understands (`.pt`/`.pth`, `.h5`/`.keras`, `.pb`, `.tflite`, Paddle, SavedModel directory), an already-loaded `torch.nn.Module`, or an existing `r2://` reference. Everything non-ONNX is exported to ONNX **locally** before upload, so it needs the export options below. Local image file paths (or `r2://` refs), uploaded alongside the model. Required for `method="ptq_static"`; omitting it raises `CalibrationRequiredError` locally, before anything is uploaded or a run is spent. Ignored by `ptq_dynamic`. Local directory the deliverable bundle is downloaded and unpacked into. Print live job progress (status, percentage, elapsed) on stderr while waiting. #### Export options (non-ONNX input only) Consumed by the local export step; ignored when `model` is already an `.onnx` file. Force an export adapter instead of auto-detecting by suffix: `onnx`, `ultralytics` (alias `yolo`), `pytorch` (alias `torch`), `tensorflow` (aliases `tf`, `keras`), `paddle` (alias `paddlepaddle`), `tflite`. Required to disambiguate `.pt`/`.pth`, which both YOLO and plain PyTorch use. `list_export_sources()` returns the live list. Base filename for the exported `.onnx`. Required when `model` is an in-memory `nn.Module` (there is no source path to derive a name from); otherwise defaults to the checkpoint's stem. Example input shape used to trace the model, e.g. `[1, 3, 224, 224]`. Required by the PyTorch export path. ONNX opset for the export. The engine upgrades anything below 13 before quantizing, since per-channel QDQ needs opset ≥ 13. Image size for exporters that take one (Ultralytics YOLO). Run the exporter's graph simplification pass where it offers one. Export with a dynamic batch axis instead of a fixed one. Names for the exported graph inputs. Names for the exported graph outputs. Escape hatch: extra adapter-specific options. Recognised keys (`input_shape`, `opset`, `imgsz`, `simplify`, `dynamic_batch`, `input_names`, `output_names`) are read out of it, and anything left over is forwarded to the adapter as-is. Local directory the intermediate `.onnx` (and its `r3al_export_manifest.json`) is written to before upload. #### Engine options Forwarded verbatim to the quantization job. Free-form tag (e.g. `"object-detection"`) recorded in the deliverable's manifest and stamped into the ONNX metadata as `r3al_domain`. Cap on how many calibration images are actually used. Histogram methods want 256-1024; raise this when you pass more than 100 images with `percentile` or `entropy`. Graph input to feed calibration data into. Defaults to the graph's first input, which is correct for every single-input vision model. Per-channel weight scales (one per output channel) instead of one scale per tensor. Silently downgraded to per-tensor when the model's opset cannot be upgraded to 13. How many hops back from the graph outputs stay in float32 during `ptq_static`. Detection/pose heads end in numerically fragile ops (Sigmoid, multi-scale Concat) where int8 can collapse every confidence to zero; leaving the last few layers unquantized costs almost nothing in size or speed. Set `0` to quantize the head too. Keep `Softmax`, `LayerNormalization`, `Gelu` and `Erf` — plus the MatMul/Gemm nodes directly feeding or consuming a Softmax — in float32. This is the standard transformer PTQ do-not-quantize list; it is a no-op for Conv-only backbones. Applies to both `ptq_static` and `ptq_dynamic`. Explicit list of node names to leave in float32. **Replaces** both automatic exclusion lists above, so pass it only when you want full manual control. Also reconstruct a `.pt` checkpoint from the quantized graph via onnx2torch. Best-effort: quantized ops are not all supported, so a failure is recorded in the manifest as `native_export_error` instead of failing the job. The `.onnx` deliverable is always the primary contract. `calibration_method`, `calibration_percentile`, `calibration_num_bins` and `calibration_symmetric` are **rejected** as `quantize()` kwargs with `UnsupportedConfigError`. Set them on `QuantConfig` so they are validated and recorded in the deliverable's manifest. #### Returns Handle to the downloaded bundle. See [QuantizedModel](#quantizedmodel). ### train\_qat() (runs on the platform) Fine-tunes the model with quantization simulated in the loop on R3AL GPU infrastructure, then downloads the deliverable. Requires `mode="qat"`; anything else raises `UnsupportedConfigError`. ```python theme={null} config = QuantConfig(mode="qat", qat_wbit=8, qat_abit=8) result = Quantizer(config).train_qat( "model.onnx", calibration_data=["img1.jpg", "img2.jpg"], validation_data=["held1.jpg", "held2.jpg"], # never trained on, only measured output_dir="./qat_out", epochs=3, learning_rate=1e-4, ptq_runtime=True, # also emit a real int8 deliverable ) result.path # downloaded bundle (contains model_qat.quantized.onnx) ``` #### Signature Same inputs as `quantize()`. Non-ONNX models are exported to ONNX locally before upload, so the [export options](#export-options-non-onnx-input-only) apply here too. Training/calibration data. Keyword-only and always required — `None` raises `UnsupportedConfigError`. Used twice: once to initialize activation ranges, once as the training set. Accepts image files (JPG, PNG, BMP, GIF, TIFF, WebP, PPM), which are resized to the model's input, or `.npy`/`.npz` arrays already shaped like the model input, which are used as-is. Unreadable entries are skipped and reported in the job's `steps`; the run only fails if *nothing* was readable. Local directory the deliverable bundle is downloaded into. Print live job progress while waiting. #### Training options Forwarded verbatim to the QAT pipeline. Training epochs. Falls back to `qat_epochs` (1) when omitted. A held-out set, never trained on, in the same formats as `calibration_data`. When given, the pipeline also reports validation fidelity before training, after every epoch, and at the end — which is what tells you whether QAT generalized or just memorized the training batches. If none of its entries are readable the run fails rather than silently training without validation. Images per training batch. A graph exported with a *fixed* batch axis overrides this — its baked-in batch size is used instead, and the run says so. Re-export with a dynamic batch axis to train in larger batches. Adam learning rate for the clip thresholds (`wgt_alpha`/`act_alpha`), the scalars that decide where the quantization grid sits. Also train the quantized layers' weights, not just the clip thresholds. Set `False` to tune thresholds alone. Universal ONNX QAT only — the native Ultralytics path always trains thresholds only. Learning rate for the weight parameter group when `train_weights=True`. Defaults to a tenth of `learning_rate`: the weights arrive already converged and only need nudging onto grid points, while a threshold is a single scalar that tolerates a larger step. Universal ONNX QAT only. Torch device for training, e.g. `cuda:0` or `cpu`. `auto` picks CUDA when available. Cap on calibration images. Also bounds training: the batch budget becomes `min(epochs × qat_calib_batches, max_calib_samples // batch_size)`. After training, run a real ONNX Runtime static-int8 pass over the QAT-trained weights and write it to `onnxruntime_int8/`. **Recommended.** By itself QAT folds the fake-quant values back into float32 Conv layers, so `*_qat.onnx` is the same size and speed as the original model; this option is what produces an actually smaller, actually int8 deliverable. Also write the trained weights back out as `{stem}_qat.pt`. For Ultralytics inputs this goes through `YOLO.save()`, so the result reloads as a normal YOLO checkpoint; otherwise it is a plain `nn.Module` state. Best-effort — a failure is recorded in `steps` instead of failing the run. Free-form manifest tag, same as `quantize()`. QAT bit widths and activation calibration come from the config and are forwarded automatically: `qat_wbit`, `qat_abit`, `qat_calib_method`, `qat_calib_q`, `qat_calib_batches`, plus `qat_calib_num_bins` when the method is `entropy`. `qat_first_layer_bit` and `qat_quant_act` are applied engine-side from the same config. `imgsz`, `simplify` and `opset` are claimed by the local export step (they are export options), so they are **not** forwarded to the training job. The QAT pipeline derives its image size from the uploaded ONNX graph's input shape. The pipeline's `progress_callback` hook is engine-internal — the platform wires it to the job's progress field, which is what `verbose=True` prints. It cannot be passed from the SDK. #### Returns Handle to the downloaded bundle: `model_qat.quantized.onnx`, `r3alai_manifest.json`, plus `onnxruntime_int8/` and `{stem}_qat.pt` when those options were enabled. With `verbose=True` the run also prints the fidelity MSE between the quantized model's outputs and the original's: once per epoch while training, then a summary of the before/after figures with a note on what they imply about the learning rate. The same numbers are recorded as `fidelity_mse` in the manifest and in the job result, so they survive the terminal scrollback. Without `validation_data` only the training-set figures can be reported, and a training-set MSE that improves says nothing about whether the improvement generalises — it can equally mean the run fitted those specific images. Pass held-out samples if you intend to act on the numbers. ### benchmark() (local) Runs on CPU or GPU (whichever ONNX Runtime provider is available), cycles through all `eval_data` samples, prints live progress, and supports Ctrl+C early stopping with partial results. Never touches the network. ```python theme={null} import numpy as np samples = [np.random.randn(1, 3, 640, 640).astype(np.float32) for _ in range(3)] bench = Quantizer().benchmark( original_model="model.onnx", quantized_model=result, # QuantizedModel or a bundle path eval_data=samples, warmup_runs=5, benchmark_runs=20, data_yaml="coco8.yaml", # optional: real mAP for both models imgsz=640, verbose=True, ) bench.latency_speedup bench.metadata["hardware"] # CPU/GPU, OS, architecture bench.metadata["early_stopped"] # True if interrupted with Ctrl+C ``` Path to the original `.onnx` model. A `QuantizedModel` handle, or a path to a downloaded bundle directory (or the `.onnx` next to its manifest). Image samples as NCHW float32 arrays or tensors. Every sample is cycled through in each run. Keyword-only. Custom quality metric, called once per model. Leave unset to measure latency only. Keyword-only. Name recorded for the quality metric in the result. Keyword-only. Untimed passes before measurement, to let the provider warm its kernels. Keyword-only. Timed passes per model. Keyword-only. YOLO-format labeled dataset config (`path`/`train`/`val`/`names`, plus `kpt_shape` for pose models). When given, real box (and pose) mAP50/mAP50-95 are computed for **both** models through Ultralytics' own validator. Requires the `yolo` extra; left `None` on the result if ultralytics is missing or validation fails — mAP is never fabricated. Keyword-only. Image size for the mAP validation pass, used only with `data_yaml`. Should match the size the model was exported/quantized at. Keyword-only. Print per-run progress (current sample, running average) and a final summary including hardware details. #### Returns: BenchmarkResult Mean inference latency of the original model, in milliseconds. Mean inference latency of the quantized model, in milliseconds. `latency_ms_original / latency_ms_quantized`. Peak VRAM for the original model, in MB. Peak VRAM for the quantized model, in MB. VRAM reduction, in percent. Name of the quality metric used. Quality score for the original model — set only when `quality_fn` was provided. Quality score for the quantized model. `quality_quantized - quality_original`. Detection mAP\@0.5 for the original model, via Ultralytics `val()`. Set only with `data_yaml`. Detection mAP\@0.5 for the quantized model. Detection mAP\@0.5:0.95 for the original model. Detection mAP\@0.5:0.95 for the quantized model. Pose mAP\@0.5 for the original model. Pose models only. Pose mAP\@0.5 for the quantized model. Pose mAP\@0.5:0.95 for the original model. Pose mAP\@0.5:0.95 for the quantized model. Run context: `original_model`, `hardware`, `provider_original`, `provider_quantized`, `num_samples`, `warmup_runs`, `benchmark_runs`, `runs_completed_original`, `runs_completed_quantized`, `early_stopped`. See [Benchmarking](/guides/benchmarking) for the full output reference. ### load() (local) ```python theme={null} bundle = Quantizer().load("./out") bundle["session"].run(None, {"images": batch}) ``` A downloaded bundle directory, or a quantized `.onnx` sitting next to its `r3alai_manifest.json`. Raises `ModelLoadError` if neither is found, or if the manifest says the model is not quantized. Takes no other options. Session built with the best available provider (CUDA when present, else CPU). Absolute path of the `.onnx` that was loaded. Parsed `r3alai_manifest.json` — method, bits, calibration settings, domain, excluded nodes. See [Manifest](/concepts/manifest). ### QuantizedModel Returned by `quantize()` and `train_qat()`. Bundle directory on local disk. The config the run was submitted with. Copies the bundle into `output_dir` (created if missing, merged if it exists) and returns that path. With no argument it is a no-op that returns `path` — the model is already saved when `quantize()` returns. Same as `Quantizer.load(self.path)`: `{"session": ort.InferenceSession, "path": str, "manifest": dict}`. Takes no options. ## PlatformClient The lower-level client, when you want direct control over upload, submission, and download. ```python theme={null} from r3alai.platform import PlatformClient client = PlatformClient() # key resolved from login()/configure()/env ``` Overrides `login()`/`configure()`/`R3AL_API_KEY` for this client. Keyword-only. Platform host override. Keyword-only. Per-request HTTP timeout. | Method | Returns | Notes | | -------------------------------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | | `upload_model(path)` | `str` | Uploads a file, returns an `r2://` reference. 5 GB single-upload limit | | `quantize(model, *, method=None, calibration_data=None, **kw)` | `Job` | Submits a PTQ job (async). Same export and engine options as `Quantizer.quantize()`; calibration knobs are **not** rejected here | | `qat(model, *, calibration_data, epochs=None, **kw)` | `Job` | Submits a QAT job (async). Same training options as `Quantizer.train_qat()` | | `get_job(id)` | `Job` | Fetches an existing job and refreshes it | | `usage()` | `dict` | Plan and remaining free runs | ```python theme={null} job = client.quantize( "model.onnx", calibration_data=["img1.jpg", "img2.jpg", "img3.jpg"], # required for the default ptq_static ) job.wait() bundle_dir = job.download("./out") ``` ## Job A handle to a running job: | Member | Description | | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- | | `.id` | Job id | | `.status` | `pending`, `running`, `completed`, or `error` | | `.progress` | Percentage (or `None`) | | `.progress_message` | Current pipeline step (or `None`) | | `.refresh()` | Re-fetch the job state, returns `self` | | `.result()` | The raw result payload (or `None`) | | `.error()` | The raw error payload (or `None`) | | `.wait(poll_interval=5.0, timeout=None, verbose=True)` | Block until done, printing live progress; raises `JobFailedError` on failure and `R3ALPlatformError` on timeout | | `.download(output_dir="./quantized_output")` | Download and unpack the deliverable, returns the bundle `Path` | ## Errors Platform errors all subclass `R3ALPlatformError` (itself a subclass of the quant `R3ALQuantError`), so existing `except R3ALQuantError` handling keeps working: | Error | When | | --------------------- | -------------------------------------- | | `AuthenticationError` | Missing or invalid API key (HTTP 401) | | `PlanLimitError` | Free-run quota reached (HTTP 402) | | `UploadError` | A model or calibration upload failed | | `JobFailedError` | The job reached a terminal error state | | `R3ALPlatformError` | Base class for the above | ```python theme={null} from r3alai.platform import PlanLimitError from r3alai.quant import Quantizer try: Quantizer().quantize( "model.onnx", calibration_data=["img1.jpg", "img2.jpg", "img3.jpg"], output_dir="./out", ) except PlanLimitError: print("Free plan limit reached. Upgrade at platform.r3al.ai.") ``` Config and quantization errors live in `r3alai.quant`: `UnsupportedConfigError`, `UnsupportedBitsError`, `CalibrationRequiredError`, `ModelLoadError`, `QuantizationError`, `BenchmarkError`, `BackendNotInstalledError`. ## Local helpers These never touch the network: ```python theme={null} from r3alai.quant import ( list_methods_local, validate_config_local, list_export_sources, export_to_onnx, maybe_export_to_onnx, load_quantized_bundle, ) validate_config_local({"method": "ptq_static"}) list_methods_local(scope="any_onnx_vision_model") list_export_sources() ``` | Helper | Signature | Returns | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `validate_config_local` | `(config=None, **kwargs)` | `{"status", "valid", "config", "requires_calibration", "is_dynamic"}`, or `{"status": "error", ...}` instead of raising | | `list_methods_local` | `(model_type=None, *, scope=None)` | `{"status", "methods"}` | | `list_export_sources` | `()` | One `{"source", "source_format", "description"}` per adapter | | `export_to_onnx` | `(model, output_dir="./exported", *, source, stem, input_shape, opset, imgsz, simplify, dynamic_batch, input_names, output_names, export_opts)` | `ExportResult` with `.onnx_path`, `.output_dir`, `.source_model`, `.source_format`, `.export_tool`, `.manifest_path` | | `maybe_export_to_onnx` | `(model, *, auto_export=False, output_dir=None, source=None, export_opts=None, **kwargs)` | `Path` — passes an `.onnx` through untouched, exports only when `auto_export=True` | | `load_quantized_bundle` | `(model_path)` | `{"session", "path", "manifest"}` |