User Guide

Quark for PyTorch

There are several steps to quantize a floating-point model with Quark for PyTorch:

1. Load original float model

2. Set quantization configuration

3. Define dataloader

4. Use the Quark API to perform in-place replacement of the model’s modules with quantized module.

5. (Optional) Export quantized model to other format such as ONNX

More details:

• Configuring Quark for PyTorch

• Adding Calibration Datasets

• Exporting & Saving & Loading

• Feature Description

Quark for ONNX

There are several steps to quantize a floating-point model with Quark for ONNX:

1. Load original float model

2. Set quantization configuration

3. Define datareader

4. Use the Quark API to perform in-place replacement of the model’s modules with quantized module.

More details:

• Configuring Quark for ONNX

• Adding Calibration Datasets

• Feature Description

• Supported Datatype and OpType

• Accuracy Improvement

• Optional Utilities

• Tools