Adding Calibration Datasets

Class DataReader for Quark Quantizer

Quark for ONNX utilizes ONNX Runtime’s CalibrationDataReader for normalization during quantization calibration. The following code is an example of how to define the class for the calibration data loader.

import onnxruntime
from onnxruntime.quantization.calibrate import CalibrationDataReader

class ImageDataReader(CalibrationDataReader):

    def __init__(self, calibration_image_folder: str, input_name: str,
     input_height: int, input_width: int):
        self.enum_data = None

        self.input_name = input_name

        self.data_list = self._preprocess_images(
                calibration_image_folder, input_height, input_width)

    # The pre-processing of calibration images should be defined by users.
    # Recommended batch_size is 1.
    def _preprocess_images(self, image_folder: str, input_height: int, input_width: int, batch_size: int = 1):
        data_list = []
        '''
        The pre-processing for each image
        '''
        return data_list

    def get_next(self):
        if self.enum_data is None:
            self.enum_data = iter([{self.input_name: data} for data in self.data_list])
        return next(self.enum_data, None)

    def rewind(self):
        self.enum_data = None

input_model_path = "path/to/your/resnet50.onnx"
output_model_path = "path/to/your/resnet50_quantized.onnx"
calibration_image_folder = "path/to/your/images"

input_name = 'input_tensor_name'
input_shape = (1, 3, 224, 224)
calib_datareader = ImageDataReader(calibration_image_folder, input_name,
 input_shape[2], input_shape[3])

Calibration Data Path for Quark Quantizer

Quark for ONNX supports specifying the path to calibration datasets, making it easy to load them for quantization. Currently, this feature only supports data in .npy format.

Note

No preprocessing is applied to the calibration datasets after loading. Ensure that the calibration data is stored in the following format:

For Single-Input Models:

Place the calibration data files in a directory as follows:

calib_data/
  calib1.npy
  calib2.npy
  calib3.npy
  calib4.npy
  calib5.npy
  ...

For Multi-Input Models:

Organize the calibration data in sub-directories named after the input models:

calib_data/
  model_input1_name/
    calib1.npy
    calib2.npy
    calib3.npy
    calib4.npy
    calib5.npy
    ...
  model_input2_name/
    calib1.npy
    calib2.npy
    calib3.npy
    calib4.npy
    calib5.npy
    ...
  ...

Example Code:

import onnxruntime
from quark.onnx import ModelQuantizer
from quark.onnx.quantization.config.config import (Config, get_default_config)

input_model_path = "path/to/your/resnet50.onnx"
output_model_path = "path/to/your/resnet50_quantized.onnx"
calib_data_path= "path/to/your/calib/data/folder"

quant_config = get_default_config("XINT8")
config = Config(global_quant_config=quant_config)

quantizer = ModelQuantizer(config)
quantizer.quantize_model(input_model_path, output_model_path, calibration_data_reader=None, calibration_data_path=calib_data_path)

Using Random Data for Quark Quantizer

Random Data Calibration uses random numbers when no calibration data is available. To enable this feature, set the UseRandomData parameter to True. This option is useful for testing but may yield worse quantization results than using a real calibration dataset. It is recommended to use a real calibration dataset when performing static quantization.

Example Code:

import onnxruntime
from quark.onnx import ModelQuantizer
from quark.onnx.quantization.config.config import Config, get_default_config

input_model_path = "path/to/your/resnet50.onnx"
output_model_path = "path/to/your/resnet50_quantized.onnx"

quant_config = get_default_config("XINT8")
quant_config.extra_options['UseRandomData'] = True
config = Config(global_quant_config=quant_config)

quantizer = ModelQuantizer(config)
quantizer.quantize_model(input_model_path, output_model_path, calibration_data_reader=None)