Quark for ONNX - Adding Calibration Datasets#

Class DataReader to Quark quantizer#

Quark for ONNX utilizes [ONNXRuntime Quantization Dataloader] for normalization during quantization calibration. The code below is an example showing how to define the class of calibration data loader.

import onnxruntime
from onnxruntime.quantization.calibrate import CalibrationDataReader

class ImageDataReader(CalibrationDataReader):

    def __init__(self, calibration_image_folder: str, input_name: str,
     input_height: int, input_width: int):
        self.enum_data = None

        self.input_name = input_name

        self.data_list = self._preprocess_images(
                calibration_image_folder, input_height, input_width)

    # The pre-processing of calibration images should be defined by users.
    # Recommended batch_size is 1.
    def _preprocess_images(self, image_folder: str, input_height: int, input_width: int, batch_size: int = 1):
        data_list = []
        '''
        The pre-processing for each image
        '''
        return data_list

    def get_next(self):
        if self.enum_data is None:
            self.enum_data = iter([{self.input_name: data} for data in self.data_list])
        return next(self.enum_data, None)

    def rewind(self):
        self.enum_data = None

input_model_path = "path/to/your/resnet50.onnx"
output_model_path = "path/to/your/resnet50_quantized.onnx"
calibration_image_folder = "path/to/your/images"

input_name = 'input_tensor_name'
input_shape = (1, 3, 224, 224)
calib_datareader = ImageDataReader(calibration_image_folder, input_name,
 input_shape[2], input_shape[3])

Calibration Data Path to Quark quantizer#

Quark for ONNX supports specifying the path to calibration datasets, making it easy to load them for quantization. Currently, this feature only supports data in .npy format. Note that no preprocessing will be applied to the calibration datasets after loading. Please ensure that the calibration data is stored in the following format:

For Single-Input Models:#

Place the calibration data files in a directory as follows:

calib_data/
  calib1.npy
  calib2.npy
  calib3.npy
  calib4.npy
  calib5.npy
  ...

For Multi-Input Models:#

Organize the calibration data in subdirectories named after the input models:

calib_data/
  model_input1_name/
    calib1.npy
    calib2.npy
    calib3.npy
    calib4.npy
    calib5.npy
    ...
  model_input2_name/
    calib1.npy
    calib2.npy
    calib3.npy
    calib4.npy
    calib5.npy
    ...
  ...

Example Code:#

import onnxruntime
from quark.onnx import ModelQuantizer
from quark.onnx.quantization.config.config import (Config, get_default_config)

input_model_path = "path/to/your/resnet50.onnx"
output_model_path = "path/to/your/resnet50_quantized.onnx"
calib_data_path= "path/to/your/calib/data/folder"

quant_config = get_default_config("U8S8_AAWS")
config = Config(global_quant_config=quant_config)

quantizer = ModelQuantizer(config)
quantizer.quantize_model(input_model_path, output_model_path, calibration_data_reader=None, calibration_data_path=calib_data_path)

Quark for ONNX - Adding Calibration Datasets

Contents