quark.torch.export.config.config

quark.torch.export.config.config#

Module Contents#

Classes#

class quark.torch.export.config.config.ExporterConfig#

A class that encapsulates comprehensive exporting configurations for a machine learning model, allowing for detailed control over exporting parameters across different exporting formats.

Parameters:

  • json_export_config (Optional[JsonExporterConfig]) – Global configuration for json-safetensors exporting.

  • onnx_export_config (Optional[OnnxExporterConfig]) – Global configuration onnx exporting. Default is None.

class quark.torch.export.config.config.JsonExporterConfig#

A data class that specifies configurations for json-safetensors exporting.

Parameters:

  • weight_merge_groups (Optional[List[List[str]]]) – A list of operators group that share the same weight scaling factor. These operators’ names should correspond to the original module names from the model. Additionally, wildcards can be used to denote a range of operators. Default is None.

  • kv_cache_group (List[str]) – A list of operators group that should be merged to kv_cache. These operators’ names should correspond to the original module names from the model. Additionally, wildcards can be used to denote a range of operators.

  • weight_format (str) – The flag indicating whether to export the real quantized weights.

  • pack_method (str) – The flag indicating whether to reorder the quantized tensors.

class quark.torch.export.config.config.OnnxExporterConfig#

A data class that specifies configurations for onnx exporting.