Pruning configuration

Quark Pruning Config API for PyTorch

class quark.torch.pruning.config.PConfig(algo_config: BaseAlgoConfig | None = None, blockwise_tuning_config: BaseAlgoConfig | None = None)

A class that encapsulates comprehensive pruning configurations for a machine learning model, allowing for detailed and hierarchical control over pruning parameters across different model components.

Parameters:

algo_config (Optional[BaseAlgoConfig]) – Optional configuration for the pruning algorithm, such as OSSCAR. After this process, the params will be reduced. Default is None.

class quark.torch.pruning.config.OSSCARConfig(name: 'str' = 'osscar', damp_percent: 'float' = 0.01, true_sequential: 'bool' = True, inside_layer_modules: 'list[str]' = [], mlp_pruning_modules: 'list[str]' = [], mlp_scaling_layers: 'dict[str, str | None]' = {}, mlp_pruning_ratio: 'float' = 0.1, mlp_intermediate_size_name: 'str' = '', model_decoder_layers: 'str' = '')

class quark.torch.pruning.config.LayerImportancePruneConfig(name: str = 'layer_importance_depth_pruning', delete_layer_num: int = 1, delete_layers_index: list[int] = [], save_gpu_memory: bool = False, layer_num_field: str = '', model_decoder_layers: str = '', layer_norm_field: str = '')

Configuration for layer importance depth wise prune algorithm (for LLM model).

Parameters:

• delete_layer_num (int) – Number of layers to delete (at least 1).

• delete_layers_index (List[int]) – Specific indexes of layers to delete.

• save_gpu_memory (bool) – Whether to save GPU memory (tradeoff speed vs. memory).

• layer_num_field (str) – Field name for number of layers.

• model_decoder_layers (str) – Field name for decoder layers.

• layer_norm_field (str) – Field name for normalization layer.