neural_compressor.compression.pruner.pruners.sparse_gpt

Module Contents

Classes

SparseGPTPruner

Pruning Pruner.
class neural_compressor.compression.pruner.pruners.sparse_gpt.SparseGPTPruner(config, modules, framework='pytorch')[source]

Pruning Pruner. The sparse_gpt pruner_class is derived from PytorchBasePruner. SparseGPTPruner supports one-shot pruning of most Large Language Models(LLMs). Please refer to SparseGPT: Massive Language Models Can be Accurately Pruned in One-shot.

Parameters:

  • modules – A dict {“module_name”: Tensor} that stores the pruning modules’ weights.

  • config – A config dict object that contains the pruner information.

pattern[source]

A Pattern object that defines pruning weights’ arrangements within space.

criterion[source]

A Criterion Object that defines which weights are to be pruned

scheduler[source]

A Scheduler object that defines how the model’s sparsity changes as training/pruning proceeds.

reg[source]

A Reg object that defines regulization terms.