neural_compressor.pruner.regs

Regularizer.

Module Contents

Classes

BaseReg

Regularizer.

GroupLasso

Regularizer.

Functions

register_reg(name)

Register a regularizator to the registry.

get_reg_type(config)

Obtain the regularizer type.

get_reg(config, modules, pattern)

Get registered regularizator class.
neural_compressor.pruner.regs.register_reg(name)

Register a regularizator to the registry.

Parameters:

name – A string that defines the scheduler type.

Returns:

The class of register.

Return type:

cls

neural_compressor.pruner.regs.get_reg_type(config)

Obtain the regularizer type.

Parameters:

config – A config dict object that includes information of the regularizer.

neural_compressor.pruner.regs.get_reg(config, modules, pattern)

Get registered regularizator class.

Parameters:

  • config – A config dict object that includes information of the regularizer.

  • modules – A dict {“module_name”: Tensor} that stores the pruning modules’ weights.

  • pattern – A config dict object that includes information of the pattern.

class neural_compressor.pruner.regs.BaseReg(config: dict, modules: dict, pattern: neural_compressor.pruner.patterns.BasePattern)

Regularizer.

The class that performs regularization.

Parameters:

  • modules – A dict {“module_name”: Tensor} that stores the pruning modules’ weights.

  • config – A config dict object that includes information of the regularizer.

  • pattern – A config dict object that includes information of the pattern.

on_before_optimizer_step()

Implement before optimizer.step().

on_after_optimizer_step()

Implement after optimizer.step().

class neural_compressor.pruner.regs.GroupLasso(config: dict, modules: dict, pattern: neural_compressor.pruner.patterns.BasePattern, coeff)

Bases: BaseReg

Regularizer.

A regularizer class derived from BaseReg. In this class, the Group-lasso regularization will be performed. Group-lasso is a variable-selection and regularization method.

Parameters:

  • modules – A dict {“module_name”: Tensor} that stores the pruning modules’ weights.

  • config – A config dict object that includes information of the regularizer.

  • pattern – A config dict object that includes information of the pattern.

reg_terms

A dict {“module_name”: Tensor} of regularization terms.

alpha

A float representing the coeffient related to group lasso.

on_before_optimizer_step()

Calculate the group-lasso score map.

on_after_optimizer_step()

Perform group lasso regularization after optimization.