neural_compressor.common.base_tuning

The auto-tune module.

Classes

EvaluationFuncWrapper	Evaluation function wrapper.
Evaluator	Evaluator is a collection of evaluation functions.
ConfigSet	A class representing a set of configurations.
Sampler	Base class for samplers.
SequentialSampler	Samples elements sequentially, always in the same order.
ConfigLoader	ConfigLoader is a generator that yields configs from a config set.
TuningConfig	Config for auto tuning pipeline.
TuningMonitor	The tuning monitor class for auto-tuning.

Functions

init_tuning(→ Tuple[ConfigLoader, ...)

Initializes the tuning process.

Module Contents

class neural_compressor.common.base_tuning.EvaluationFuncWrapper(eval_fn: Callable, eval_args=None)

Evaluation function wrapper.

class neural_compressor.common.base_tuning.Evaluator

Evaluator is a collection of evaluation functions.

Note: will deprecate this class in the future.

Examples

def eval_acc(model):

…

def eval_perf(molde):

…

# Usage user_eval_fns1 = eval_acc user_eval_fns2 = {“eval_fn”: eval_acc} user_eval_fns3 = {“eval_fn”: eval_acc, “weight”: 1.0, “name”: “accuracy”} user_eval_fns4 = [

{“eval_fn”: eval_acc, “weight”: 0.5}, {“eval_fn”: eval_perf, “weight”: 0.5, “name”: “accuracy”}, ]

class neural_compressor.common.base_tuning.ConfigSet(config_list: List[neural_compressor.common.base_config.BaseConfig])

A class representing a set of configurations.

Parameters:

config_list (List[BaseConfig]) – A list of BaseConfig objects.

config_list

The list of BaseConfig objects.

Type:

List[BaseConfig]

class neural_compressor.common.base_tuning.Sampler(config_source: ConfigSet | None)

Base class for samplers.

class neural_compressor.common.base_tuning.SequentialSampler(config_source: Sized)

Samples elements sequentially, always in the same order.

Parameters:

config_source (_ConfigSet) – config set to sample from

class neural_compressor.common.base_tuning.ConfigLoader(config_set: ConfigSet, sampler: Sampler = default_sampler, skip_verified_config: bool = True)

ConfigLoader is a generator that yields configs from a config set.

class neural_compressor.common.base_tuning.TuningConfig(config_set: neural_compressor.common.base_config.BaseConfig | List[neural_compressor.common.base_config.BaseConfig] = None, sampler: Sampler = default_sampler, tolerable_loss=0.01, max_trials=100)

Config for auto tuning pipeline.

Examples

from neural_compressor.torch.quantization import TuningConfig tune_config = TuningConfig(

config_set=[config1, config2, …], max_trials=3, tolerable_loss=0.01)

The tuning process stops when either of the following conditions is met:

1. The number of trials reaches the maximum trials.

2. The metric loss is within the tolerable loss.

For condition 2), we calculate the metric loss as follows:

relative_loss = (fp32_baseline - eval_result_of_q_model) / fp32_baseline If relative_loss <= tolerable_loss, we stop the tuning process. For example:

tolerable_loss = 0.01 fp32_baseline = 100 eval_result_of_q_model = 99 relative_loss = (100 - 99) / 100 = 0.01 The metric loss is within the tolerable loss, so the tuning process is stopped.

class neural_compressor.common.base_tuning.TuningMonitor(tuning_config: TuningConfig)

The tuning monitor class for auto-tuning.

neural_compressor.common.base_tuning.init_tuning(tuning_config: TuningConfig) → Tuple[ConfigLoader, neural_compressor.common.utils.TuningLogger, TuningMonitor]

Initializes the tuning process.

Parameters:

tuning_config (TuningConfig) – The configuration for the tuning process.