neural_compressor.common.base_tuning

Module Contents

Classes

EvaluationFuncWrapper
Evaluator	Evaluator is a collection of evaluation functions.
ConfigSet
Sampler
SequentialSampler	Samples elements sequentially, always in the same order.
ConfigLoader
TuningConfig	Config for auto tuning pipeline.
TuningMonitor

Functions

init_tuning(→ Tuple[ConfigLoader, ...)

Attributes

default_sampler

class neural_compressor.common.base_tuning.Evaluator

Evaluator is a collection of evaluation functions.

Note: will deprecate this class in the future.

Examples

def eval_acc(model):

…

def eval_perf(molde):

…

# Usage user_eval_fns1 = eval_acc user_eval_fns2 = {“eval_fn”: eval_acc} user_eval_fns3 = {“eval_fn”: eval_acc, “weight”: 1.0, “name”: “accuracy”} user_eval_fns4 = [

{“eval_fn”: eval_acc, “weight”: 0.5}, {“eval_fn”: eval_perf, “weight”: 0.5, “name”: “accuracy”}, ]

class neural_compressor.common.base_tuning.SequentialSampler(config_source: Sized)

Samples elements sequentially, always in the same order.

Parameters:

config_source (_ConfigSet) – config set to sample from

class neural_compressor.common.base_tuning.TuningConfig(config_set: neural_compressor.common.base_config.BaseConfig | List[neural_compressor.common.base_config.BaseConfig] = None, sampler: Sampler = default_sampler, tolerable_loss=0.01, max_trials=100)

Config for auto tuning pipeline.

Examples

from neural_compressor.torch.quantization import TuningConfig tune_config = TuningConfig(

config_set=[config1, config2, …], max_trials=3, tolerable_loss=0.01)

The tuning process stops when either of the following conditions is met:

1. The number of trials reaches the maximum trials.

2. The metric loss is within the tolerable loss.

For condition 2), we calculate the metric loss as follows:

relative_loss = (fp32_baseline - eval_result_of_q_model) / fp32_baseline If relative_loss <= tolerable_loss, we stop the tuning process. For example:

tolerable_loss = 0.01 fp32_baseline = 100 eval_result_of_q_model = 99 relative_loss = (100 - 99) / 100 = 0.01 The metric loss is within the tolerable loss, so the tuning process is stopped.