Objective
Introduction
In terms of evaluating the status of a specific model during tuning, we should have general objectives. Intel® Neural Compressor Objective supports code-free configuration through neural_compressor.config.TuningCriterion. With built-in objectives, users can compress models with different objectives easily. In special cases, users can also register their own objective classes.
Single Objective
The objective supported by Intel® Neural Compressor is driven by accuracy. If users want to evaluate a specific model with other objectives, they can realize it with objective in a yaml file. Default value for objective is performance, and the other values are modelsize and footprint.
Multiple Objectives
In some cases, users want to use more than one objective to evaluate the status of a specific model and they can realize it with multi_objectives in a yaml file. Currently multi_objectives supports built-in objectives.
If users use multi_objectives to evaluate the status of the model during tuning, Neural Compressor will return a model with the best score of multi_objectives and meeting accuracy_criterion after tuning ending.
Intel® Neural Compressor will normalize the results of each objective to calculate the final weighted multi_objective score.
Objective Support Matrix
Built-in objectives support list:
| Objective | Usage |
|---|---|
| accuracy | Evaluate the accuracy |
| performance | Evaluate the inference time |
| footprint | Evaluate the peak size of memory blocks during inference |
| modelsize | Evaluate the model size |
Get Started with Objective API
Config Single Objective
Users can specify a built-in objective in neural_compressor.config.TuningCriterion as shown below:
from neural_compressor.config import TuningCriterion
tuning_criterion = TuningCriterion(objective="accuracy")
Config Multiple Objectives
Users can specify built-in multiple objectives in neural_compressor.config.TuningCriterion as shown below:
from neural_compressor.config import TuningCriterion
tuning_criterion = TuningCriterion(objective=["performance", "accuracy"])
Example
Refer to example as an example.