neural_compressor.objective

The objectives supported by neural_compressor, which is driven by accuracy.

To support new objective, developers just need implement a new subclass in this file.

Module Contents

Classes

Objective	The base class for precise benchmark supported by neural_compressor.
Accuracy	Configuration Accuracy class.
Performance	Configuration Performance class.
Footprint	Configuration Footprint class.
ModelSize	Configuration ModelSize class.
MultiObjective	The base class for multiple benchmarks supported by neural_compressor.

Functions

objective_registry(cls)	The class decorator used to register all Objective subclasses.
objective_custom_registry(name, obj_cls)	Register a customized objective.

neural_compressor.objective.objective_registry(cls)

The class decorator used to register all Objective subclasses.

Parameters:

cls (object) – The class of register.

Returns:

The class of register.

Return type:

cls (object)

neural_compressor.objective.objective_custom_registry(name, obj_cls)

Register a customized objective.

class neural_compressor.objective.Objective

Bases: object

The base class for precise benchmark supported by neural_compressor.

property model

The interface benchmark model.

abstract reset()

The interface reset benchmark measuring.

abstract start()

The interface start benchmark measuring.

abstract end()

The interface end benchmark measuring.

result(start=None, end=None)

The result will return the total mean of the result.

The interface to get benchmark measuring result measurer may sart and end many times, can set the start and end index of the result list to calculate.

Parameters:

• start (int) – start point to calculate result from result list used to skip steps for warm up

• end (int) – end point to calculate result from result list

result_list()

The interface to get benchmark measuring result list.

This interface will return a list of each start-end loop measure value.

class neural_compressor.objective.Accuracy

Bases: Objective

Configuration Accuracy class.

start()

The interface start the measuring.

end(acc)

The interface end the measuring.

class neural_compressor.objective.Performance

Bases: Objective

Configuration Performance class.

start()

Record the start time.

end()

Record the duration time.

class neural_compressor.objective.Footprint

Bases: Objective

Configuration Footprint class.

start()

Record the space allocation.

end()

Calculate the space usage.

class neural_compressor.objective.ModelSize

Bases: Objective

Configuration ModelSize class.

start()

Start to calculate the model size.

end()

Get the actual model size.

class neural_compressor.objective.MultiObjective(objectives, accuracy_criterion, metric_criterion=[True], metric_weight=None, obj_criterion=None, obj_weight=None, is_measure=False)

The base class for multiple benchmarks supported by neural_compressor.

property baseline

Get the actual model performance.

property accuracy_target

Set the accuracy target.

compare(last, baseline)

The interface of comparing if metric reaches the goal with acceptable accuracy loss.

Parameters:

• last (tuple) – The tuple of last metric.

• baseline (tuple) – The tuple saving FP32 baseline.

accuracy_meets()

Verify the new model performance is better than the previous model performance.

evaluate(eval_func, model)

The interface of calculating the objective.

Parameters:

• eval_func (function) – function to do evaluation.

• model (object) – model to do evaluation.

reset()

Reset the objective value.

start()

Start to measure the objective value.

end(acc)

Calculate the objective value.

result()

Get the results.

set_model(model)

Set the objective model.

best_result(tune_data, baseline)

Calculate the best results.

metric + multi-objectives case:

tune_data = [

[acc1, [obj1, obj2, …]], [acc2, [obj1, obj2, …]], …

] multi-metrics + multi-objectives case: tune_data = [

[[acc1, acc2], [[acc1, acc2], obj1, obj2]], [[acc1, acc2], [[acc1, acc2], obj1, obj2]]

]