neural_compressor.metric

Intel Neural Compressor Metric.

Submodules

Package Contents

Classes

METRICS

Intel Neural Compressor Metrics.

Metric

A wrapper of the information needed to construct a Metric.

BaseMetric

The base class of Metric.

Functions

metric_registry(metric_type, framework)

Decorate for registering all Metric subclasses.
class neural_compressor.metric.METRICS(framework: str)

Bases: object

Intel Neural Compressor Metrics.

metrics

The collection of registered metrics for the specified framework.

register(name, metric_cls) → None

Register a metric.

Parameters:

  • name – The name of metric.

  • metric_cls – The metric class.

class neural_compressor.metric.Metric(metric_cls, name='user_metric', **kwargs)

Bases: object

A wrapper of the information needed to construct a Metric.

The metric class should take the outputs of the model as the metric’s inputs, neural_compressor built-in metric always take (predictions, labels) as inputs, it’s recommended to design metric_cls to take (predictions, labels) as inputs.

class neural_compressor.metric.BaseMetric(metric, single_output=False, hvd=None)

Bases: object

The base class of Metric.

property metric

Return its metric class.

Returns:

The metric class.

property hvd

Return its hvd class.

Returns:

The hvd class.

abstract update(preds, labels=None, sample_weight=None)

Update the state that need to be evaluated.

Parameters:

  • preds – The prediction result.

  • labels – The reference. Defaults to None.

  • sample_weight – The sampling weight. Defaults to None.

Raises:

NotImplementedError – The method should be implemented by subclass.

abstract reset()

Clear the predictions and labels.

Raises:

NotImplementedError – The method should be implemented by subclass.

abstract result()

Evaluate the difference between predictions and labels.

Raises:

NotImplementedError – The method should be implemented by subclass.

neural_compressor.metric.metric_registry(metric_type: str, framework: str)

Decorate for registering all Metric subclasses.

The cross-framework metric is supported by specifying the framework param as one of tensorflow, pytorch, mxnet, onnxrt.

Parameters:

  • metric_type – The metric type.

  • framework – The framework name.

Returns:

The function to register metric class.

Return type:

decorator_metric