neural_compressor.adaptor.torch_utils.util

Util Class and Functions.

Module Contents

Functions

get_embedding_contiguous(model)

This is a helper function for nn.Embedding, and it will get input contiguous.

is_fused_module(module)

This is a helper function for _propagate_qconfig_helper to detect if this module is fused.

collate_torch_preds(results)

Fetch collated results.

input2tuple(input)

This is a helper function to converting a inputting dict values or a list to a tuple.

append_attr(fx_model, model)

This is a helper method to append attributes for the symbolic traced model.

generate_activation_observer(scheme, algorithm)

This is a helper method to generate an activation observer.

check_cfg_and_qconfig(tune_cfg, cfgs, ...)

Check configs and quantization configs.

paser_cfgs(cfgs)

Parse configs.

get_quantizable_ops_from_cfgs(ops_name, ...)

Get quantizable ops from configs, combine fused ops as one op.

auto_copy(module)

Get an IPEX prepared model and return a fp32 model.

fetch_module(model, op_name)

Get module with a given op name.

set_module(model, op_name, new_module)

Set module with a given op name.

simple_inference(model, input)

Record model output tensor.

get_example_input(dataloader[, i])

Get the example input.

get_fallback_order(adaptor, fp32_model, dataloader, ...)

Get the fall back order for strategy.

get_mse_order_per_fp32(adaptor, model, example_inp, ...)

This is a helper method to check the mse influence to last module after QDQ(quant/dequant).

get_mse_order_per_int8(adaptor, fp32_model, ...)

This is a helper method to check the mse influence to last module after QDQ(quant/dequant).

get_torch_version()

Get torch version.
neural_compressor.adaptor.torch_utils.util.get_embedding_contiguous(model)

This is a helper function for nn.Embedding, and it will get input contiguous.

Parameters:

model (object) – the input model

Returns:

None

neural_compressor.adaptor.torch_utils.util.is_fused_module(module)

This is a helper function for _propagate_qconfig_helper to detect if this module is fused.

Parameters:

module (object) – the input module

Returns:

is fused or not

Return type:

(bool)

neural_compressor.adaptor.torch_utils.util.collate_torch_preds(results)

Fetch collated results.

Parameters:

result (list) – input result

Returns:

collated results

Return type:

collate_results (list)

neural_compressor.adaptor.torch_utils.util.input2tuple(input)

This is a helper function to converting a inputting dict values or a list to a tuple.

Parameters:

input (list or dict) –

Returns:

A tuple.

neural_compressor.adaptor.torch_utils.util.append_attr(fx_model, model)

This is a helper method to append attributes for the symbolic traced model.

Parameters:

  • fx_model (torch.fx.GraphModule) – The symbolic traced model.

  • model (torch.nn.Module) – The original model.

Returns:

The symbolic traced model with additional attributes.

Return type:

fx_model (dir)

neural_compressor.adaptor.torch_utils.util.generate_activation_observer(scheme, algorithm)

This is a helper method to generate an activation observer.

Parameters:

  • scheme (str) – Quantization scheme to be used.

  • algorithm (str) – What algorithm for computing the quantization parameters based on.

Returns:

An observer.

neural_compressor.adaptor.torch_utils.util.check_cfg_and_qconfig(tune_cfg, cfgs, op_infos_from_cfgs, output_tensor_ids_op_name)

Check configs and quantization configs.

Parameters:

  • tune_cfg (dict) – dictionary of quantization configuration.

  • cfgs (dict) – the input configs.

  • op_infos_from_cfgs (dict) – op infos from configs.

  • output_tensor_ids_op_name (dict) – dictionary of output tensor op names.

Returns:

cfgs (dict).

neural_compressor.adaptor.torch_utils.util.paser_cfgs(cfgs)

Parse configs.

Parameters:

cfgs (dict) – the input configs.

Returns:

list of op names. tune_cfg (dict): dictionary of quantization configuration. op_infos_from_cfgs (dict): op infos from configs. output_tensor_ids_op_name (dict): dictionary of output tensor op names.

Return type:

ops_name (list)

neural_compressor.adaptor.torch_utils.util.get_quantizable_ops_from_cfgs(ops_name, op_infos_from_cfgs, input_tensor_ids_op_name)

Get quantizable ops from configs, combine fused ops as one op.

Parameters:

  • ops_name (list) – list of op names.

  • op_infos_from_cfgs (dict) – op infos from configs.

  • input_tensor_ids_op_name (dict) – dictionary of input tensor op names.

Returns:

cfgs (dict).

neural_compressor.adaptor.torch_utils.util.auto_copy(module)

Get an IPEX prepared model and return a fp32 model.

Parameters:

module (object) – IPEX prepared model.

Returns:

fp32 model.

neural_compressor.adaptor.torch_utils.util.fetch_module(model, op_name)

Get module with a given op name.

Parameters:

  • model (object) – the input model.

  • op_name (str) – name of op.

Returns:

module (object).

neural_compressor.adaptor.torch_utils.util.set_module(model, op_name, new_module)

Set module with a given op name.

Parameters:

  • model (object) – the input model.

  • op_name (str) – name of op.

  • new_module (object) – the input model.

Returns:

module (object).

neural_compressor.adaptor.torch_utils.util.simple_inference(model, input)

Record model output tensor.

Parameters:

  • model (object) – the input model.

  • input (object) –

Returns:

output (object).

neural_compressor.adaptor.torch_utils.util.get_example_input(dataloader, i=1)

Get the example input.

Parameters:

dataloader (object) – calibration dataset.

Returns:

example_inp (object).

neural_compressor.adaptor.torch_utils.util.get_fallback_order(adaptor, fp32_model, dataloader, tune_cfg, confidence_batches, fallback=False, requantize_cfgs=None)

Get the fall back order for strategy.

Parameters:

  • fp32_model (object) – the input model.

  • dataloader (torch.utils.data.DataLoader) – The calibration dataloader.

  • tune_cfg (dict) – dictionary of quantization configuration.

  • confidence_batches (int) – number of confidence batches.

  • fallback (bool) – if the order is fallback.

Returns:

The fallback order for strategy.

Return type:

ordered_ops (dict/list)

neural_compressor.adaptor.torch_utils.util.get_mse_order_per_fp32(adaptor, model, example_inp, tune_cfg)

This is a helper method to check the mse influence to last module after QDQ(quant/dequant).

Parameters:

  • model (torch.fx.GraphModule/torch.nn.Module) – A torch model.

  • example_inp (object) – example inputs.

  • tune_cfg (dict) – dictionary of quantization configuration.

Returns:

The fallback order for strategy.

Return type:

fallback_order (dict/list)

neural_compressor.adaptor.torch_utils.util.get_mse_order_per_int8(adaptor, fp32_model, example_input, tune_cfg)

This is a helper method to check the mse influence to last module after QDQ(quant/dequant).

Parameters:

  • model (torch.fx.GraphModule/torch.nn.Module) – A torch model.

  • example_inp (object) – example inputs.

  • tune_cfg (dict) – dictionary of quantization configuration.

Returns:

The fallback order for strategy.

Return type:

fallback_order (dict/list)

neural_compressor.adaptor.torch_utils.util.get_torch_version()

Get torch version.