neural_compressor.utils.utility

Quantization auto-tuning config system.

This file specifies default config options for quantization auto-tuning tool. User should not change values in this file. Instead, user should write a config file (in yaml) and use cfg_from_file(yaml_file) to load it and override the default options.

Classes

LazyImport	Lazy import python module till use.
CpuInfo	Get CPU Info.
CaptureOutputToFile	Not displayed in API Docs.
Statistics	The statistics printer.
MODE	Mode: Quantization, Benchmark or Pruning.
GLOBAL_STATE	Access the global model.
DotDict	Access yaml using attributes instead of using the dictionary notation.
OpEntry	OP entry class.

Functions

version1_lt_version2(version1, version2)	Check whether version1 is less than version2.
version1_gt_version2(version1, version2)	Check whether version1 is greater than version2.
version1_eq_version2(version1, version2)	Check whether version1 is equal to version2.
version1_gte_version2(version1, version2)	Check whether version1 is greater than version2 or is equal to it.
version1_lte_version2(version1, version2)	Check whether version1 is less than version2 or is equal to it.
singleton(cls)	Not displayed in API Docs.
time_limit(seconds)	Limit the time for context execution.
get_size(obj[, seen])	Recursively finds size of objects.
compute_sparsity(tensor)	Compute the sparsity.
fault_tolerant_file(name)	Make another temporary copy of the file.
equal_dicts(d1, d2[, compare_keys, ignore_keys])	Check whether two dicts are same except for those ignored keys.
dump_elapsed_time([customized_msg])	Get the elapsed time for decorated functions.
combine_histogram(old_hist, arr)	Collect layer histogram for arr and combine it with old histogram.
get_tensor_histogram(tensor_data[, bins])	Get the histogram of the tensor data.
get_all_fp32_data(data)	Get all the fp32 data.
get_tuning_history(tuning_history_path)	Get tuning history.
recover(fp32_model, tuning_history_path, num, **kwargs)	Get offline recover tuned model.
str2array(s)	Get the array of the string.
dequantize_weight(weight_tensor, min_filter_tensor, ...)	Dequantize the weight with min-max filter tensors.
Dequantize(data, scale_info)	Dequantize the data with the scale_info.
load_data_from_pkl(path, filename)	Load data from local pkl file.
dump_data_to_local(data, path, filename)	Dump data to local as pkl file.
set_random_seed(seed)	Set the random seed in config.
set_workspace(workspace)	Set the workspace in config.
set_resume_from(resume_from)	Set the resume_from in config.
set_tensorboard(tensorboard)	Set the tensorboard in config.
show_memory_info(hint)	Show process full memory.
dump_class_attrs(obj[, result])	Dump the attributes and values of a config class.
deep_get(dictionary, keys[, default])	Get the dot key's item in nested dict.
deep_set(dictionary, keys, value)	Set the dot key's item in nested dict.
compare_objects(obj1, obj2, ignore_attrs)	Compare two objects and ignore the specified attributes.
alias_param(param_name, param_alias)	Decorator for aliasing a param in a function.
print_table(→ None)	Print table with prettytable.
get_tensors_info(→ dict)	Get information about tensors.
get_weights_details(→ list)	Get weights details for model.
dump_table(→ None)	Dump table data to file.
dump_table_to_csv(filepath, column_mapping, table_entries)	Dump table data to csv file.
get_number_of_sockets(→ int)	Get number of sockets in platform.
print_op_list(workload_location)	Print OP table and dump it to CSV.
get_op_list(→ List[OpEntry])	Get OP list for model.
calculate_mse(→ Optional[float])	Calculate MSE for specified OP.
mse_metric_gap(→ float)	Calculate the euclidean distance between fp32 tensor and int8 dequantize tensor.
check_key_exist(data, key)	Recursively checks if a key exists in a dictionary or list.

Module Contents

neural_compressor.utils.utility.version1_lt_version2(version1, version2)

Check whether version1 is less than version2.

neural_compressor.utils.utility.version1_gt_version2(version1, version2)

Check whether version1 is greater than version2.

neural_compressor.utils.utility.version1_eq_version2(version1, version2)

Check whether version1 is equal to version2.

neural_compressor.utils.utility.version1_gte_version2(version1, version2)

Check whether version1 is greater than version2 or is equal to it.

neural_compressor.utils.utility.version1_lte_version2(version1, version2)

Check whether version1 is less than version2 or is equal to it.

class neural_compressor.utils.utility.LazyImport(module_name)

Lazy import python module till use.

neural_compressor.utils.utility.singleton(cls)

Not displayed in API Docs.

Singleton decorator.

neural_compressor.utils.utility.time_limit(seconds)

Limit the time for context execution.

neural_compressor.utils.utility.get_size(obj, seen=None)

Recursively finds size of objects.

neural_compressor.utils.utility.compute_sparsity(tensor)

Compute the sparsity.

Parameters:

tensor – Tensorflow or Pytorch tensor

Returns:

(the original tensor size, number of zero elements, number of non-zero elements)

neural_compressor.utils.utility.fault_tolerant_file(name)

Make another temporary copy of the file.

neural_compressor.utils.utility.equal_dicts(d1, d2, compare_keys=None, ignore_keys=None)

Check whether two dicts are same except for those ignored keys.

class neural_compressor.utils.utility.CpuInfo

Get CPU Info.

neural_compressor.utils.utility.dump_elapsed_time(customized_msg='')

Get the elapsed time for decorated functions.

Parameters:

customized_msg (string, optional) – The parameter passed to decorator. Defaults to None.

neural_compressor.utils.utility.combine_histogram(old_hist, arr)

Collect layer histogram for arr and combine it with old histogram.

neural_compressor.utils.utility.get_tensor_histogram(tensor_data, bins=2048)

Get the histogram of the tensor data.

neural_compressor.utils.utility.get_all_fp32_data(data)

Get all the fp32 data.

neural_compressor.utils.utility.get_tuning_history(tuning_history_path)

Get tuning history.

Parameters:

tuning_history_path – The tuning history path, which need users to assign

neural_compressor.utils.utility.recover(fp32_model, tuning_history_path, num, **kwargs)

Get offline recover tuned model.

Parameters:

• fp32_model – Input model path

• tuning_history_path – The tuning history path, which needs user to assign

• num – tune index

neural_compressor.utils.utility.str2array(s)

Get the array of the string.

neural_compressor.utils.utility.dequantize_weight(weight_tensor, min_filter_tensor, max_filter_tensor)

Dequantize the weight with min-max filter tensors.

neural_compressor.utils.utility.Dequantize(data, scale_info)

Dequantize the data with the scale_info.

class neural_compressor.utils.utility.CaptureOutputToFile(tmp_file_path, stream=sys.stderr)

Not displayed in API Docs.

Capture the output to file.

class neural_compressor.utils.utility.Statistics(data, header, field_names, output_handle=logger.info)

The statistics printer.

class neural_compressor.utils.utility.MODE

Mode: Quantization, Benchmark or Pruning.

class neural_compressor.utils.utility.GLOBAL_STATE

Access the global model.

neural_compressor.utils.utility.load_data_from_pkl(path, filename)

Load data from local pkl file.

Parameters:

• path – The directory to load data

• filename – The filename to load

neural_compressor.utils.utility.dump_data_to_local(data, path, filename)

Dump data to local as pkl file.

Parameters:

• data – Data used to dump

• path – The directory to save data

• filename – The filename to dump

Returns:

loaded data

neural_compressor.utils.utility.set_random_seed(seed: int)

Set the random seed in config.

neural_compressor.utils.utility.set_workspace(workspace: str)

Set the workspace in config.

neural_compressor.utils.utility.set_resume_from(resume_from: str)

Set the resume_from in config.

neural_compressor.utils.utility.set_tensorboard(tensorboard: bool)

Set the tensorboard in config.

neural_compressor.utils.utility.show_memory_info(hint)

Show process full memory.

neural_compressor.utils.utility.dump_class_attrs(obj, result={})

Dump the attributes and values of a config class.

Parameters:

• obj – An instance of a config class.

• result – An dict for recording attributes and values.

neural_compressor.utils.utility.deep_get(dictionary, keys, default=None)

Get the dot key’s item in nested dict.

For example, person = {‘person’:{‘name’:{‘first’:’John’}}} deep_get(person, “person.name.first”) will output ‘John’.

Parameters:

• dictionary (dict) – The dict object to get keys

• keys (dict) – The deep keys

• default (object) – The return item if key not exists

Returns:

the item of the deep dot keys

Return type:

item

neural_compressor.utils.utility.deep_set(dictionary, keys, value)

Set the dot key’s item in nested dict.

For example, person = {‘person’:{‘name’:{‘first’:’John’}}} deep_set(person, “person.sex”, ‘male’) will output {‘person’: {‘name’: {‘first’: ‘John’}, ‘sex’: ‘male’}}

Parameters:

• dictionary (dict) – The dict object to get keys

• keys (dict) – The deep keys

• value (object) – The value of the setting key

class neural_compressor.utils.utility.DotDict(value=None)

Access yaml using attributes instead of using the dictionary notation.

Parameters:

value (dict) – The dict object to access.

neural_compressor.utils.utility.compare_objects(obj1, obj2, ignore_attrs)

Compare two objects and ignore the specified attributes.

Parameters:

• obj1 – The first object to compare.

• obj2 – The second object to compare.

• ignore_attrs – A list of attribute names to ignore during the comparison.

Returns:

True if the objects are equal ignoring the specified attributes, False otherwise.

neural_compressor.utils.utility.alias_param(param_name: str, param_alias: str)

Decorator for aliasing a param in a function.

Parameters:

• param_name – Name of param in function to alias.

• param_alias – Alias that can be used for this param.

neural_compressor.utils.utility.print_table(column_mapping: Dict[str, str], table_entries: List[Any], output_handler=logger.info, title: str | None = None, insert_newlines=False, precision: int | None = None) → None

Print table with prettytable.

:param : :param column_mapping: Dictionary where key is a column header and value corresponds

to object’s attribute.

Parameters:

• table_entries (list) – List of objects to be included in the table.

• output_handler (func, optional) – Output handler function.

• title (str) – Title for the table

• insert_newlines (bool) – Add newlines to each row

• precision (int) – Number of digits after the decimal point

Returns:

None

neural_compressor.utils.utility.get_tensors_info(workload_location, model_type: str = 'optimized') → dict

Get information about tensors.

Parameters:

• workload_location – path to workload directory

• model_type – type of model. Supported model types: “input”, “optimized”

Returns:

dictionary with tensors info

neural_compressor.utils.utility.get_weights_details(workload_location: str) → list

Get weights details for model.

Parameters:

workload_location – path to workload directory

Returns:

list of WeightDetails objects

neural_compressor.utils.utility.dump_table(filepath: str, column_mapping: Dict[str, str], table_entries: List[Any], file_type: str = 'csv') → None

Dump table data to file.

:param : :param filepath: The path of the file to which the data is to be dumped. :type filepath: str :param column_mapping: Dictionary where key is a column header and value corresponds

to object’s attribute.

Parameters:

• table_entries (list) – List of objects to be included in the table.

• file_type (str) – name of the file extension. Currently only csv is supported.

Returns:

None

neural_compressor.utils.utility.dump_table_to_csv(filepath: str, column_mapping: Dict[str, str], table_entries: List[Any])

Dump table data to csv file.

:param : :param filepath: The path of the file to which the data is to be dumped. :type filepath: str :param column_mapping: Dictionary where key is a column header and value corresponds

to object’s attribute.

Parameters:

table_entries (list) – List of objects to be included in the table.

Returns:

None

neural_compressor.utils.utility.get_number_of_sockets() → int

Get number of sockets in platform.

class neural_compressor.utils.utility.OpEntry(op_name: str, mse: float, activation_min: float, activation_max: float)

OP entry class.

neural_compressor.utils.utility.print_op_list(workload_location: str)

Print OP table and dump it to CSV.

Parameters:

workload_location – path to workload directory

Returns:

None

neural_compressor.utils.utility.get_op_list(minmax_file_path, input_model_tensors, optimized_model_tensors) → List[OpEntry]

Get OP list for model.

Parameters:

• minmax_file_path – path to activation_min_max.pkl

• input_model_tensors – dict with input tensors details

• optimized_model_tensors – dict with optimized tensors details

Returns:

list of OpEntry elements

neural_compressor.utils.utility.calculate_mse(op_name: str, input_model_tensors: dict, optimized_model_tensors: dict) → float | None

Calculate MSE for specified OP.

Parameters:

• op_name – name of the OP

• input_model_tensors – dict with input tensors details

• optimized_model_tensors – dict with optimized tensors details

Returns:

MSE value for specified op_name

neural_compressor.utils.utility.mse_metric_gap(fp32_tensor: Any, dequantize_tensor: Any) → float

Calculate the euclidean distance between fp32 tensor and int8 dequantize tensor.

Parameters:

• fp32_tensor (tensor) – The FP32 tensor.

• dequantize_tensor (tensor) – The INT8 dequantize tensor.

Returns:

MSE value

neural_compressor.utils.utility.check_key_exist(data, key)

Recursively checks if a key exists in a dictionary or list.

Parameters:

• data (dict or list) – The dictionary or list to search.

• key (any) – The key to search for.

Returns:

True if the key exists in the data structure, False otherwise.

Return type:

bool

Examples

>>> check_key_exist({'a': 1, 'b': {'c': 2}}, 'c')
True
>>> check_key_exist([{'a': 1}, {'b': 2}], 'b')
True
>>> check_key_exist({'a': 1, 'b': [1, 2, 3]}, 'c')
False