neural_compressor.benchmark

Benchmark is used for evaluating the model performance.

Module Contents

Functions

set_env_var(env_var, value[, overwrite_existing])	Set the specified environment variable.
set_all_env_var(conf[, overwrite_existing])	Set all the environment variables with the configuration dict.
get_architecture()	Get the architecture name of the system.
get_threads_per_core()	Get the threads per core.
get_threads()	Get the list of threads.
get_physical_ids()	Get the list of sockets.
get_core_ids()	Get the ids list of the cores.
get_bounded_threads(core_ids, threads, sockets)	Return the threads id list that we will bind instances to.
fit(model[, config, b_dataloader, b_func])	Benchmark the model performance with the configure.

neural_compressor.benchmark.set_env_var(env_var, value, overwrite_existing=False)

Set the specified environment variable.

Only set new env in two cases: 1. env not exists 2. env already exists but overwrite_existing params set True

neural_compressor.benchmark.set_all_env_var(conf, overwrite_existing=False)

Set all the environment variables with the configuration dict.

Neural Compressor only uses physical cores

neural_compressor.benchmark.get_architecture()

Get the architecture name of the system.

neural_compressor.benchmark.get_threads_per_core()

Get the threads per core.

neural_compressor.benchmark.get_threads()

Get the list of threads.

neural_compressor.benchmark.get_physical_ids()

Get the list of sockets.

neural_compressor.benchmark.get_core_ids()

Get the ids list of the cores.

neural_compressor.benchmark.get_bounded_threads(core_ids, threads, sockets)

Return the threads id list that we will bind instances to.

neural_compressor.benchmark.fit(model, config=None, b_dataloader=None, b_func=None)

Benchmark the model performance with the configure.

Parameters:

• model (object) – The model to be benchmarked.

• config (BenchmarkConfig) – The configuration for benchmark containing accuracy goal, tuning objective and preferred calibration & quantization tuning space etc.

• b_dataloader – The dataloader for frameworks.

• b_func – Customized benchmark function. If user passes the dataloader, then b_func is not needed.

Example:

# Run benchmark according to config
from neural_compressor.benchmark import fit

conf = BenchmarkConfig(iteration=100, cores_per_instance=4, num_of_instance=7)
fit(model='./int8.pb', config=conf, b_dataloader=eval_dataloader)