tlt.models.text_classification.tf_hf_text_classification_model.TFHFTextClassificationModel.benchmark

TFHFTextClassificationModel.benchmark(dataset, saved_model_dir=None, warmup=10, iteration=100, cores_per_instance=None, num_of_instance=1, inter_num_of_threads=None, intra_num_of_threads=None)

Use Intel Neural Compressor to benchmark the model with the dataset argument. The dataset’s validation or test subset will be used for benchmarking, if present. Otherwise, the full training dataset is used. The model to be benchmarked can also be explicitly set to a saved_model_dir containing for example a quantized saved model.

Parameters

  • dataset (ImageClassificationDataset) – Dataset to use for benchmarking

  • saved_model_dir (str) – Optional, path to the directory where the saved model is located

  • warmup (int) – The number of iterations to perform before running performance tests, default is 10

  • iteration (int) – The number of iterations to run performance tests, default is 100

  • cores_per_instance (int or None) – The number of CPU cores to use per instance, default is None

  • num_of_instance (int) – The number of instances to use for performance testing, default is 1

  • inter_num_of_threads (int or None) – The number of threads to use for inter-thread operations, default is None

  • intra_num_of_threads (int or None) – The number of threads to use for intra-thread operations, default is None

Returns

Benchmarking results from Intel Neural Compressor

Raises

  • NotADirectoryError – if the saved_model_dir is not a directory

  • FileNotFoundError – if a saved_model.pb is not found in the saved_model_dir or if the inc_config_path file

  • is not found –