CLI Reference¶

tlt¶

tlt [OPTIONS] COMMAND [ARGS]...

benchmark¶

Uses the Intel Neural Compressor to benchmark a trained model

tlt benchmark [OPTIONS]

Options

--model-dir, --model_dir <model_dir>¶: Required Model directory to reload for benchmarking. The model directory should contain a saved_model.pb for TensorFlow models or a model.pt file for PyTorch models.

--dataset-dir, --dataset_dir <dataset_dir>¶: Required Dataset directory for a custom dataset. Benchmarking is not supported with dataset catalogs at this time.

--dataset-file, --dataset_file <dataset_file>¶: Name of a file in the dataset directory to load. Used for loading a .csv file for text classification fine tuning.

--delimiter <delimiter>¶: Delimiter used when loading a dataset from a csv file. [default: ,]

--batch-size, --batch_size <batch_size>¶

Batch size used for benchmarking, if an INC config file is not provided. If an INC config file is provided, the batch size from the config file will be used.

Default: 32

--output-dir, --output_dir <output_dir>¶: A writeable output directory. The output directory will be used as a location to write the INC config file, if a config file is not provided. If no output directory is provided, a temporary folder will be created and then deleted after benchmarking has completed.

eval¶

Evaluates a model that has already been trained

tlt eval [OPTIONS]

Options

--model-dir, --model_dir <model_dir>¶: Required Model directory to reload and evaluate a previously exported model.

--model-name, --model_name <model_name>¶: Name of the model to evaluate. If a model name is not provided, the CLI will try to get the model name from the model directory path. For example, if the model directory is /tmp/efficientnet_b0/10, it will use ‘efficientnet_b0’ as the model name.

--dataset-dir, --dataset_dir <dataset_dir>¶: Required Dataset directory for a custom dataset, or if a dataset name and catalog are being provided, the dataset directory is the location where the dataset will be downloaded.

--dataset-file, --dataset_file <dataset_file>¶: Name of a file in the dataset directory to load. Used for loading a .csv file for text classification evaluation.

--delimiter <delimiter>¶: Delimiter used when loading a dataset from a csv file. [default: ,]

--class-names, --class_names <class_names>¶: Comma separated string of class names for a text classification dataset being loaded from .csv

--dataset-name, --dataset_name <dataset_name>¶: Name of the dataset to use from a dataset catalog.

--dataset-catalog, --dataset_catalog <dataset_catalog>¶

Name of a dataset catalog for a named dataset (Options: tf_datasets, torchvision, huggingface). If a dataset name is provided and no dataset catalog is given, it will default to use tf_datasets for a TensorFlow model, torchvision for PyTorch CV models, and huggingface datasets for HuggingFace models.

Options: tf_datasets | torchvision | huggingface

--dataset-file, --dataset_file <dataset_file>¶: Name of a file in the dataset directory to load. Used for loading a .csv file for text classification fine tuning, or a json / txt file for text generation

--prompt-with-context, --prompt_with_context <prompt_with_context>¶: Prompt with added context used to build the prompt dictionary

--prompt-without-context, --prompt_without_context <prompt_without_context>¶: Prompt without added context used to build the prompt dictionary

generate¶

Generates text from the model

tlt generate [OPTIONS]

Options

--model-name, --model_name <model_name>¶: Name of the model to use

--model-dir, --model_dir <model_dir>¶: Model directory to reload a previously exported model.

--prompt <prompt>¶: Prompt with added context used to build the prompt dictionary

--temperature <temperature>¶: The value used to modulate the next token probabilities (default: 1.0)

--top-p, --top_p <top_p>¶: If set to float < 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation (default: 0.75)

--top-k, --top_k <top_k>¶: The number of highest probability vocabulary tokens to keep for top-k-filtering (default: 40)

--repetition-penalty, --repetition_penalty <repetition_penalty>¶: The parameter for repetition penalty. 1.0 means no penalty. (default: 1.0)

--num-beams, --num_beams <num_beams>¶: Number of beams for beam search. 1 means no beam search. (default: 4)

--max-new-tokens, --max_new_tokens <max_new_tokens>¶: The maximum number of new tokens generated (default: 128)

list¶

Lists the available frameworks, use cases, or models

tlt list [OPTIONS] COMMAND [ARGS]...

frameworks¶

List the available frameworks

tlt list frameworks [OPTIONS]

models¶

List the available models

tlt list models [OPTIONS]

Options

-f, --framework <framework>¶: Filter the list of models by framework.

--use-case, --use_case <use_case>¶: Filter the list of models to a single use case

--verbose¶: Verbose output with extra information about each model

use_cases¶

List the available use cases

tlt list use_cases [OPTIONS]

optimize¶

Uses the Intel Neural Compressor to perform graph optimization on a trained model

tlt optimize [OPTIONS]

Options

--model-dir, --model_dir <model_dir>¶: Required Model directory to reload for graph optimization. The model directory should contain a saved_model.pb TensorFlow model.

--output-dir, --output_dir <output_dir>¶: Required A writeable output directory. The output directory will be used as a location to save the optimized model.

quantize¶

Uses the Intel Neural Compressor to perform post-training quantization on a trained model

tlt quantize [OPTIONS]

Options

--model-dir, --model_dir <model_dir>¶: Required Model directory to reload for quantization. The model directory should contain a saved_model.pb for TensorFlow models or a model.pt file for PyTorch models.

--dataset-dir, --dataset_dir <dataset_dir>¶: Required Dataset directory for a custom dataset. Quantization is not supported with dataset catalogs at this time.

--dataset-file, --dataset_file <dataset_file>¶: Name of a file in the dataset directory to load. Used for loading a .csv file for text classification fine tuning.

--delimiter <delimiter>¶: Delimiter used when loading a dataset from a csv file. [default: ,]

--batch-size, --batch_size <batch_size>¶

Batch size used during quantization, if an INC config file is not provided. If an INC config file is provided, the batch size from the config file will be used.

Default: 32

--approach <approach>¶

Specify to use static or dynamic quantization. Generally, static is recommended for image models and dynamic is recommended for text models.

Default: static
Options: static | dynamic

--accuracy-criterion, --accuracy_criterion <accuracy_criterion>¶

Relative accuracy loss to allow (for example, a value of 0.01 allows for a relative accuracy loss of 1%), if an INC config file is not provided. If an INC config file is provided, the accuracy criterion from the config file will be used.

Default: 0.01

--timeout <timeout>¶

Tuning timeout in seconds, if an INC config file is not provided. If an INC config file is provided, the timeout from the config file will be used. Tuning processing finishes when the timeout or max trials is reached. A tuning timeout of 0 means that the tuning phase stops when the accuracy criterion is met.

Default: 0

--max-trials, --max_trials <max_trials>¶

Maximum number of tuning trials, if an INC config file is not provided. If an INC config file is provided, the number of max trials from the config file will be used. Tuning processing finishes when the timeout or max trials is reached.

Default: 50

--output-dir, --output_dir <output_dir>¶: Required A writeable output directory. The output directory will be used as a location to save the quantized model, the tuning workspace, and the INC config file, if a config file is not provided.

train¶

Trains the model

tlt train [OPTIONS]

Options

-f, --framework <framework>¶

Deep learning framework [default: tensorflow]

Options: tensorflow | pytorch

--model-name, --model_name <model_name>¶: Required Name of the model to use

--use-case, --use_case <use_case>¶

Specify the use case if the model is supported by multiple use cases

Options: image_classification | anomaly_detection | image_anomaly_detection | text_classification | text_generation

--output-dir, --output_dir <output_dir>¶: Required Output directory for saved models, logs, checkpoints, etc

--dataset-dir, --dataset_dir <dataset_dir>¶: Required Dataset directory for a custom dataset, or if a dataset name and catalog are being provided, the dataset directory is the location where the dataset will be downloaded.

--dataset-file, --dataset_file <dataset_file>¶: Name of a file in the dataset directory to load. Used for loading a .csv file for text classification fine tuning, or a json / txt file for text generation

--delimiter <delimiter>¶: Delimiter used when loading a dataset from a csv file. [default: ,]

--class-names, --class_names <class_names>¶: Comma separated string of class names for a text classification dataset being loaded from .csv

--dataset-name, --dataset_name <dataset_name>¶: Name of the dataset to use from a dataset catalog. Not yet supported for Text Generation.

--dataset-catalog, --dataset_catalog <dataset_catalog>¶

Name of a dataset catalog for a named dataset (Options: tf_datasets, torchvision, huggingface). If a dataset name is provided and no dataset catalog is given, it will default to use tf_datasets for a TensorFlow model, torchvision for PyTorch CV models, and huggingface datasets for HuggingFace models. Not yet supported for Text Generation.

Options: tf_datasets | torchvision | huggingface

--epochs <epochs>¶: Number of training epochs [default: 1]

--init-checkpoints, --init_checkpoints <init_checkpoints>¶: Optional path to checkpoint weights to load to resume training. If the path provided is a directory, the latest checkpoint from the directory will be used.

--add-aug, --add_aug <add_aug>¶

Choice of data augmentation to be applied during training.

Options: hvflip | hflip | vflip | rotate | zoom

--ipex_optimize, --ipex-optimize¶: Boolean option to optimize model with Intel Extension for PyTorch.

-d, --distributed¶: Boolean option to trigger a distributed training job.

--nnodes <nnodes>¶: Number of nodes to run the training job [default: 1]

--nproc_per_node, --nproc-per-node <nproc_per_node>¶: Number of processes per node for the distributed training job [default: 1]

--hostfile <hostfile>¶: hostfile with a list of nodes to run distributed training.

--early-stopping, --early_stopping¶: Enable early stopping if convergence is reached while training (bool)

--lr-decay, --lr_decay¶: If lr_decay is True and do_eval is True, learning rate decay on the validation loss is applied at the end of each epoch.

--use-horovod, --use_horovod¶: Use horovod instead of default MPI

--hvd-start-timeout, --hvd_start_timeout <hvd_start_timeout>¶: Horovodrun has to perform all the checks and start the processes before the specified timeout. The default value is 30 seconds. Alternatively, The environment variable HOROVOD_START_TIMEOUT can also be used to specify the initialization timeout. Currently only supports PyTorch.

--simsiam¶: Use SimSiam anomaly detection

--cutpaste¶: Use CutPaste anomaly detection

--instruction-key, --instruction_key <instruction_key>¶: The instruction key from the dataset file used to determine the dataset schema

--context-key, --context_key <context_key>¶: The context key from the dataset file used to determine the dataset schema

--response-key, --response_key <response_key>¶: The response key from the dataset file used to determine the dataset schema

--prompt-with-context, --prompt_with_context <prompt_with_context>¶: Prompt with added context used to build the prompt dictionary

--prompt-without-context, --prompt_without_context <prompt_without_context>¶: Prompt without added context used to build the prompt dictionary

--lora-rank, --lora_rank <lora_rank>¶: LoRA rank parameter (default: 8)

--lora-alpha, --lora_alpha <lora_alpha>¶: LoRA alpha parameter (default: 32)

--lora-dropout, --lora_dropout <lora_dropout>¶: LoRA dropout parameter (default: 0.05)