Cheat Sheet

Get started with Intel® Extension for PyTorch* using the following commands:

Description Command
Basic CPU Installation python -m pip install intel_extension_for_pytorch
Basic GPU Installation pip install torch==<version> -f https://developer.intel.com/ipex-whl-stable-xpu
pip install intel_extension_for_pytorch==<version> -f https://developer.intel.com/ipex-whl-stable-xpu
Import Intel® Extension for PyTorch* import intel_extension_for_pytorch as ipex
Capture a Verbose Log (Command Prompt) export ONEDNN_VERBOSE=1
Optimization During Training model = ...
optimizer = ...
model.train()
model, optimizer = ipex.optimize(model, optimizer=optimizer)
Optimization During Inference model = ...
model.eval()
model = ipex.optimize(model)
Optimization Using the Low-Precision Data Type bfloat16
During Training (Default FP32)
model = ...
optimizer = ...
model.train()

model, optimizer = ipex.optimize(model, optimizer=optimizer, dtype=torch.bfloat16)

with torch.no_grad():
with torch.cpu.amp.autocast():
model(data)
Optimization Using the Low-Precision Data Type bfloat16
During Inference (Default FP32)
model = ...
model.eval()

model = ipex.optimize(model, dtype=torch.bfloat16)

with torch.cpu.amp.autocast():
model(data)
[Experimental] Fast BERT Optimization from transformers import BertModel
model = BertModel.from_pretrained("bert-base-uncased")
model.eval()

model = ipex.fast_bert(model, dtype=torch.bfloat16)
Run CPU Launch Script (Command Prompt):
Automate Configuration Settings for Performance
ipexrun [knobs] <your_pytorch_script> [args]
[Experimental] Run HyperTune to perform hyperparameter/execution configuration search python -m intel_extension_for_pytorch.cpu.hypertune --conf-file <your_conf_file> <your_python_script> [args]
[Experimental] Enable Graph capture model = …
model.eval()
model = ipex.optimize(model, graph_mode=True)
Post-Training INT8 Quantization (Static) model = …
model.eval()
data = …

qconfig = ipex.quantization.default_static_qconfig

prepared_model = ipex.quantization.prepare(model, qconfig, example_inputs=data, anyplace=False)

for d in calibration_data_loader():
prepared_model(d)

converted_model = ipex.quantization.convert(prepared_model)
Post-Training INT8 Quantization (Dynamic) model = …
model.eval()
data = …

qconfig = ipex.quantization.default_dynamic_qconfig

prepared_model = ipex.quantization.prepare(model, qconfig, example_inputs=data)

converted_model = ipex.quantization.convert(prepared_model)
[Experimental] Post-Training INT8 Quantization (Tuning Recipe): model = …
model.eval()
data = …

qconfig = ipex.quantization.default_static_qconfig

prepared_model = ipex.quantization.prepare(model, qconfig, example_inputs=data, inplace=False)

tuned_model = ipex.quantization.autotune(prepared_model, calibration_data_loader, eval_function, sampling_sizes=[100],
accuracy_criterion={'relative': .01}, tuning_time=0)

convert_model = ipex.quantization.convert(tuned_model)