Stable Diffusion Inference for Text2Image on Intel GPU
Intel® Extension for TensorFlow* is compatible with stock TensorFlow*. This example shows Stable Diffusion Inference for Text2Image.
Install the Intel® Extension for TensorFlow* in legacy running environment, Tensorflow will execute the Inference on Intel GPU.
Verified Hardware Platforms:
Intel® Data Center GPU Max Series
Intel® Data Center GPU Flex Series 170
Model Code change
We optimized official keras-cv Stable Diffusion, for example, concatenate two forward passes, combine computation in loops to reduce op number, and add fp16 mode for model. However, this optimization hasn’t been up streamed. To get better performance, instead of installing official keras-cv, you may want to clone keras-cv, apply patch, then install it as shown here:
git clone https://github.com/keras-team/keras-cv.git
git reset --hard 66fa74b6a2a0bb1e563ae8bce66496b118b95200
git apply patch
pip install .
Prepare for GPU (Skip this step for CPU)
Refer to Prepare
Setup Running Environment
Setup for GPU
Enable Running Environment
Enable oneAPI running environment (only for GPU) and virtual running environment.
For GPU, refer to Running
Running the Jupyter Notebook
Add kernel for env_itex environment:
python3 -m ipykernel install --name env_itex --user
Change to the sample directory.
Launch Jupyter Notebook.
jupyter notebook --no-browser --port=8888
Follow the instructions to open the URL with the token in your browser.
Locate and select the Notebook.
Run every cell in the Notebook in sequence.
Executes the Example with Python API
python stable_diffusion_inference.py --precision fp32
python stable_diffusion_inference.py --precision fp16
Note: At present, we evaluate accuracy by calculating the Fréchet inception distance (FID) between FP16 outcomes of 7 images on XPU and FP16 outcomes on NVIDIA A100. This may change with subsequent releases.
python stable_diffusion_accuracy.py --precision fp16 \
--load_ref_result --ref_result_dir "./nv_results/img_arrays_for_acc.txt"
With successful execution, it will print out the following results:
latency 81.1146879196167 ms, throughput 12.328223477737884 it/s
If you get the following error log, refer to Enable Running Environment to Enable oneAPI running environment.
tensorflow.python.framework.errors_impl.NotFoundError: libmkl_sycl.so.2: cannot open shared object file: No such file or directory