High Performance Compute Reference Stack

The High Performance Computing Reference Stack (HPCRS) meets the needs of deploying HPC and AI workloads on the same system. This software solution reduces the complexities associated with integrating software components for High Performance Computing (HPC) workloads.

This guide explains how to use the pre-built HPCRS container image, build your own HPCRS container image, and use the reference stack.https://github.com/intel/stacks/blob/master/hpcrs/NEWS.md

Overview

HPCRS abstracts away the complexity of integrating multiple performance oriented software components including compilers, libraries, and frameworks. HPCRS enables users and developers of HPC and AI workloads to accomplish their work using a simple containerized solution.

Releases

Refer to the System Stacks for Linux* OS repository for information and download links for the different versions and offerings of the stack.

  • HPCRS v0.3.0 includes a pre-built container based on CentOS with optimized Intel oneAPI compilers and libraries for building HPC applications.

  • HPCRS v0.2.0 includes a pre-built container based on CentOS integrating proprietary software components for single-node usage.

  • HPCRS Release notes on GitHub* for the latest release of HPCRS.

Prerequisites

HPCRS can run on any host system running Linux which supports Intel® Advanced Vector Extensions 512 (Intel® AVX-512) and Docker*.

  • Linux OS (CentOS)

  • Docker

Stack features

The HPCRS provides a pre-built Docker image, which includes instructions on building the image from source. HPCRS is open-sourced to make sure developers have easy access to the source code and are able to customize it.

HPCRS provides the following software components:

NOTE:

Please see the third party program file and README text.

Get the pre-built HPCRS container image

Pre-built HPCRS Docker images are available. To use the HPCRS:

  1. Pull the image:

    docker pull sysstacks/hpcrs-centos

    NOTE:

    If you are on a network with outbound proxies, be sure to configure Docker to allow access. See the Docker service proxy and Docker client proxy documentation for more details.

  2. Once you have downloaded the image, run it using the following command:

    docker run -it sysstacks/hpcrs-centos

    This will launch the image and drop you into a bash shell inside the container in the /workspace folder.

    Paths from the host can be shared to the container with the --volume switch using Docker volumes.

Build the HPCRS container image from source

If you choose to build your own HPCRS container image, you can optionally add customizations as needed. The Dockerfile for the HPCRS is available on GitHub and can be used as a reference when creating your own container image.

  1. Clone the hpcrs repository.

    git clone https://github.com/intel/stacks.git

  2. Navigate to the stacks/hpcrs directory which contains the Dockerfile for the HPCRS.

    cd ./stacks/hpcrs

  3. Use the make to build the HPCRS container image. The make target will run the appropriate docker build command.

    make build

Use the HPCRS container image

This section shows examples of how the HPCRS container image can be used.

PyTorch benchmarks

This section describes running the PyTorch benchmarks for Caffe2 in single node.

  1. Get the pre-built HPCRS container image

  2. Run the image with Docker:

    docker run --name <image name>  --rm -i -t sysstacks/hpcrs-centos /bin/bash

    NOTE:

    Launching the Docker image with the -i argument starts interactive mode within the container. Enter the following commands in the running container.

  3. Navigate to to where Pytorch and Caffe are installed:

    cd /usr/lib/python3.8/site-packages/caffe2/python/

  4. Install dependencies for the benchmark:

    pip install protobuf
pip install future

  5. Execute the benchmark script:

    python convnet_benchmarks.py --batch_size 32 \
                             --cpu \
                             --model AlexNet

Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.

Convert the HPCRS image to a Singularity image

You may want to run the HPCRS image on Singularity for HPC workloads. A tool is available to easily convert the HPCRS Docker image to a Singularity image.

Refer to the README.md file in the d2s directory.

Reporting Security Issues

If you have discovered potential security vulnerability in an Intel product, please contact the iPSIRT at secure@intel.com.

It is important to include the following details:

  • The products and versions affected

  • Detailed description of the vulnerability

  • Information on known exploits

Vulnerability information is extremely sensitive. The iPSIRT strongly recommends that all security vulnerability reports sent to Intel be encrypted using the iPSIRT PGP key. The PGP key is available here: https://www.intel.com/content/www/us/en/security-center/pgp-public-key.html

Software to encrypt messages may be obtained from:

  • PGP Corporation

  • GnuPG

For more information on how Intel works to resolve security issues, see: Vulnerability handling guidelines