Intel Homomorphic Encryption Acceleration Library for FPGAs
(Intel HEXL for FPGA)

Intel:registered: HEXL for FPGA is an open-source library that provides an implementation of homomorphic encryption primitives on Intel FPGAs. Intel HEXL for FPGA targets integer arithmetic with word-sized primes, typically 40-60 bits. Intel HEXL for FPGA provides an API for 64-bit unsigned integers and targets Intel FPGAs.

Contents

• Intel Homomorphic Encryption Acceleration Library for FPGAs (Intel HEXL for FPGA)
  • Contents
  • Introduction
  • Setting up Environment
  • Building Intel HEXL for FPGA
    • Dependencies
    • Create Build Directory and Configure cmake Build
      • Configuration Options
    • Compiling Intel HEXL for FPGA
      • Compiling Device Kernels
        • Compile Kernels for Emulation
        • Compile Kernels for Generating FPGA Bitstream
      • Compiling Host Side
  • Installing Intel HEXL for FPGA
  • Testing Intel HEXL for FPGA
    • Run Tests in Emulation Mode
    • Run Tests on FPGA Card
  • Benchmarking Intel HEXL for FPGA
    • Run Benchmarks in Emulation Mode
    • Run Benchmarks on FPGA Card
  • Using Intel HEXL for FPGA
  • Debugging
• Documentation
  • Doxygen
• Contributing
  • Pull request acceptance criteria (Pending performance validation)
  • Repository layout
• Citing Intel HEXL for FPGA
  • Version 1.0
• Contributors
• Contact us

Introduction

Many cryptographic applications, particularly homomorphic encryption (HE), rely on integer polynomial arithmetic in a finite field. HE, which enables computation on encrypted data, typically uses polynomials with degree N: a power of two roughly in the range N=[2^{10}, 2^{14}]. The coefficients of these polynomials are in a finite field with a word-sized primes, p, up to p~62 bits. More precisely, the polynomials live in the ring Z_p[X]/(X^N + 1). That is, when adding or multiplying two polynomials, each coefficient of the result is reduced by the prime modulus p. When multiplying two polynomials, the resulting polynomials of degree 2N is additionally reduced by taking the remainder when dividing by X^N+1.

The primary bottleneck in many HE applications is polynomial-polynomial multiplication in Z_p[X]/(X^N + 1). Intel HEXL for FPGA provides the basic primitives that allow polynomial multiplication. For efficient implementation, Intel HEXL for FPGA uses the negacyclic number-theoretic transform (NTT). To multiply two polynomials, p_1(x), p_2(x) using the NTT, we perform the forward number-theoretic transform on the two input polynomials, then perform an element-wise modular multiplication, and perform the inverse number-theoretic transform on the result.

Intel HEXL for FPGA implements the following functions:

• The forward and inverse negacyclic number-theoretic transform (NTT).
• Dyadic multiplication.

For each function, the library provides an FPGA implementation using OpenCL.

For additional functionality, see the public headers, located in include/hexl-fpga.h

Note: we provide an integrated kernel implementing the NTT/INTT and the dyadic multiplication in one file. We also provide for convenience kernels implementing only one function stand alone. Those FPGA kernels work independently of each other, i.e. one does not require the use of another. The stand alone kernels allow testing and experimentation on a single primitive.

Setting up Environment

To use this code, a prerequisite is to install a PCIe card Intel PAC D5005 and its software stack, named Intel Acceleration Stack, which includes Quartus Prime, Intel FPGA SDK and Intel PAC D5005 board software package. See PREREQUISITE.md for details. If you have already installed the PCIe card and above mentioned softwares you can skip the procedure in the link given below.

You can find installation instructions for the FPGA PAC D5005 board software package following this link:
Hardware/Software Installation link

Check that your installation is functional with the software environment by running the Hello FPGA test code as indicated in the above link.

Building Intel HEXL for FPGA

Building Intel HEXL for FPGA library requires building all the depedencies ( mostly dealt automatically by cmake scripts) and two other separate pieces:

• Host application and related dependencies.
• FPGA kernels and HLS libraries needed by the kernels.

From user point of view it is required to go through these two main steps. Without building kernels, tests, benchmark and examples cannot be launched.

Dependencies

We have tested Intel HEXL for FPGA on the following operating systems:

• Centos 7.9.2009
  
• To check your Centos 7 version:
  ``` cat /etc/centos-release ```

Intel HEXL for FPGA requires the following dependencies:

Create Build Directory and Configure cmake Build

After cloning the git repository into your local area, you can use the following commands to set the install path and create a build directory. It will also create cmake cache files and make files that will be used for building host and kernels. Most of the build options described in previous section can be enabled or disabled by modifying the command given below:

``` cmake -S . -B build -DCMAKE_INSTALL_PREFIX=./hexl-fpga-install -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_COMPILER=gcc -DENABLE_FPGA_DEBUG=ON -DENABLE_TESTS=ON -DENABLE_DOCS=ON -DENABLE_BENCHMARK=ON ```

Different cmake options are provided allowing users to configure the overall build process. With these options the user can control if it is required to build tests, benchmark etc. Note that by default all options are off: the user must enable at least a few options to create a useful code. The recommended options can be found below. The details of these options is given in next section with default selection:

Configuration Options

In addition to the standard CMake configuration options, Intel HEXL for FPGA supports several cmake options to configure the build. For convenience, they are listed below:

Compiling Intel HEXL for FPGA

Compiling HEXL for FPGA requires two steps: compiling the C++ host code and compiling the OpenCL kernels. Start by compiling the kernels as they will be needed during the host installation. Before proceeding to the compilations and installation, make sure that your environment variables are set according to the instructions in the Intel PACD5005 Software Package installation guide.

Compiling Device Kernels

The kernels can be compiled in two different modes, emulation and FPGA. The emulation mode runs the kernels on the CPU. Compiling in emulation mode takes only a few minutes. The resulting bitstream can be used to verify the functionality of kernels on the CPU. The FPGA mode builds the kernel bitstream for FGPA card. Compiling the kernels in FPGA mode can take a few hours.

Compile Kernels for Emulation

To compile the device kernel for running in emulation mode:
``` cmake –build build –target emulation ```

This command takes a few minutes to execute.

**_NOTE:_** If you are interested to run kernels in software emulation mode only then this step is enough and you can move to building the host code. If you want to run the kernels on actual FPGA board please follow the next steps for building bitstream for the FPGA card.

Compile Kernels for Generating FPGA Bitstream

To compile the device kernel in fpga mode:
``` cmake –build build –target fpga ``` This command takes a few hours to execute.

The bitstreams will be located in the installation directory specified when calling the cmake command.(See installation below)

Compiling Host Side

To build the host application, tests, benchmark, and documentation (depending on the options selected above) run the following command: ``` cmake –build build ```

This will build the Intel HEXL for FPGA library in the build/host/ directory.

Installing Intel HEXL for FPGA

After compiling both host side and device kernels, users need to install HEXL for FPGA as a standalone library. The library is used for building and running HEXL for FPGA tests and benchmarks, and it can also be used as a third-party library. To install Intel HEXL for FPGA to the installation directory specified at configuration time:
``` cmake –install build ```

Testing Intel HEXL for FPGA

To run a set of unit tests via Googletest run the following command ( for running the test you should have chosen -DENABLE_TESTS=ON otherwise tests may not be enabled) (see Configuration Options).
Make sure that the .aocx files have been installed in the install directory that was chosen during configuration. The default choice we made was "./hexl-fpga-install".

Run Tests in Emulation Mode

In emulation mode the kernel will run on the CPU and the user will be able to test and validate the kernel.
To run in emulation mode (setting RUN_CHOICE to different values informs host code about emulation mode or FPGA run):

``` export RUN_CHOICE=1 cmake –build build –target tests ```

Run Tests on FPGA Card

To run using actual FPGA card, run the following command (setting RUN_CHOICE to different values informs host code about emulation mode or FPGA run):

``` export RUN_CHOICE=2 cmake –build build –target tests `` The tests executables are located inbuild/tests/` directory

Benchmarking Intel HEXL for FPGA

To run a set of benchmarks via Google benchmark, configure and build Intel HEXL for FPGA with -DENABLE_BENCHMARK=ON (see Configuration Options).
Make sure that the .aocx files have been installed in <chosen install directory>/bench/ directory.

Run Benchmarks in Emulation Mode

To run the benchmark in emulation mode:
``` export RUN_CHOICE=1 cmake –build build –target bench ```

Run Benchmarks on FPGA Card

To run the benchmark on the fpga, run
``` export RUN_CHOICE=2 cmake –build build –target bench `` The benchmark executables are located inbuild/benchmark/` directory

Using Intel HEXL for FPGA

The examples folder contains an example showing how to use Intel HEXL for FPGA library in a third-party project. See examples/README.md for details.

Debugging

For optimal performance, Intel HEXL for FPGA does not perform input validation. In many cases the time required for the validation would be longer than the execution of the function itself. To debug Intel HEXL for FPGA, configure and build Intel HEXL for FPGA with the option
-DCMAKE_BUILD_TYPE=Debug This will generate a debug version of the library that can be used to debug the execution. To enable the FPGA logs, configure the build with -DENABLE_FPGA_DEBUG=ON (see Configuration Options).

**_NOTE:_** Enabling -DCMAKE_BUILD_TYPE=Debug will result in a significant runtime overhead.

Documentation

See https://intel.github.io/hexl-fpga for Doxygen documentation.

Intel HEXL for FPGA supports documentation via Doxygen. To build documentation, first install doxygen and graphviz, e.g. ```bash sudo yum install doxygen graphviz `` Then, configure Intel HEXL for FPGA with-DENABLE_DOCS=ON` (see Configuration Options).

Doxygen

To build Doxygen documentation, after configuring Intel HEXL for FPGA with -DENABLE_DOCS=ON, run ``` cmake –build build –target docs `` To view the generated Doxygen documentation, open the generatedbuild/doc/doxygen/html/index.html` file in a web browser.

**_NOTE:_** After running the cmake –install build command, the documentation will also be available in:

<chosen install directory>/doc/doxygen/html/index.html.

Contributing

At this time, Intel HEXL for FPGA welcomes external contributions. To contribute to Intel HEXL for FPGA, see CONTRIBUTING.md. We encourage feedback and suggestions via Github Issues as well as discussion via Github Discussions.

Please use pre-commit to validate the formatting of the code before submitting a pull request.
To install pre-commit:
``` pip install –user cpplint pre-commit ``` To run pre-commit: ``` pre-commit run –all ```

Pull request acceptance criteria (Pending performance validation)

Pull requests will be accepted if they provide better acceleration, fix a bug or add a desirable new feature.

Before contributing, please run ```bash cmake –build build –target tests ``` and make sure pre-commit checks and all unit tests pass.

``` pre-commit run –all ```

Repository layout

Public headers reside in the hexl-fpga-install/include folder. Private headers, e.g. those containing fpga code should not be put in this folder.

Citing Intel HEXL for FPGA

To cite Intel HEXL for FPGA, please use the following BibTeX entry.

Version 1.0

```tex {IntelHEXLFPGA, author={Meng,Yan and de Souza, Fillipe and Butt, Shahzad and de Lassus, Hubert and Gonzales Aragon, Tomas and Zhou, Yongfa and Wang, Yong and others}, title = {{I}ntel {Homomorphic Encryption Acceleration Library for FPGAs for FPGA} (release 1.0)}, howpublished = {{https://github.com/intel/hexl-fpga}}, month = December, year = 2021, key = {Intel HEXL for FPGA} } ```

Contributors

The Intel contributors to this project, sorted by last name, are

• Paky Abu-Alam
• Tomas Gonzalez Aragon
• Flavio Bergamaschi
• Shahzad Butt
• Hubert de Lassus
• Fillipe D. M. de Souza
• Anil Goteli
• Jingyi Jin
• Yan Meng
• Nir Peled
• Yong Wang
• Yongfa Zhou

Contact us

• he_fp.nosp@m.ga_s.nosp@m.uppor.nosp@m.t@in.nosp@m.tel.c.nosp@m.om