Migrating Applications

This section provides a practical guide for migrating an application from the Intel QuickAssist Technology (QAT) out-of-tree (OOT) software package to the in-tree Linux* driver stack and QATlib.

Use this section together with the following references:

• System Requirements for supported devices, firmware, kernel modules, and boot parameters

• Installation for distribution-based and source-based installation steps

• Configuration and Tuning for managed mode, standalone mode, POLICY, and ServicesEnabled

• In-Tree vs. Out-of-Tree for a broader platform comparison

Overview

The migration from OOT to QATlib is not only a package replacement. It usually includes four areas of change:

• The kernel stack moves from a separately installed driver to the in-tree Linux kernel driver.

• User-space libraries move from legacy OOT library names to QATlib libraries.

• Platform configuration moves from per-device OOT configuration files to QATlib service management and /etc/sysconfig/qat.

• Some legacy or OOT-specific capabilities require application changes before migration is complete.

The following table summarizes the main differences that affect application migration.

Aspect	OOT Package	QATlib / In-Tree Stack
Kernel driver	Separate out-of-tree driver	In-tree Linux kernel driver
Supported devices	Devices up to and including GEN5	QAT devices from GEN4 onwards
Service management	OOT scripts and service flow	Two modes: (1) qat service with qatmgr in managed mode, or (2) standalone mode
Service provision model	Service provision via PFs or VFs	Service provision only via VFs
Configuration	Per-device *.conf files	Simplified /etc/sysconfig/qat plus in-tree kernel driver configuration
Device access	UIO-based model for OOT user-space acceleration; VFIO is used only for VF assignment to guests	VFIO-based model for QATlib user-space acceleration
DMA memory model	Separate kernel memory driver allocates physically contiguous memory; user-space allocates DMA-capable buffers via libusdm_s.so	VFIO maps and pins memory for DMA; user-space allocates DMA-capable buffers via libusdm.so
IOMMU usage	Deployment-dependent: often disabled for host UIO usage, enabled when VFIO is used for VF assignment to guests	Recommended with IOMMU enabled for DMA protection
Libraries	OOT library names libqat_s.so and libusdm_s.so	libqat.so and libusdm.so
Build system	OOT build flow and variables	Autotools-based build and standard packaging

Supported Devices and Migration Eligibility

Before starting the migration, verify that the target platform is supported by QATlib.

QATlib supports the following device families:

• QAT Gen 4: 4xxx, 401xx, 402xx

• QAT Gen 5: 420xx

Refer to Supported Devices for the complete list of device IDs, kernel modules, and firmware files.

Important

Applications deployed on legacy OOT-only QAT generations, i.e. before GEN4 (devices such as QAT1.7, QAT1.8, QAT1.9) cannot be migrated to QATlib without changing to a hardware platform supported by QATlib. Refer to Supported Devices.

Pre-Migration Checklist

Complete the following checks before changing the application or host configuration:

• Verify the platform uses a device listed in Supported Devices.

• Verify BIOS prerequisites such as Intel VT-d and SR-IOV. See System Requirements.

• Verify the required firmware is present in /lib/firmware. See Firmware.

• Verify the required kernel modules are loaded. See Kernel Drivers.

• Verify the required Linux boot parameters are configured. See Linux Boot Parameters.

• Back up any existing OOT configuration files and deployment scripts.

• Review the application for dependencies on legacy-only behavior, custom OOT configuration sections, or unsupported features.

• Identify the existing build and packaging assumptions, including include paths, linked libraries, and any OOT-specific environment variables.

If you are unsure whether the platform is currently in a migratable state, first complete the verification flow in System Requirements and then return to this page.

Step-by-Step Migration

1. Remove the OOT software stack

Before installing or enabling the in-tree stack, remove the OOT software to avoid conflicts between kernel modules, services, and configuration.

• Stop the OOT service and disable any startup logic that loads the OOT stack.

• Uninstall the OOT package from the build tree used to install it.

• Remove stale OOT configuration files if they are no longer needed.

• Unload OOT modules if they are still resident.

Refer to Uninstall Acceleration Software for the detailed uninstall procedure.

Before continuing to Step 2, confirm that OOT components remain inactive after reboot; partial removal can cause module, service, or configuration conflicts at runtime.

2. Install the in-tree driver stack and QATlib

After the OOT package is removed, install QATlib either from distribution packages or from source.

• For distribution-based installation, follow Installation.

• For source-based installation, follow Installing from Sources.

• If managed mode is required, enable and start the qat service after installation.

Typical deployment results in the following runtime model:

• The in-tree kernel driver exposes VFs for direct user-space access by QATlib and can also provide acceleration services to applications via kernel-space APIs.

• The qat service launches qatmgr in managed mode.

• User-space applications consume QAT resources through QATlib.

Milestone Check: Validate the In-Tree Stack Before Application Migration

Before moving to application-specific migration work, validate that the base in-tree stack is operating correctly.

• Confirm legacy OOT service/scripts are no longer active on the host.

• Run the QATlib sample application flow in Running Sample Code.

• Confirm cpa_sample_code runs successfully in the same mode you plan to use for your application (managed mode or standalone mode).

• Confirm platform state using Kernel Drivers and Linux Boot Parameters.

This milestone gives a clear before/after checkpoint: the platform can run known-good sample workloads on the in-tree stack before application migration begins.

3. Update platform and system configuration

Most migration problems are caused by platform configuration rather than application code.

Review and update the following areas:

• Linux boot parameters such as intel_iommu=on and vfio-pci.ids=...

• User permissions, including membership in the qat group

• Locked memory limits required for VFIO mappings

• Firmware placement and kernel module availability

Use System Requirements to verify the platform prerequisites, especially:

• Kernel/Firmware Requirements

• Supported Devices

Use Installation for post-install updates such as user group membership, service enablement, and memlock configuration.

4. Update the application build

Applications built against the OOT package usually require build-system updates even when the core API usage is unchanged.

At minimum, review the following items:

Build Updates Required During Migration

Build Area	Migration Action
Include paths	Use the installed QATlib headers, typically with the qat include prefix, for example #include <qat/cpa.h>
Library names	Link against libqat.so and libusdm.so instead of legacy OOT library names
Legacy compatibility	If legacy library names are required, build QATlib with --enable-legacy-lib-names
Packaging assumptions	Replace OOT-specific paths, install variables, and deployment logic with standard package or Autotools outputs

When building QATlib from source, the following configuration options are commonly relevant during migration:

• --enable-service to enable the qat service during installation

• --enable-systemd=no for environments that do not use systemd

• --enable-legacy-lib-names to preserve legacy library naming

• Users may wish to configure QATlib with the same settings used in their OOT environment when validating behavior or benchmarking performance parity: --enable-icp-debug, --disable-param-check, --disable-stats.

See Configuration Options in the installation section for the full option list.

5. Review application code and feature usage

Process Flow APIs

For many applications, the usage model does not change after migration.

• Initialize user-space access at startup

• Run QAT operations

• Release resources during shutdown

In practice, this means applications typically continue to call icp_sal_userStart before invoking QAT APIs, and call icp_sal_userStop during shutdown.

For API behavior and return-code details, refer to User Space Access Configuration Functions in the Programmer’s Guide.

Targeted Review Areas

Even when the start/stop flow remains familiar, applications should still perform a targeted migration review.

Review Areas and Migration Guidance

Review Area	What To Verify	Migration Guidance
Process name and section assumptions	Whether process names or section names are used to select fixed resources	Continue using process names as logical identifiers, but do not rely on static instance mappings from OOT configuration files. Validate with managed-mode policy and service settings.
OOT configuration file dependencies	Whether the application or scripts depend on per-device /etc/*_dev*.conf content	Replace dependencies on per-device OOT *.conf files with /etc/sysconfig/qat and current QATlib configuration workflow.
Unsupported or legacy-only features	Whether the application depends on capabilities that are not available in QATlib	Add fallback handling, feature gating, or migration blockers for unsupported functionality before final cutover.
Resource allocation model	Whether code assumes manually pinned VFs/instances from OOT deployment	In managed mode, rely on qatmgr allocation behavior. For non-managed deployments, review standalone mode policy and validate runtime allocation behavior explicitly.

For related API and configuration details, refer to User Space Access Configuration Functions, Logical Instances Section, Managed Mode, Policy, Services Enabled, and Standalone Mode.

Note: OOT Configuration Dependencies

OOT applications often depend on per-device configuration files such as /etc/4xxx_dev*.conf and named process sections such as [SSL].

With QATlib, the common managed-mode configuration entry point is /etc/sysconfig/qat.

The most important migration difference is:

• OOT typically uses detailed per-device configuration files.

• QATlib managed mode uses POLICY and ServicesEnabled to define how resources are exposed and allocated.

For example:

POLICY=1
ServicesEnabled=sym;asym

or

POLICY=1
ServicesEnabled=dc

Refer to Managed Mode, Policy, and Services Enabled for the supported values and scaling implications.

If the application previously depended on explicit instance-to-process mapping in OOT configuration files, that logic must be revisited. In QATlib managed mode, resource allocation is handled by qatmgr rather than by manually authored per-process sections.

Note: Feature and Capability Differences

The following tables compare common features and migration-relevant capabilities across the two stacks. Entries marked as pending are planned for a future release.

Platform and Distribution

Feature / Capability	OOT Package	QATlib / In-Tree Stack
Natively included in Linux kernel	No	Yes
Available in Red Hat Enterprise Linux (RHEL)	No	Yes
Available in SUSE / SLES	No	Yes
Available in Fedora	No	Yes
Library source on GitHub (qatlib)	No	Yes
QAT GEN4 (4xxx, 401xx, 402xx)	Yes	Yes
QAT GEN5 (420xx)	Yes	Yes

Compression

Feature / Capability	OOT Package	QATlib / In-Tree Stack
Deflate compression	Yes	Yes
Deflate decompression	Yes	Yes
Compress and Verify (CnV)	Yes	Yes
Batch and Pack (BnP) compression	Yes	No
ZSTD compression	No	Yes 1

1

Supported via QAT-ZSTD-PLUGIN.

Symmetric Cryptography

Feature / Capability	OOT Package	QATlib / In-Tree Stack
AES-CBC	Yes	Yes
AES-CTR	Yes	Yes
AES-GCM	Yes	Yes
AES-GMAC	Yes	Yes
AES-CCM	Yes	Yes
AES-XTS	Yes	Yes
SM4 (ECB, CBC, CTR)	Yes	Yes
SHA-1	Yes	Yes
SHA-2 (224, 256, 384, 512)	Yes	Yes
SHA-3	Yes	Yes
SM3	Yes	Yes
ChaCha20-Poly1305	Yes	Yes
Partial packet operations	Yes (traditional APIs only; not supported with DP APIs)	No
ZUC (EEA3, EIA3)	Yes	Yes 2
Snow3G (UEA2, UIA2)	Yes	Yes 2
KASUMI (F8, F9)	Yes	No 2

2(1,2,3)

ZUC, Snow3G, and KASUMI are supported on GEN5 platforms.

Asymmetric Cryptography

Feature / Capability	OOT Package	QATlib / In-Tree Stack
RSA (encrypt, decrypt, sign, verify)	Yes	Yes
ECDSA (sign, verify)	Yes	Yes
SM2	Yes	Yes
SSL/TLS key generation	Yes	Yes
ECDH (key generation)	Yes	Yes 3
DH (Diffie-Hellman)	Yes	Yes 3
DSA (sign, verify)	Yes	Yes 3

3(1,2,3)

DH, DSA, and ECDH are supported on GEN4 and GEN5 platforms.

Device Access

Feature / Capability	OOT Package	QATlib / In-Tree Stack
VFIO-based device access	Yes	Yes (kernel API only)
UIO-based device access	Yes	No
VFIO no-IOMMU mode	Yes	Yes
Service via Physical Functions (PFs)	Yes	No
Service via Virtual Functions (VFs)	Yes	Yes
SR-IOV	Yes	Yes

Configuration and Management

Feature / Capability	OOT Package	QATlib / In-Tree Stack
Per-device configuration files (*.conf)	Yes	No
Managed mode with qatmgr	No	Yes
Standalone mode	Yes	Yes
Systemd service integration	Limited	Yes
Rate Limiting	Yes	Yes 4
Dynamic Instance Allocation	Limited 5	No

4

Rate Limiting is supported but uses a different configuration model than OOT. Refer to the in-tree driver documentation for details.

5

Dynamic instance allocation in OOT was limited to legacy platforms and is not generally available across OOT deployments.

Scalability 6

Feature / Capability	OOT Package	QATlib / In-Tree Stack
Maximum DC processes per endpoint	Up to 64	Up to 16
Maximum total DC instances per endpoint	64	64

Feature / Capability	OOT Package	QATlib / In-Tree Stack
Maximum crypto processes per endpoint (SYM/ASYM)	Up to 32	Up to 16
Maximum total crypto instances per endpoint (SYM/ASYM)	32	32
Crypto instance exposure model	No CY abstraction; instances are service-specific (for example, SYM-only configuration exposes SYM instances)	CY instances are supported. With SYM+ASYM configuration, an instance can advertise both services; with single-service configuration, instances are service-specific

6

Scalability figures apply to GEN4 and GEN5 per-endpoint maximums. While QATlib limits the number of concurrent processes per endpoint, OOT and QATlib use different crypto instance exposure models. Higher OOT process counts apply only in UIO (no-IOMMU) mode.

6. Validate the migrated application

After the application and host have been updated, validate the migration in stages:

• Confirm the application can initialize, discover instances, and run the required services.

• Compare functional results and performance against the OOT baseline.

Common Migration Issues

The following issues are common when moving from OOT to the in-tree stack.

Issue	Likely Cause	Recommended Action
Library not found	New libraries installed outside the default search path	Run ldconfig or update the library search path after installation
Permission denied on VFIO	User is not in the qat group or the session has not been refreshed	Add the user to the qat group and re-login
No device found	VFs are not bound correctly, required boot parameters are missing, or the service is not running	Re-check device support, VFIO binding, firmware, boot parameters, and the qat service state
Compression instances missing	Services are configured for crypto only or the platform setup is incomplete	Review ServicesEnabled and kernel support for the intended compression mode
RLIMIT_MEMLOCK exceeded	Locked memory limit is too low for VFIO mappings	Increase the memlock limit and re-login
Process name behavior changed	The application still assumes OOT section-based resource selection	Remove assumptions that process names alone control instance allocation
Managed vs standalone mode mismatch	Runtime mode does not match the configured services/policy	Verify whether the deployment is using managed mode or standalone mode, then align service state and configuration

Migration Summary

For most applications, the migration can be reduced to the following sequence:

1. Confirm the target platform is supported by QATlib.

2. Remove the OOT stack and install the in-tree driver stack with QATlib.

3. Update system prerequisites such as firmware, VFIO, boot parameters, user permissions, and memlock limits.

4. Replace OOT-specific include paths, library names, and deployment assumptions.

5. Refactor code that depends on unsupported OOT-only features.

6. Reconfigure resource allocation using /etc/sysconfig/qat and validate the final application behavior.