Rate Limiting

Rate Limiting Overview

Rate Limiting is implemented by monitoring the utilization of the device on a per-VF, per-service basis and comparing that to the SLA allocated to that VF and service. This ensures that resources are allocated according to predefined agreements, preventing any single VF from monopolizing device capacity.

Rate Limiting is set up on the host system, allowing administrators to manage and allocate resources effectively. It can be configured for each Physical Function (PF) on the device, providing granular control over resource distribution across different services and virtual functions.

Resources are shared across guests, and the resource utilization of each guest is measured relative to the capacity of the physical function. The feature is supported for SYM, ASYM, and DC services.

This document provides instructions for enabling Rate Limiting for both out-of-tree (OOT) and in-tree stacks.

Important

Rate limiting availability depends on the release and platform in the in-tree stack (QATlib). Refer to Linux Kernel Release Timeline & Features Overview in the QATlib Release Notes for details on which platforms and kernel versions support rate limiting.

Rate Limiting Reference Algorithms and Capacity

Important

Understanding reference algorithms is critical for proper SLA configuration across both In-Tree and Out-of-Tree implementations.

Device Capacity Reference Algorithms

QAT devices report capacity based on these reference algorithms:

  • Symmetric Crypto - AES-128-GCM with 4KB packet size

  • Compression - Dynamic DEFLATE level 1 with 64KB packet size

  • Asymmetric Crypto - RSA-2048 decrypt operations

Note

Migration Note: Customers upgrading from QAT 1.6/1.7 should note that the symmetric crypto reference changed from AES-128-CBC + SHA256-HMAC at 1KB to AES-128-GCM at 4KB.

SKU-Specific Capacity Values

Device capacity and performance characteristics vary significantly between different SKUs (such as MCC and XCC variants). The same reference algorithm may achieve different throughput levels depending on the specific hardware configuration.

In the QAT 2.0 access-layer/API path, SKU fields are informational for reporting and diagnostics. SLA/RL behavior should be treated as capability-driven rather than controlled by SKU-based API branching.

Warning

Always consult your device’s performance brief for SKU-specific capacity values and performance characteristics before configuring SLAs.

Out-of-Tree Rate Limiting

Enabling Rate Limiting With Out-of-Tree Driver

To enable the Rate Limiting feature for the Out-of-Tree stack:

  1. Install the driver package on the host with Single-Root Input/Output Virtualization (SR-IOV) enabled.

  2. Set ServicesEnabled to asym or sym or dc (or any combination of up to two of these services).

  3. Perform qat_service shutdown and qat_service start.

Important

For Out-of-Tree (OOT) PKE, the total CIR for all SLAs should equal 1000 to ensure proper rate limiting. For symmetric crypto and data compression services, the total CIR should equal the total capacity as returned by the sla_mgr tool.

Service Level Agreement (SLA)

Service Level Agreement enforcement allocates a specified amount of capacity for a specified service to a specified VF: max SLA enforced = (number of VFs) X (number of services) where:

  • Number of VFs varies based on device type

  • Number of services = 2 (asymmetric or symmetric or compression)

SLA Units With Out-of-Tree Driver

SLA units are measured as follows:

  • Symmetric Crypto - 1Mbps of throughput (based on AES-128-GCM @ 4KB reference algorithm).

  • Asymmetric Crypto - 1 unit is equal to 0.1 percent of available utilization.

  • Compression - 1Mbps of throughput (based on Dynamic DEFLATE level 1 @ 64KB reference algorithm).

Note

  1. In Gen4 devices, for Asymmetric Crypto services, SLA units are measured in terms of percentage of slice utilization. This metric is more accurate than operations/second or throughput/second because it directly reflects the hardware resources consumed by each user, independent of algorithm processing speed, providing a fair representation of resource usage.

  2. Gen4 devices use a Hardware-assisted Rate limiting approach whereas legacy devices use a firmware-only Rate limiting approach.

  3. For asymmetric service, SLAs shall be allocated at a granularity of 1 unit of device utilization percentage for RSA2K.

  4. Below is a sample mapping table for the 5th Gen Intel® Xeon® Scalable Processer - MCC SKU platform that translates the SLA units to equivalent ops/sec.

  5. Users can run tests with the required algorithm to determine the mapping for other SKUs with different performance. Gen4 asymmetric performance numbers can differ based on the SKU.

Sample Mapping Table for 5th Gen Intel® Xeon® Scalable Processer - MCC SKU

Unit

RSA2K decrypt with CRT (Ops/sec)

RSA4K decrypt with CRT (Ops/sec)

1

60

–

5

300

–

10

600

60

50

3000

300

100

6000

600

300

18000

1800

500

30000

3000

750

45000

4500

1000

60000

6000

Migration from QAT 1.x

Customers upgrading from QAT 1.6/1.7 should be aware of reference algorithm changes:

  • Symmetric Crypto: The reference algorithm changed from AES-128-CBC + SHA256-HMAC at 1KB to AES-128-GCM at 4KB. This may require SLA scaling based on your specific SKU and algorithms.

  • Algorithm Impact: Performance varies significantly with algorithm choice and packet size compared to the reference.

  • Decompression Behavior: Decompression operations may achieve approximately 2x the throughput of compression operations for the same SLA setting.

Note

Test your specific algorithms and packet sizes to determine appropriate SLA values for your use case.

SLA Manager Application

The sla_mgr tool is used to create, update, delete, list and get SLA capabilities. The SLA Manager executable is available in $ICP_ROOT/build/sla_mgr after the package is built and installed using ./configure; make install commands.

SLA Commands

Rate Limiting SLA Commands

Operation

Command

Rate Limiting V1 (Legacy)

Create SLA

./sla_mgr create <vf_addr> <rate_in_sla_units> <service>

Update SLA

./sla_mgr update <pf_addr> <sla_id> <rate_in_sla_units>

Rate Limiting V2

Create SLA

./sla_mgr create <vf_addr> <cir> <pir> <service>

Update SLA

./sla_mgr update <pf_addr> <sla_id> <cir> <pir>

For Legacy and Rate Limiting V2

Delete SLA

./sla_mgr delete <pf_addr> <sla_id>

Delete all SLAs

./sla_mgr delete_all <pf_addr>

Query SLA capabilities

./sla_mgr caps <pf_addr>

Query list of SLAs

./sla_mgr list <pf_addr>

Options:

  • pf_addr - Physical address in domain:bus:device.function(xxxx:xx:xx.x) format.

  • vf_addr - Virtual address in domain:bus:device.function(xxxx:xx:xx.x) format.

  • Service - Asym(=0) or Sym(=1) or DC(=2).

  • rate_in_sla_units - [ 0-MAX]. MAX is found by querying the capabilities.

  • cir/pir - committed/peak information rate [0-MAX]. MAX is found by querying the capabilities.

  • sla_id - Value returned by create command.

In Legacy mode, to create/update SLA we use rate_in_sla_units. With Rate Limiting V2, we use cir/pir. These units are equal to:

  • 1 operation per second - for asymmetric service (Legacy) or 0.1 percent of available utilization (Rate Limiting V2).

  • 1 Megabits per second - for symmetric service/compression service.

Note

To use Legacy Rate limiting sla_mgr application, user needs to configure with option –enable-legacy-sla-mgr.

Best Practices for SLA Management

  • Ensure all VFs are included in SLAs to prevent unregulated resource usage.

  • Regularly monitor performance and adjust CIR and PIR values as needed to maintain optimal throughput.

Out-of-Tree Troubleshooting

Issue: SLA performance doesn’t match expectations

Solutions:

  1. Check if your algorithm matches the reference (AES-128-GCM @ 4KB for symmetric crypto)

  2. For different algorithms, scale SLA based on performance brief ratios

  3. Verify total SLA allocation doesn’t exceed device capacity

Issue: Decompression achieves higher throughput than expected

Solution: This is expected behavior. Decompression can achieve ~2x compression throughput.

Issue: Asymmetric performance varies significantly between workloads

Solution: Gen4 devices use slice utilization measurement which provides more accurate resource allocation than operations-per-second metrics.

Debugging Commands

Check current SLA configuration:

./sla_mgr list <pf_addr>

Query device capabilities:

./sla_mgr caps <pf_addr>

Verify SLA is active:

./sla_mgr get <pf_addr> <sla_id>

In-tree Rate Limiting

Note

For additional details on Rate Limiting with the In-tree solution, refer to sysfs-driver-qat_rl documentation.

Rate Limiting for the in-tree stack is configured per individual Physical Function (PF) using sysfs calls. Each PF has a directory structure that includes several files used to manage SLAs:

Directory Structure

The rate limiting attributes for each PF are located at:

/sys/bus/pci/devices/<BDF>/qat_rl/

The files included in this directory are:

  • cap_rem: Reports the remaining capability for a particular service/SLA. This is the remaining value that a new SLA can be set to or a current SLA can be increased with.

  • cir: Committed Information Rate (CIR). The guaranteed rate of throughput that a VF can achieve under its SLA. The value is expressed in permille scale, i.e., 1000 refers to the maximum device throughput for a selected service.

  • id: Used to retrieve a particular SLA and operate on it. Valid for update, rm, and get operations.

  • pir: Peak Information Rate (PIR). The maximum rate that can be achieved by that particular SLA. An SLA can reach a value between CIR and PIR when the device is not fully utilized by requests from other users.

  • rp: Configures the ring pairs associated with an SLA. The value is a 64-bit bit mask and is written/displayed in hex.

  • sla_op: Used to perform operations on an SLA, such as add, update, rm, rm_all, and get.

  • srv: Represents the service (sym, asym, dc) associated with an SLA.

Enabling Rate Limiting With In-Tree Driver

To enable the Rate Limiting feature for the In-tree stack:

  1. Identify the device using the Bus-Device-Function (BDF) format, e.g., <BDF>.

  2. Configure the SLA using the sysfs attributes available for qat_4xxx devices.

  3. Ensure the total CIR for all VFs equals 1000 to ensure proper rate limiting.

SLA Units With In-Tree Driver

For the In-tree stack, SLA units are measured as follows:

  • All services (sym, asym, dc) - Values are expressed in permille scale, where 1000 refers to the maximum device throughput for the selected service.

Note

The percentage-based approach provides consistent units across all services and abstracts away underlying reference algorithm details. Use the cap_rem attribute to check remaining capacity for each service.

Example Setting of SLAs

This example demonstrates setting up SLAs for all VFs for a specified PF, focusing on symmetric and asymmetric crypto services. The RP value is shifted for each VF to allocate resources appropriately.

  1. Remove Existing SLAs: Clear any existing SLAs for the device.

    echo "rm_all" > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/sla_op

  2. Set SLAs for Symmetric Service: For each VF, set the SLA parameters and add the SLA:

    for vf in {0..15}; do
    cir_pir_value=62
    rp_value=$(printf "0x%x" $((0xa << (vf * 4))))
    echo $cir_pir_value > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/cir
    echo $cir_pir_value > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/pir
    echo "sym" > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/srv
    echo $rp_value > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/rp
    echo "add" > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/sla_op
    echo "SLA added for BDF: 0000:6b:00.0, VF: $vf, Service: sym, RP: $rp_value, CIR/PIR: $cir_pir_value"
done

  3. Set SLAs for Asymmetric Service: Similarly, set the SLA parameters for the asymmetric service:

    for vf in {0..15}; do
    cir_pir_value=62
    rp_value=$(printf "0x%x" $((0x5 << (vf * 4))))
    echo $cir_pir_value > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/cir
    echo $cir_pir_value > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/pir
    echo "asym" > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/srv
    echo $rp_value > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/rp
    echo "add" > /sys/bus/pci/devices/0000:6b:00.0/qat_rl/sla_op
    echo "SLA added for BDF: 0000:6b:00.0, VF: $vf, Service: asym, RP: $rp_value, CIR/PIR: $cir_pir_value"
done

This example illustrates setting up rate limiting for one QAT endpoint, evenly distributing the PF capacity among 16 VFs for both symmetric and asymmetric services.

Advanced qat_sla_rl.py Reference (In-Tree Only)

Important

qat_sla_mgr is the primary and supported customer-facing CLI for in-tree rate limiting management. Use qat_sla_rl.py directly only for advanced scenarios documented below.

Purpose and Scope

qat_sla_rl.py is the underlying implementation used by qat_sla_mgr for in-tree kernel rate limiting configuration. It manages SLA creation, update, query, and deletion through the in-tree sysfs ABI at:

/sys/bus/pci/devices/<BDF>/qat_rl/

Warning

qat_sla_rl.py is compatible with the in-tree kernel stack only. It is not compatible with out-of-tree (OOT) driver packages.

Use qat_sla_mgr for normal operational workflows.

Use qat_sla_rl.py directly only when you need capabilities not available through standard qat_sla_mgr arguments, such as:

  • Explicit ring-pair mask assignment with -r.

  • Direct use of internal permille units (without Mbps abstraction).

  • SLA layouts that intentionally span custom ring-pair patterns.

  • Low-level sysfs troubleshooting and validation workflows.

  • Implementation-level automation tightly coupled to in-tree sysfs behavior.

System Requirements

  • In-tree intel_qat module loaded (/sys/module/intel_qat exists).

  • Rate limiting sysfs exposed at /sys/bus/pci/devices/*/qat_rl/.

  • Root privileges for sysfs write operations.

  • Python 3.8 or later.

  • Supported Intel QAT 4xxx family devices: device IDs 0x4940, 0x4942, 0x4944, 0x4946, 0x4948.

Installation and Deployment

qat_sla_rl.py is primarily an internal utility that is expected to be distributed with qat_sla_mgr and installed together by the QAT package build/install flow.

Recommended production deployment:

  • Install qat_sla_mgr and qat_sla_rl.py from the same package release.

  • Keep both tools co-located (for example in the same package-managed binary directory).

  • Set ownership and permissions to root-owned executables (mode 755).

  • Expose qat_sla_mgr as the user-facing CLI for administrators.

  • Reserve direct qat_sla_rl.py invocation for advanced admin automation and low-level debugging.

Note

If your deployment uses OOT driver packages, use the OOT-provided tools and workflows instead of qat_sla_rl.py.

Supported Operations

qat_sla_rl.py operation reference

Operation

Description

Key Parameters

add

Create SLA for a VF context

VF BDF, -c, -p, -s, optional -r

update

Modify CIR/PIR for an existing SLA

PF BDF, -i, -c, optional -p

get

Query one SLA

PF BDF, -i

rm

Remove one SLA

PF BDF, -i

rm_all

Remove all SLAs on a PF

PF BDF

list

Enumerate active SLAs

PF BDF

caps

Query capacity across configured services

PF BDF

cap_rem

Query remaining capacity for one service

PF BDF, -s

Warning

rm_all affects all SLAs on the PF and can immediately impact all consumers sharing the device.

Rate Units and Conversion

  • CLI default in qat_sla_rl.py: internal permille units (0..1000).

  • Optional CLI mode: --scale_mbps (or -sm) for Mbps input/output.

  • Internal kernel/sysfs representation: permille.

Conversion logic uses the device/service capacity from DEVICE_CAPS:

  • Mbps to permille: permille = floor(mbps * 1000 / max_mbps)

  • Permille to Mbps: mbps = floor(permille * max_mbps / 1000)

Note

In scale mode, update flows can include an extra read/convert/write cycle, so minor double-rounding effects may occur. Validate with get after updates.

Ring Pair Allocation

Each QAT device exposes 4 service slots (ring pairs 0-3 per device) for service mapping. Service-to-ring-pair mapping is available at:

/sys/bus/pci/devices/<BDF>/qat/rp2srv

The -r parameter is a 64-bit hex mask representing allocation of these service slots to virtual functions (VFs). Each VF occupies 4 bits (one nibble) in the mask, encoding which of the 4 device service slots (0-3) are allocated to that VF. This is distinct from global telemetry ring-pair numbering (0-63 across devices).

qat_sla_rl.py supports two allocation models:

  • Explicit mask via -r (64-bit hex bit mask, 4 bits per VF).

  • Auto-allocation from VF address when -r is omitted.

If VF-based auto-allocation is used, qat_sla_rl.py derives VF identity from BDF, builds a service mask from rp2srv (selecting which of the 4 device service slots), and shifts it into the appropriate VF nibble within the 64-bit mask.

Known Limitations and Restrictions

  • In-tree only (requires /sys/bus/pci/devices/*/qat_rl/ ABI).

  • OOT users must use OOT package utilities.

  • Supported hardware is limited to device IDs in DEVICE_CAPS.

  • Ring-pair constraints are hardware/kernel limited (4 ring pairs per VF map).

  • Service selection must match configured ring allocations.

  • Root privileges are required by kernel sysfs permissions.

  • One SLA per service per VF is kernel-enforced.

Error Handling and Validation Guidance

Common failures and checks:

  • Permission denied: run as root and verify sysfs write access.

  • Device not found: verify BDF and presence under /sys/bus/pci/devices.

  • Rate limiting unsupported: indicates missing in-tree qat_rl sysfs ABI.

  • Invalid rate: ensure CIR > 0 and PIR >= CIR.

  • Service invalid/not enabled: verify service is configured on the PF.

Recommended troubleshooting sequence:

  1. Verify module presence (/sys/module/intel_qat).

  2. Verify qat_rl directory for target PF.

  3. Check remaining capacity before allocation (cap_rem or caps).

  4. Re-read SLA state with get after create/update.

Usage Examples

Customer-default workflow (recommended):

sudo ./qat_sla_mgr caps 0000:6b:00.0
sudo ./qat_sla_mgr create 0000:6b:01.0 5000 9000 sym
sudo ./qat_sla_mgr list 0000:6b:00.0

Create, query, and verify:

# Create SLA (VF-based auto RP mask)
sudo ./qat_sla_rl.py add 0000:6b:01.0 -c 62 -p 62 -s sym

# Query created SLA (replace <sla_id> with returned value)
sudo ./qat_sla_rl.py get 0000:6b:00.0 -i <sla_id>

Capacity-first workflow:

sudo ./qat_sla_rl.py caps 0000:6b:00.0
sudo ./qat_sla_rl.py cap_rem 0000:6b:00.0 -s sym

Update existing SLA:

sudo ./qat_sla_rl.py update 0000:6b:00.0 -i <sla_id> -c 80 -p 80

List state:

sudo ./qat_sla_rl.py list 0000:6b:00.0

Remove and validate removal:

sudo ./qat_sla_rl.py rm 0000:6b:00.0 -i <sla_id>
sudo ./qat_sla_rl.py list 0000:6b:00.0

OOT/non-supported path example:

./qat_sla_rl.py add 0000:6b:00.0 -c 62 -p 62 -s sym
# Operation `add` failed with: QAT device with address: `0000:6b:00.0` does not support Rate Limiting

Integration Notes

qat_sla_mgr remains the recommended interface for standard users. qat_sla_rl.py is the low-level implementation that qat_sla_mgr wraps, and should be used directly when you need advanced behavior not exposed by wrapper arguments.

Situations where direct qat_sla_rl.py invocation is typically required:

  • You must provide an explicit ring-pair mask (-r) rather than VF-derived mapping.

  • You need direct permille-level control for debugging or precise internal validation.

  • You are troubleshooting sysfs ABI behavior with step-by-step low-level operations.

  • You are implementing advanced runbooks that rely on qat_sla_rl.py-specific arguments/flows.

In production, prefer:

  • Standard package installation of both tools.

  • Wrapper-based operations for day-1/day-2 administration.

  • Direct qat_sla_rl.py automation for controlled operational runbooks.

Performance Validation

Validating SLA Performance

To ensure your SLA configuration meets performance requirements:

  1. Baseline Testing: Test your application without rate limiting enabled to establish maximum performance.

  2. SLA Testing: Configure SLAs and test with rate limiting enabled.

  3. Performance Comparison: Compare results and adjust SLA values if needed.

Expected Performance Ranges

  • Symmetric Crypto: 90-100% of calculated SLA performance

  • Compression: 90-100% of calculated SLA performance

  • Decompression: May achieve higher throughput than compression (expected behavior)

  • Asymmetric: Performance varies by algorithm complexity

Note

Performance measurements should be taken over at least 1-second intervals for accurate rate limiting assessment.