Memory Driver APIs

These APIs provide memory management functionality for allocating DMA-able memory, address translation, and IOVA mapping for QAT operations.

All function definitions are located in: qae_mem.h

Core Memory Management

qaeMemAlloc

When used in user space, allocates memsize bytes of virtual memory. When used in kernel space, allocates memsize bytes of contiguous and pinned memory.

Syntax

void *qaeMemAlloc(size_t memsize);

Parameters

memsize

The amount of memory in bytes to be allocated.

Return Value

Pointer to the allocated memory or NULL if the allocation failed.

qaeMemFree

Frees memory allocated by the qaeMemAlloc function. Applicable for both user and kernel spaces.

Syntax

void qaeMemFree(void **ptr);

Parameters

**ptr

Address of the pointer to the memory to be freed.

Return Value

None.

Note

Pre-condition: *ptr points to memory previously allocated by qaeMemAlloc.

Post-condition: Memory is freed and pointer value is set to NULL.

qaeMemAllocNUMA

Allocates and returns virtual memory mapped to pinned, contiguous physical memory aligned to phys_alignment_byte. This API enables user to choose a CPU node nearest to QAT device. This API is applicable for both user and kernel spaces. Based on the address space used, memory mapped from corresponding virtual address space will be returned.

Syntax

void *qaeMemAllocNUMA(
    size_t size,
    int node,
    size_t phys_alignment_byte);

Parameters

`size`	A non-zero value representing the amount of memory in bytes to be allocated. It cannot exceed `QAE_MAX_ALLOC_SIZE`.
`node`	NUMA node.
`phys_alignment_byte`	A non-zero value representing memory boundary alignment in bytes. It must be in powers of 2 not exceeding 4MB.

Return Value

Pointer to the allocated memory or NULL if the allocation failed.

Note

In user space, NUMA_ANY_NODE (-1) can be used to select any available NUMA node.

qaeMemFreeNUMA

Frees memory allocated by the qaeMemAllocNUMA function. Applicable for both user and kernel spaces.

Syntax

void qaeMemFreeNUMA(void **ptr);

Parameters

**ptr

Address of pointer to the memory to be freed.

Return Value

None.

Note

Pre-condition: *ptr points to memory previously allocated by qaeMemAllocNUMA.

Post-condition: Memory is freed and the pointer value is set to NULL.

qaeMemFreeNonZeroNUMA

Frees memory allocated by qaeMemAllocNUMA function. Does not clear the memory region before freeing unlike other API. Applicable for user space only.

Syntax

void qaeMemFreeNonZeroNUMA(void **ptr);

Parameters

**ptr

Address of pointer to the memory to be freed.

Return Value

None.

Note

Pre-condition: *ptr points to memory previously allocated by qaeMemAllocNUMA.

Post-condition: Memory is freed and the pointer value is set to NULL.

Performance consideration: This function does not zero the memory before freeing, which may provide better performance when the memory content does not need to be cleared for security reasons.

Address Translation

qaeVirtToPhysNUMA

Converts a virtual address provided by qaeMemAllocNUMA to a physical one. Applicable for both user and kernel spaces.

Syntax

uint64_t qaeVirtToPhysNUMA(void *pVirtAddr);

Parameters

*pVirtAddr

Pointer to the virtual address.

Return Value

Pointer to the physical address or 0 (NULL) on error.

Note

Pre-condition: pVirtAddr points to memory previously allocated by qaeMemAllocNUMA.

qaePhysToVirtNUMA

Translates a physical address to a virtual one of a memory allocated by qaeMemAllocNUMA() function. Applicable for user space.

Syntax

void *qaePhysToVirtNUMA(uint64_t physAddress);

Parameters

physAddress

Physical address.

Return Value

Pointer to the virtual address or 0 (NULL) on error.

Note

Pre-condition: physAddress points to memory previously allocated by qaeMemAllocNUMA.

VFIO Device Management

qaeRegisterDevice

Registers a VFIO device file descriptor.

Syntax

int qaeRegisterDevice(int fd);

Parameters

`fd`	File descriptor.

Return Value

Returns 0 on success.

qaeUnregisterDevice

Unregisters a VFIO device file descriptor.

Syntax

int qaeUnregisterDevice(int fd);

Parameters

`fd`	File descriptor.

Return Value

Returns 0 on success.

Advanced IOVA Mapping

qaeMemMapContiguousIova

Allocates a contiguous IOVA (I/O Virtual Address) region and maps the provided user-allocated virtual memory to this IOVA for DMA operations with QAT hardware. This function allows users to use their own memory buffers (e.g., pre-allocated buffers) for zero-copy DMA transfers.

The memory region is mapped with both read and write permissions for DMA operations. The IOVA is aligned to IOVA_SLAB_SIZE (2MB) internally.

After successful mapping, qaeVirtToPhysNUMA() can be used to translate addresses within the mapped region to their corresponding IOVA.

This function is thread-safe and not supported when ICP_THREAD_SPECIFIC_USDM is defined.

Syntax

uint64_t qaeMemMapContiguousIova(void *virt, size_t size);

Parameters

*virt

Pointer to the user-allocated virtual memory to map. Must be a valid, non-NULL, accessible virtual address. Must be passed to qaeMemUnmapContiguousIova() when done. Alignment requirements:

When hugepages enabled: must be 2MB-aligned
When hugepages disabled: must be 4KB-aligned

size

Size in bytes of the memory region to map. Must be greater than 0, a multiple of page size (4KB), and not exceed 2GB (2147483648 bytes). When hugepages are enabled, size must be 2MB-aligned.

Return Value

Note

Pre-conditions:

qaeRegisterDevice() must have been called prior to using this API for the DMA mapping to be effective.
The virtual address provided must be valid and accessible.
The virtual memory region must remain valid and not be freed while the IOVA mapping is active.

Post-condition: On success, the virtual memory is mapped to a per-process IOMMU domain starting with the returned IOVA and can be used for DMA operations with QAT hardware. The mapping is also registered in the USDM page table for qaeVirtToPhysNUMA() lookups.

Important considerations:

This function is only available in VFIO mode and will fail in no-IOMMU mode.
Memory allocated through qaeMemAllocNUMA() should NOT be mapped using this API as it is already managed by USDM internally.
Alignment requirements are critical for correct operation. The page table granularity depends on whether hugepages are enabled.
This function may fail due to IOVA address space fragmentation even if there is sufficient total free IOVA space.
This function may also fail if there are too many concurrent mappings. For example, VFIO might fail if the maximum number of allowed mappings is exceeded (this is 65535 with iommu type 1).

Failure conditions:

This function is not supported (e.g. in the out-of-tree driver, or when ICP_THREAD_SPECIFIC_USDM is defined)
virt is NULL
virt is not properly aligned (4KB when hugepages disabled, 2MB when hugepages enabled)
Size is 0, not a multiple of page size (4KB), or exceeds 2GB
Size is not 2MB-aligned when hugepages are enabled
No contiguous IOVA space is available (fragmentation)
VFIO container is not registered (qaeRegisterDevice not called)
VFIO IOMMU mapping failed
VFIO is configured in no-IOMMU mode (not supported for this API)

qaeMemUnmapContiguousIova

Unmaps a previously mapped IOVA region and releases the IOVA address space. This function should be called when the user-mapped memory is no longer needed for DMA operations.

This function reverses the operation performed by qaeMemMapContiguousIova(). It removes the IOMMU mapping via VFIO, clears the page table entries, and releases the IOVA address space in the global bitmap for reuse.

The IOVA is automatically looked up from the page table using the provided virtual address, so callers do not need to track the IOVA returned by qaeMemMapContiguousIova().

This function is thread-safe and not supported when ICP_THREAD_SPECIFIC_USDM is defined.

Syntax

int qaeMemUnmapContiguousIova(void *virt, size_t size);

Parameters

`*virt`	Pointer to the virtual memory that was mapped. This must be the same pointer passed to `qaeMemMapContiguousIova()`. Must not be NULL. Must meet the same alignment requirements as `qaeMemMapContiguousIova()` (4KB or 2MB based on hugepage configuration).
`size`	Size in bytes of the memory region. Must be greater than 0, a multiple of page size (4KB), and match the size used when `qaeMemMapContiguousIova()` was called. When hugepages are enabled, size must be 2MB-aligned.

Return Value

`0`	Success - the IOVA was unmapped and released.
`1`	Failure - could not unmap the IOVA.

Note

Pre-conditions:

The memory must have been previously mapped via qaeMemMapContiguousIova() and no DMA operations should be in progress.
The size parameter must match the size used during mapping.

Post-condition: The IOVA is unmapped and the address space is released for reuse. The user’s virtual memory is not affected and remains valid. qaeVirtToPhysNUMA() will no longer return valid results for this region.

Important:

This function is only available in VFIO mode and will fail in no-IOMMU mode.
Do NOT use this function to unmap memory that was allocated through qaeMemAllocNUMA() - use qaeMemFreeNUMA() instead.

Failure conditions:

This function is not supported (e.g. in the out-of-tree driver, or when ICP_THREAD_SPECIFIC_USDM is defined)
virt is NULL
virt is not properly aligned (4KB when hugepages disabled, 2MB when hugepages enabled)
Size is 0 or not a multiple of page size (4KB)
Size is not 2MB-aligned when hugepages are enabled
The virtual address was not previously mapped
VFIO IOMMU unmapping failed
VFIO is configured in no-IOMMU mode

Utility Functions

qaeAtFork

Must be called when child process is forked to adjust the kernel memory map page. User space only.

Syntax

void qaeAtFork(void);

Parameters

None.

Return Value

None.

qaeMemInitAndReturnFd

Returns the file descriptor obtained by qaeMemInit. FreeBSD only.

Syntax

int qaeMemInitAndReturnFd(int *mem_fd);

Parameters

*mem_fd

Address of fd which is updated by qaeMemInit.

Return Value

Status from qaeMemInit. Returns 0 if the open of the device was successful and non-zero otherwise.

Note

Pre-condition: File /dev/qae_mem is opened successfully.