Kernel caching

Kernel caching#

It might be necessary to recreate the same plan multiple times. To facilitate fast plan creation, kernels might be cached and looked up at plan creation.

JIT cache interface#

The general interface of a JIT cache is defined below. Users may derive from jit_cache in order to implement their own caching strategies should the provided caching strategies be insufficent.

class jit_cache#

Interface for jit_caches.

Template Parameters:

backend-specific – kernel bundle type

Subclassed by bbfft::aot_cache, bbfft::jit_cache_all

Public Functions

virtual ~jit_cache()#

Destructor.

virtual auto get(jit_cache_key const &key) const -> shared_handle<module_handle_t> = 0#

Get FFT kernel bundle.

Parameters:

key – FFT kernel identifier

Returns:

kernel bundle

virtual void store(jit_cache_key const &key, shared_handle<module_handle_t> mod) = 0#

Store FFT kernel bundle.

Parameters:

  • key – FFT kernel identifier

  • mod – kernel bundle

Cache keys#

struct jit_cache_key#

Unique identifier for fft kernel.

Public Functions

bool operator==(jit_cache_key const &other) const#

equality check

Public Members

std::string kernel_name = {}#

Name of the OpenCL kernel.

std::uint64_t device_id = std::numeric_limits<std::uint64_t>::max()#

Unique device id.

struct jit_cache_key_hash#

Hash function for jit_cache_key.

Public Functions

std::size_t operator()(jit_cache_key const &key) const noexcept#

Compute hash.

Parameters:

key – cache key

Returns:

hash

JIT cache all#

Simple cache that stores all encountered kernels.

class jit_cache_all : public bbfft::jit_cache#

Cache that stores all kernels.

Public Functions

virtual auto get(jit_cache_key const &key) const -> shared_handle<module_handle_t> override#

Get FFT kernel bundle.

Parameters:

key – FFT kernel identifier

Returns:

kernel bundle

virtual void store(jit_cache_key const &key, shared_handle<module_handle_t> mod) override#

Store FFT kernel bundle.

Parameters:

  • key – FFT kernel identifier

  • mod – kernel bundle

auto kernel_names() const -> std::vector<std::string>#

Get all kernel names stored in this cache.

Ahead-of-time cache#

Kernels may be compiled ahead-of-time. The ahead-of-time “cache” is used to look-up kernels at plan creation time.

class aot_cache : public bbfft::jit_cache#

Cache to look up ahead-of-time compiled FFT kernels.

Public Functions

virtual auto get(jit_cache_key const &key) const -> shared_handle<module_handle_t> override#

Get FFT kernel bundle.

Parameters:

key – FFT kernel identifier

Returns:

kernel bundle

virtual void store(jit_cache_key const &key, shared_handle<module_handle_t> mod) override#

Store FFT kernel bundle.

Parameters:

  • key – FFT kernel identifier

  • mod – kernel bundle

void register_module(aot_module aot_mod)#

register module with this cache

struct aot_module#

Collection of native module handle, set of kernel names stored in the native module, and a device identifier.

Public Members

shared_handle<module_handle_t> mod#

Native module handle.

std::unordered_set<std::string> kernel_names#

Set of kernel names.

std::uint64_t device_id#

Device id for mod.