YASK

Table of Contents

• Introduction
  • Define a Stencil Solution and Generate Kernel Code
  • Create a Stencil Kernel Library and Stencil-based Application
• YASK Stencil Compiler API
  • Compiler Overview
  • Typical Program Flow using the Compiler API
• YASK Stencil Kernel API
  • Kernel Overview
  • Typical Program Flow using the Kernel API
• Example Tests

Introduction

For an overview of YASK, see the YASK tutorial.

The typical high-level YASK workflow is as follows:

1. Define a Stencil Solution and Generate Kernel Code
   • Define a stencil solution using the YASK domain-specific language (DSL).
   • Use the YASK stencil compiler to generate C++ kernel code from the stencil solution.
2. Create a Stencil Kernel Library and Stencil-based Application
   • Compile the generated C++ kernel code to create a YASK kernel library.
   • Create and use a stencil-based application using the kernel library.
3. Test and deploy your new YASK-enabled application.

There are two sets of APIs provided by YASK corresponding to the first two tasks:

1. The YASK Stencil Compiler API (available in C++ and Python).
2. The YASK Stencil Kernel API (available in C++ and Python).

For each of the tasks, you can either use the YASK-provided application or create your own application built with the corresponding API.

These alternatives may be mixed-or-matched in all combinations. For example, you can use the YASK-provided stencil compiler to generate a YASK kernel library and then use that library via the kernel API to create your own stencil-based Python application.

The following sub-sections describe each of the tasks in the workflow and when the APIs may be used.

Define a Stencil Solution and Generate Kernel Code

A stencil solution consists of the variables containing the problem data and one or more equations that describe how points in the variables are calculated. An equation consists of a point to be calculated that "EQUALS" an expression consisting of other points, constants, mathematical operators, etc.

• Example equation: u(t+1, x, y) EQUALS (u(t, x, y) + u(t, x+1, y) + u(t, x, y+1)) / 3.

A new stencil solution may be defined in one of the following ways:

1. Write a stencil in C++ that will be compiled into the standard YASK stencil compiler utility, bin/yask_compiler.exe.
   • This approach is typically taken when the programmer or scientist wants to express the equations of the stencil directly, e.g., in a text editor or IDE.
   • In this case, you must extend the provided base class yc_solution_base (or yc_solution_with_radius_base). Equations can be typed exactly as shown in the above example (C++ operator overloading is used automatically to construct an internal representation of each equation), and they must be part of the code accessed via the overloaded method yc_solution_base::define(). All classes and methods used to create expressions in the YASK compiler API may be used.
   • To generate the kernel code for the defined stencil, [re]compile and run the YASK stencil compiler. This is done automatically when building a YASK kernel using the YASK Makefile, or you can build it explicity via make compiler and run it from your shell command-prompt.
   • When you run make to build the kernel library, the generated output from the compiler utility is automatically written to a pre-determined temporary file and compiled into the optimized kernel library.
   • See example stencils of this type in src/stencils.
2. Create a custom application that defines stencils.
   • This approach is typically taken when a 3rd-party front-end tool will be creating stencils automatically from some other existing format or description.
   • In this case, you aren't required to use the provided yc_solution_base class or overload yc_solution_base::define(), but equations may be expressed in the same way, because both techniques use the YASK compiler API.
   • The output from the custom application may be written to any file that is then specified via YK_CODE_FILE=filename when running make to build the kernel library.
   • See src/compiler/tests/yask_compiler_api_test.cpp for an example stencil definition in C++ or src/compiler/tests/yask_compiler_api_test.py for an example stencil definition in Python.

In either case, the resulting generated code should written to the C++ stencil-code file to be compiled into the kernel library.

Create a Stencil Kernel Library and Stencil-based Application

Once the stencil-code file is created, it must be compiled into a YASK kernel library.

• There will be a separate library created for each stencil solution and target architecture combination.
• This is done automatically when building a YASK kernel using the YASK Makefile, e.g., via make -j stencil=iso3dfd arch=knl, which builds the "iso3dfd" stencil for the Intel Xeon Phi processor.
• If you generated the stencil-code file via a custom application, you may still create a kernel library using the Makefile; you'll just need to specify the filename via YK_CODE_FILE=filename as described above. You'll also still need to specify the target architecture and give a descriptive name to the stencil. Example: make stencil=my-stencil arch=hsw kernel-only YK_CODE_FILE=my_stencil_code.hpp.

If make is invoked as in one of the above examples, it will create the kernel library as lib/libyask_kernel.stencil.arch.so, where stencil and arch match the corresponding variables provided during make. (If you want the file named differently, you can override stencil with YK_STENCIL=stencil_name and/or arch with YK_ARCH=arch_name.)

To use the kernel library, an executable must be created from it. This may be done in one of the following ways:

1. The default YASK kernel executable will automatically be created as bin/yask_kernel.stencil.arch.exe when make is invoked as in the above examples.
   • This application may be run via bin/yask.sh to obtain a performance measurement of the kernel.
2. You can also create your own stencil application using the YASK stencil kernel API.
   • This approach would be taken to integrate the kernel into a larger application that would inject real-world initial-state data into the YASK variables and extract the final-state data for analysis or further processing.
   • See src/kernel/tests/yask_kernel_api_test.cpp for an example kernel usage in C++ or src/kernel/tests/yask_kernel_api_test.py for an example kernel usage in Python.
   • See YASK Stencil Kernel API for documentation on the kernel API.

Note

Anytime you want to change any compile-time properties of the kernel (like vector-folding or prefetching), be sure to run make clean to force the removal of all kernel-specific intermediate code. Otherwise, you will likely see some unexpected errors when building the new kernel.

YASK Stencil Compiler API

Compiler Overview

This section provides usage information for the YASK stencil compiler API (application-programmer interface).
The API is available for C++ and for Python via SWIG. Type names are prefixed with 'yc_' to indicate "YASK compiler"; this distinguishes them from the 'yk_'-prefixed types used in the "YASK kernel" API.

The types, classes, and functions are listed in YASK Compiler.

Typical Program Flow using the Compiler API

• If using the provided YASK stencil compiler utility, bin/yask_compiler.exe:
  • Extend the provided base class yc_solution_base (or yc_solution_with_radius_base).
  • Create index variables for solution domain dimensions via yc_node_factory::new_domain_index() and via yc_node_factory::new_step_index() for the step dimension (e.g., "t" for time).
  • Create an instance of the class to make it visible to the compiler utility.
  • See the examples in src/stencils showing how to define solutions from these base classes.
• If writing your own YASK compiler:
  • Create a yc_factory. This is the "bootstrap" object that will be used to create others.
  • Create a yc_solution object via yc_factory::new_solution().
  • Create index variablse for solution domain dimensions via yc_node_factory::new_domain_index() and via yc_node_factory::new_step_index() for the step dimension (e.g., "t" for time).
  • Create one or more yc_var objects via yc_solution::new_var() as needed by the stencil(s) being implemented. Calls to yc_solution::new_var() specify the name and dimensions of each var. Vars may be read-only (constants) or read-write. Each var will be a part of the solution from which it was created. (Or, you may construct a yc_var_proxy object, which is a wrapper around a yc_var pointer providing some syntactic alternatives that you may prefer.)
• Create an equation for each read-write var.
  • Think of expressions as abstract syntax trees (ASTs) that describe the mathematical formula that defines each point to be calculated at each new time-step.
  • Leaf nodes may be floating-point (FP) constants or references to var points.
  • Constants may created via yc_node_factory::new_number_node(), but this is seldom needed explicitly thanks to operator overloading.
  • References to points are created via yc_var::new_var_point(), which specifies the indices relative to the point being evaluated within the problem domain. (If you created yc_var_proxy objects, see the alternative syntax shown in the documentation for that class.)
  • Create operator nodes via normal math operators (+, *, etc.), to build up larger expressions. Some math functions like square-root and cosine are also available.
  • To complete each equation, use the special EQUALS operator to specify an expression on the right-hand side (RHS) and the point in a variable that is defined to be equal to it on the left-hand side (LHS).
  • Boolean expressions may be added to equations to specify sub-domains in which they are applied. In this way, boundary layers and conditions may be implemented. It is also possible to add boolean expressions to specify certain time-steps in which equations are valid.
  • Many examples of using the YASK compiler API to create stencil equations are provided in the src/stencils directory.
• Specify the number of bytes in a floating-point element via yc_solution::set_element_bytes(). This should be 4 or 8. If you are using the provided YASK compiler utility, this is controlled via the real_bytes=4|8 make parameter.
• Optionally specify the vector-folding and/or vector-clustering via yc_solution::set_fold_len() and/or yc_solution::set_cluster_mult(). If you are using the provided YASK compiler utility, these are controlled via the fold=fold-spec and cluster=cluster-spec make parameters.
• Set the target architecture via yc_solution::set_target(). If you are using the provided YASK compiler utility, this is controlled via the arch=arch-name make parameter.
• Write the solution code file via yc_solution::output_solution(). If you are using the provided YASK compiler utility, this call will be done for you with the proper parameters based on the target architecture.

YASK Stencil Kernel API

As discussed earlier, the kernel API is only needed for integrating a YASK kernel into your final application. For evaluating performance, use the provided bin/yask.sh utility, which makes the needed calls to the kernel API for you.

Kernel Overview

This section provides usage information for the YASK stencil kernel API (application-programmer interface).
The API is available for C++ and for Python via SWIG. Type names are prefixed with 'yk_' to indicate "YASK kernel"; this distinguishes them from the 'yc_'-prefixed types used in the "YASK compiler" API.

The types, classes, and functions are listed in YASK Kernel.

Typical Program Flow using the Kernel API

• Create a yk_factory. This is the "bootstrap" object that will be used to create others.
• Create a yk_env object via yk_factory::new_env(). This initializes MPI if you have enabled it.
• Create a yk_solution object via yk_factory::new_solution(). A pointer to the environment object is provided. This structure allows you to initialize MPI early in the program. The solution contains the vars and equations defined by the stencil compiler process.
• Use the yk_solution methods to configure options like the size of the problem domain.
• Set up internal data structures via yk_solution::prepare_solution(). This adjusts any settings as needed, determines each rank's position in the problem if MPI is enabled, and allocates data for the vars. Be sure to modify any settings before calling yk_solution::prepare_solution().
• Access the solution vars via yk_solution::get_var(). Various properties of each var may be retrieved or set. In particular, you should initialize the data in each var via yk_var::set_all_elements_same() and yk_var::set_element().
• Apply the stencil(s) to the vars via yk_solution::run_solution(). There are versions for advancing one or more steps.
• Retrieve the final results via yk_var::get_element().

Example Tests

The following examples illustrate possible combinations of compilers and kernels and show how to invoke a set of tests for each combination.

• You can add stencil=_stencil-name_ to use a specific stencil for testing.
• You can add arch=_arch-name_ to target one of the architectures listed in the Makefile if desired.
• Run make clean before all of the example commands to ensure consistent builds.

Stencil Compiler	Stencil Application	Test Command
YASK-provided	YASK-provided	make -j yc-and-yk-test
YASK-provided	C++ test example	make -j yc-and-cxx-yk-api-test
YASK-provided	Python test example	make -j yc-and-py-yk-api-test
C++ test example	YASK-provided	make -j cxx-yc-api-and-yk-test
C++ test example	C++ test example	make -j cxx-yc-api-and-cxx-yk-api-test
C++ test example	Python test example	make -j cxx-yc-api-and-py-yk-api-test
Python test example	YASK-provided	make -j py-yc-api-and-yk-test
Python test example	C++ test example	make -j py-yc-api-and-cxx-yk-api-test
Python test example	Python test example	make -j py-yc-api-and-py-yk-api-test