BabelStream Navigation Chart

A navigation chart of BabelStream data.

This example uses real application data collected with the BabelStream benchmark. However, programming language and platform names have been removed. The original data is available from https://github.com/UoB-HPC/BabelStream.

Load Data into Pandas

# Load data from BabelStream results
df = pd.read_csv(performance_csv)
df_cov = pd.read_csv(coverage_csv)

Note that, unlike a cascade plot, plotting a navigation chart requires both performance and coverage data.

Project Labels into Expected Forms

The p3analysis.data.projection() method can be used to project column names from the original data into names required by the P3 Analysis Library.

df = p3analysis.data.projection(
    df, problem=["name"], platform=["arch"], application=["language"],
)

Our BabelStream data contains only one problem, and the “name” field is always “BabelStream”. Other BabelStream case studies may feature multiple array sizes, which could also be used here.

The platforms can be identified by the “arch” column, which stores an architecture name.

For this case study, we treat each implementation of BabelStream as a different application (consistent with the definition of “application” here). Each implementation of BabelStream is identified by the language it is written in.

Calculate P3 Metrics

Application Efficiency

Application efficiency values show which application(s) are most effective at utilizing a given platform. In the below, any row with an ‘app eff’ of 0 represents an application that did not run correctly for a given (problem, platform) combination; any row with an app eff of 1 represents an application achieving the best-known performance.

effs = p3analysis.metrics.application_efficiency(df)
print(effs)

        problem    platform application           fom   app eff
0   BabelStream  Platform A  Language 0  2.050000e+11  0.404722
1   BabelStream  Platform A  Language 1  1.740000e+11  0.476828
2   BabelStream  Platform A  Language 2           NaN  0.000000
3   BabelStream  Platform A  Language 3  8.296800e+10  1.000000
4   BabelStream  Platform A  Language 4  1.070000e+11  0.775402
5   BabelStream  Platform B  Language 0  4.520000e+11  0.632743
6   BabelStream  Platform B  Language 1  3.040000e+11  0.940789
7   BabelStream  Platform B  Language 2           NaN  0.000000
8   BabelStream  Platform B  Language 3  4.440000e+11  0.644144
9   BabelStream  Platform B  Language 4  2.860000e+11  1.000000
10  BabelStream  Platform C  Language 0  2.480000e+11  0.995968
11  BabelStream  Platform C  Language 1  2.500000e+11  0.988000
12  BabelStream  Platform C  Language 2           NaN  0.000000
13  BabelStream  Platform C  Language 3  2.470000e+11  1.000000
14  BabelStream  Platform C  Language 4           NaN  0.000000
15  BabelStream  Platform D  Language 0  1.900000e+11  0.952632
16  BabelStream  Platform D  Language 1  1.810000e+11  1.000000
17  BabelStream  Platform D  Language 2           NaN  0.000000
18  BabelStream  Platform D  Language 3  2.570000e+11  0.704280
19  BabelStream  Platform D  Language 4           NaN  0.000000
20  BabelStream  Platform E  Language 0  2.460000e+11  0.991870
21  BabelStream  Platform E  Language 1  2.440000e+11  1.000000
22  BabelStream  Platform E  Language 2           NaN  0.000000
23  BabelStream  Platform E  Language 3           NaN  0.000000
24  BabelStream  Platform E  Language 4           NaN  0.000000
25  BabelStream  Platform F  Language 0  1.060000e+11  0.859594
26  BabelStream  Platform F  Language 1  9.111700e+10  1.000000
27  BabelStream  Platform F  Language 2           NaN  0.000000
28  BabelStream  Platform F  Language 3           NaN  0.000000
29  BabelStream  Platform F  Language 4           NaN  0.000000
30  BabelStream  Platform G  Language 0  9.760000e+11  1.000000
31  BabelStream  Platform G  Language 1           NaN  0.000000
32  BabelStream  Platform G  Language 2           NaN  0.000000
33  BabelStream  Platform G  Language 3           NaN  0.000000
34  BabelStream  Platform G  Language 4           NaN  0.000000
35  BabelStream  Platform H  Language 0  1.440000e+11  1.000000
36  BabelStream  Platform H  Language 1  1.520000e+11  0.947368
37  BabelStream  Platform H  Language 2  1.500000e+11  0.960000
38  BabelStream  Platform H  Language 3           NaN  0.000000
39  BabelStream  Platform H  Language 4  1.510000e+11  0.953642
40  BabelStream  Platform I  Language 0  5.530000e+11  0.996383
41  BabelStream  Platform I  Language 1  5.570000e+11  0.989228
42  BabelStream  Platform I  Language 2  5.520000e+11  0.998188
43  BabelStream  Platform I  Language 3  5.520000e+11  0.998188
44  BabelStream  Platform I  Language 4  5.510000e+11  1.000000
45  BabelStream  Platform J  Language 0  7.740000e+11  1.000000
46  BabelStream  Platform J  Language 1  8.280000e+11  0.934783
47  BabelStream  Platform J  Language 2  8.330000e+11  0.929172
48  BabelStream  Platform J  Language 3  8.290000e+11  0.933655
49  BabelStream  Platform J  Language 4  8.390000e+11  0.922527
50  BabelStream  Platform K  Language 0  5.280000e+11  1.000000
51  BabelStream  Platform K  Language 1  5.540000e+11  0.953069
52  BabelStream  Platform K  Language 2  5.560000e+11  0.949640
53  BabelStream  Platform K  Language 3  5.550000e+11  0.951351
54  BabelStream  Platform K  Language 4  5.540000e+11  0.953069

Code Divergence

Code divergence values show how much code is re-used across the platforms targeted by a specific application. For BabelStream, only Language 0 has a non-zero divergence, as it is the only implementation containing different code paths for different platforms.

div = p3analysis.metrics.divergence(df, df_cov)
print(div)

       problem application  divergence
0  BabelStream  Language 0    0.057906
1  BabelStream  Language 1    0.000000
2  BabelStream  Language 2    0.000000
3  BabelStream  Language 3    0.000000
4  BabelStream  Language 4    0.000000

Performance Portability

Performance portability values show the average efficiency achieved when using all platforms in the set. An application must run across all platforms to achieve a non-zero performance portability score. For BabelStream, Language 0 is the only programming model that runs across all 11 platforms.

pp = p3analysis.metrics.pp(effs)
print(pp)

       problem application    app pp
0  BabelStream  Language 0  0.828301
1  BabelStream  Language 1  0.000000
2  BabelStream  Language 2  0.000000
3  BabelStream  Language 3  0.000000
4  BabelStream  Language 4  0.000000

Generate a Navigation Chart

navchart = p3analysis.plot.navchart(pp, div)
navchart.save("navchart.png")

The plot shows the performance portability and code convergence values for each implementation of BabelStream. Code convergence is simply (1 – code divergence).

The top-right corner of a navigation chart represents the ideal, where an application achieves the best performance across all platforms of interest using a single source code. The top-left corner represents an application that achieves the best performance but without reusing any code across platforms. Any points along the x-axis are unportable (i.e. there is at least one platform of interest where it does not run).

For BabelStream, Language 0 is the only implementation that runs across all platforms and is therefore the only point not on the x-axis. All other implementations are plotted at (1, 0) because although they use exactly the same source code to target all platforms, there is at least one platform where they do not run.