BabelStream Cascade Plot

A cascade plot of BabelStream data.

This example uses real application data collected with the BabelStream benchmark. However, programming language and platform names have been removed. The original data is available from https://github.com/UoB-HPC/BabelStream.

This example demonstrates a realistic workflow, where the original data does not use the required naming convention, and application efficiencies are not readily available in the dataset.

Load Data into Pandas

# Load performance data from BabelStream results
df = pd.read_csv(performance_csv)

Project Labels into Expected Forms

The p3analysis.data.projection() method can be used to project column names from the original data into names required by the P3 Analysis Library.

df = p3analysis.data.projection(
    df, problem=["name"], platform=["arch"], application=["language"],
)

Our BabelStream data contains only one problem, and the “name” field is always “BabelStream”. Other BabelStream case studies may feature multiple array sizes, which could also be used here.

The platforms can be identified by the “arch” column, which stores an architecture name.

For this case study, we treat each implementation of BabelStream as a different application (consistent with the definition of “application” here). Each implementation of BabelStream is identified by the language it is written in.

Calculate Application Efficiencies

Application efficiency values show which application(s) are most effective at utilizing a given platform. In the below, any row with an ‘app eff’ of 0 represents an application that did not run correctly for a given (problem, platform) combination; any row with an app eff of 1 represents an application achieving the best-known performance.

effs = p3analysis.metrics.application_efficiency(df)
print(effs)

        problem    platform application           fom   app eff
0   BabelStream  Platform A  Language 0  2.050000e+11  0.404722
1   BabelStream  Platform A  Language 1  1.740000e+11  0.476828
2   BabelStream  Platform A  Language 2           NaN  0.000000
3   BabelStream  Platform A  Language 3  8.296800e+10  1.000000
4   BabelStream  Platform A  Language 4  1.070000e+11  0.775402
5   BabelStream  Platform B  Language 0  4.520000e+11  0.632743
6   BabelStream  Platform B  Language 1  3.040000e+11  0.940789
7   BabelStream  Platform B  Language 2           NaN  0.000000
8   BabelStream  Platform B  Language 3  4.440000e+11  0.644144
9   BabelStream  Platform B  Language 4  2.860000e+11  1.000000
10  BabelStream  Platform C  Language 0  2.480000e+11  0.995968
11  BabelStream  Platform C  Language 1  2.500000e+11  0.988000
12  BabelStream  Platform C  Language 2           NaN  0.000000
13  BabelStream  Platform C  Language 3  2.470000e+11  1.000000
14  BabelStream  Platform C  Language 4           NaN  0.000000
15  BabelStream  Platform D  Language 0  1.900000e+11  0.952632
16  BabelStream  Platform D  Language 1  1.810000e+11  1.000000
17  BabelStream  Platform D  Language 2           NaN  0.000000
18  BabelStream  Platform D  Language 3  2.570000e+11  0.704280
19  BabelStream  Platform D  Language 4           NaN  0.000000
20  BabelStream  Platform E  Language 0  2.460000e+11  0.991870
21  BabelStream  Platform E  Language 1  2.440000e+11  1.000000
22  BabelStream  Platform E  Language 2           NaN  0.000000
23  BabelStream  Platform E  Language 3           NaN  0.000000
24  BabelStream  Platform E  Language 4           NaN  0.000000
25  BabelStream  Platform F  Language 0  1.060000e+11  0.859594
26  BabelStream  Platform F  Language 1  9.111700e+10  1.000000
27  BabelStream  Platform F  Language 2           NaN  0.000000
28  BabelStream  Platform F  Language 3           NaN  0.000000
29  BabelStream  Platform F  Language 4           NaN  0.000000
30  BabelStream  Platform G  Language 0  9.760000e+11  1.000000
31  BabelStream  Platform G  Language 1           NaN  0.000000
32  BabelStream  Platform G  Language 2           NaN  0.000000
33  BabelStream  Platform G  Language 3           NaN  0.000000
34  BabelStream  Platform G  Language 4           NaN  0.000000
35  BabelStream  Platform H  Language 0  1.440000e+11  1.000000
36  BabelStream  Platform H  Language 1  1.520000e+11  0.947368
37  BabelStream  Platform H  Language 2  1.500000e+11  0.960000
38  BabelStream  Platform H  Language 3           NaN  0.000000
39  BabelStream  Platform H  Language 4  1.510000e+11  0.953642
40  BabelStream  Platform I  Language 0  5.530000e+11  0.996383
41  BabelStream  Platform I  Language 1  5.570000e+11  0.989228
42  BabelStream  Platform I  Language 2  5.520000e+11  0.998188
43  BabelStream  Platform I  Language 3  5.520000e+11  0.998188
44  BabelStream  Platform I  Language 4  5.510000e+11  1.000000
45  BabelStream  Platform J  Language 0  7.740000e+11  1.000000
46  BabelStream  Platform J  Language 1  8.280000e+11  0.934783
47  BabelStream  Platform J  Language 2  8.330000e+11  0.929172
48  BabelStream  Platform J  Language 3  8.290000e+11  0.933655
49  BabelStream  Platform J  Language 4  8.390000e+11  0.922527
50  BabelStream  Platform K  Language 0  5.280000e+11  1.000000
51  BabelStream  Platform K  Language 1  5.540000e+11  0.953069
52  BabelStream  Platform K  Language 2  5.560000e+11  0.949640
53  BabelStream  Platform K  Language 3  5.550000e+11  0.951351
54  BabelStream  Platform K  Language 4  5.540000e+11  0.953069

Generate a Cascade Plot

cascade = p3analysis.plot.cascade(effs)
cascade.save("cascade.png")

The plot shows the application efficiency (line chart, left) and performance portability (bar chart, right) of BabelStream across 11 platforms (A-I).

The application efficiency for each platform is ranked from highest to lowest, such that platforms on the far left have the highest efficiency, and platforms on the far right have the lowest efficiency. If an application does not run on a platform, no efficiency value will be plotted for that platform in the line chart.

An application must run across all platforms to achieve a non-zero performance portability score. For BabelStream, Language 0 is the only programming model that runs across all 11 platforms, and is the only one without a 0 for performance portability in the bar chart.