2.7 Probes

Simics' Probes framework provides a generic mechanism for measuring counters and other statistics in Simics. The probe measurement points are distributed across the entire simulated system.

Probes can be related to target processors, device models, the host system (where Simics runs), instrumentation tools etc. There are also probes which present sums of other probes in the system on cell or global level.

Currently, most probes have been added for the purpose of measuring and understanding the performance of Simics. Additional probes could focus more on the target hardware and software aspects.

A probe offers a simple way to extract the value, and also contains type information and formatting hints on the probe itself to present the value of the probe, in an uniform and user-readable format.

A probe is identified by a probe-kind, which is a string with a hierarchical namespaces separated with dots (.). For example, cpu.cycles would be found in each processor model in the system and represents how many cycles that has elapsed on the specific processor. The cell.cycles probes are related to the cell objects and this returns the sum of executed cycles of processors which are located under the specific cell. Finally, sim.cycles is not related to any object but form a singleton sum of all the processor's cycles elapsed.

Probes can also be derived from other probes, offering some calculated metric. For example sim.slowdown is calculated as the fraction between sim.time.wallclock and sim.time.virtual, providing an indication how fast the virtual time elapsed compared to the real wallclock time on the system which Simics runs on.

Simics itself is aware of all probes that currently exist in the system, allowing commands to list the probes or read out their values from Simics command line interface.

Probes may either be explicitly implemented in the various Simics objects, or can be implicitly created by the probes framework itself. For example, Simics may detect that a new object is created that implements the cycle interface and then automatically create probes for accessing the new object's cycle information.

2.7.1 Probes Command Set

To look at the available probes, the probe-feature must first be enabled, which can be done with the enable-probes command. When the feature is enabled all probes in the system are detected and/or implicitly created. Also, any further creation or deletion of Simics objects is monitored, keeping the list of known probes in sync with the simulated system.

When the probes framework is enabled, a singleton object called probes is created, which holds all commands to examine probes. For example the probes.list-kinds command gives an overview of the probes that is available in the system, and the amount of objects for each probe-kind. See figure 22 for some examples.

The probes.read allows one or several probes values to be read out and displayed. The command can also be used in in Simics scripts to assign cli-variables. If a single probe is read, its value will be returned, while using probe-kind will return a list of tuples [(probe, value)*] that was read out.

Some probes are not active by default, typically because there could be overhead in maintaining their values. To activate a probe the probes.subscribe can be used. As long as there are at least one subscriber on a probe, it will be active, and when the number of subscribers goes down to zero, the probe becomes deactivated again.

Figure 24 shows some examples where probes first need to be subscribed in order to be read later.

Default the read command returns a string formatted in a user-readable format, according to the probe's properties. To get the actual value of the probe the -values switch can be used,

The -raw-value flag returns internal representation of advanced types. This can be useful for understanding the source-values for a calculated value.

2.7.2 Probe Samplers

Probe samplers are a group of tools which use sampling during the simulation, to read user-configurable probes and displays their values. This captures deviations of the probes during the simulation, compared to just looking at the end result when the simulation is finished.

Probe samplers are a group of tools which read user-configurable probes during simulation, through sampling. This captures deviations of the probes during the simulation, compared to just looking at the end result when the simulation is finished.

By default, added probes are all sampled and printed out during run-time with one row representing each sample, each column representing a particular probe.

With many probes sampled, the table can become very wide, and hard to read. To avoid this, a probe can be added with special flags:

The result of the measurements can be saved to csv or json formats. There are also plot capabilities built in the tools when connected to Simics-client.

There are three different samplers, with very similar command sets. To create one of these samplers, the new-probe-monitor, new-probe-streamer or new-system-perfmeter is used. Since the command interface for these tools are largely identical, the probe-monitor chapter explains details which are generally applicable.

2.7.2.1 Probe Monitor

The probe-monitor tool offers a way to read out and present the probe values in continuous samples, while the simulation is running.

All probes being monitored will be presented in a table row for each sample taken, allowing detection of abnormal values during the simulation.

It is possible to add lots of probes to the same probe-monitor, but as the number of probes increase, so does the width of the produced table. Therefore, it can be useful to create several probe-monitors, each showing some group of related feature.

To create a probe-monitor use the new-probe-monitor command. This will create a new object such as pm0 which has some additional commands. See figure 25 for an example.

Here, first a probe-monitor is created, that gets the name pm0 by default. The probe-monitor samples the probes each wallclock seconds, which is controlled by the mode and interval arguments.

The -summary flag means all probe values will be displayed when the simulation stops, providing the session or current value for the probes subscribed to, described later.

Next, the pm0.add-probe is issued with two probe-kinds using the mode session. If mode is not specified, it defaults to delta meaning the value presented for the probe is the difference from the previous sample. That is, it shows the value for the sample only. With the session mode, it instead shows the total value since the probe was added to the monitor. In this case, the accumulated times, both in host wall-clock and the virtual time of the simulated system is displayed.

Apart from session and delta there is also the current mode which can be selected, which displays the plain probe value, without any adjustments. The current mode is interesting for some probes, which are not monotonically increasing, such as a temperature. It can also be interesting to be used for probes derived from checkpointed state, for example the global virtual time, when the user wants to see the absolute time.

Finally, before the execution is started we add more probes, now using the default delta mode. Here we are mixing some singleton probes sim.* and some processor specific probes cpu.*. Since there are two processors in the simulated system used in this example, the number of all monitored probes is 14.

When the simulation is started, the probe-monitor starts to print out table rows for the probes being measured, one second between each sample. Probes which have an object associated will be sorted under that object, so here probes for the two processors have an additional heading showing where the probes belong to.

Some more detailed information on each column or probe, that was monitored in this example.

Session Wallclock (hh:mm:ss) - sim.time.wallclock
This is basically the host time that has elapsed during the simulation. Note that this time stands still if Simics is not executing. If the true wallclock is desired (which increments even when Simics is not simulating), the probe host.time.wallclock can be used instead. The time here is presented as hours, minutes and second format.
Session Virtual-Time (hh:mm:ss) - sim.time.virtual
This probe returns the virtual time of the simulation. The virtual time for all processors are pretty much in sync, but there can be small differences. The probe simply selects the first processor that it finds and presents that. Similar to wall clock, and all session or current times are presented in hours, minutes and second format.
Sched (s) - sim.time.schedule
This is another aspect of time in Simics, namely how much time that Simics scheduler have requested the different processor to be scheduled in total. In a multi-threaded simulation this time can be much greater than the wallclock time, since each execution thread can run in parallel. The monitor presents any delta times with a second metric-prefix, giving much higher resolution.
Slowdown - sim.slowdown
The slowdown probes gives an measurement on how fast virtual time elapses compared to wallclock time. Any value below one means that the virtual time goes faster then the host wallclock. This provides a hint on how fast the simulation is.

A value of 0.5 means Simics simulation goes twice as fast as the simulated virtual system, while a figure of 2.0 means the simulation goes in half the speed compared to the simulated system.
Don't take this figure for any kind of exactness to the real system, sim.time.virtual depends on:
- The frequency of the simulated processors.
- A simplistic timing models where typically one instruction always takes one cycle to execute.
- Device models are also typically simplistic on how fast transfers and interrupts occur.
- Idle processor can typically consume cycles almost infinitely fast, which does not happen in real hardware.
How fast Simics executes also depends on how many processors it schedules, the processor models, the type of workload, threading settings, internal caches and much more. Thus, one needs to take the slowdown figure with a grain of salt, before comparing with other sessions, or even with just with different workloads.
Esteps - sim.esteps
Steps in Simics are roughly equivalent to instructions. Some processor models however can consume steps while they are stalled. Therefore, the esteps (executed steps) metric is better suited for counting instructions, where the halt-steps have been subtracted from the steps counter. The simulator global sim.esteps is the total of all cpu.esteps in the system.
IPS - sim.mips
(M)IPS is an acronym of (Million) Instructions Per Second. This is a common measurement for looking at the how fast the simulation is. It is calculated as sim.esteps / sim.time.wallclock. Note that, depending on the simulation speed, other prefixes then M is used each sample, for example G for GIPS, or k for kIPS.
Esteps - cpu.esteps
The processor specific number of executed steps.
Sched (s) - cpu.time.schedule
The amount of time Simics scheduler has scheduled this processor.
Sched% - cpu.schedule_percent
This probe gives a percent value of how much of the time this processor has been scheduled cpu.time.schedule / sim.time.schedule This can be used to detect bottle-necks in the simulation, if one processor stands for the majority of the time.
IPS - cpu.mips
This probe might look like sim.mips but is actually a bit different. The processor specific MIPS number is calculated as: cpu.esteps / cpu.time.schedule. This means that the MIPS numbers here cannot be added together. Instead this gives an indication of how fast this particular processor executes, given the amount of time it got. This can be used to detect sub-optimal processor simulation even if the processor runs at a much lower frequency, and consequently is not scheduled as much.

When the simulation is finished, the probe-monitor shows a table of all the probes that are monitored. The table provides the current value for all probes, see figure 26 for the example output.

The Session Formatted Value represents the probe value formatted accordingly to the probe properties. If probes have been added with the current mode then an additional Current Formatted Value column is also displayed.

It is also possible to get the same results printed out with the pm0.summary command, where it is possible to request more floating point decimals to be presented with the float-decimals argument.

The pm0.status shows the settings of the probe-monitor, including which probes that are currently monitored. Probes can be removed with the pm0.remove-probe command and it is possible to change the sampling settings of the probe-monitor itself with the pm0.sampling-settings command.

While the probe-monitor prints out the table data while the simulation is running, all data is also saved internally. To show the entire history of all samples, the pm0.print-table can be used and the data can also be saved into a .csv with the pm0.export-table-csv command. Any data in the csv file will be unformatted, allowing easily import in a spreadsheet for further analyses.

2.7.2.2 Probe Streamer

The probe-streamer tool offers a way to read out probe values and dump the samples in a .csv, while the simulation is running.

To create a probe-streamer use the new-probe-streamer command. This will create a new object such as ps0 which has some additional commands. Most of these commands are common to the probe-monitor. See figure 27 for an example.

2.7.2.3 Probe Based system-perfmeter

This tool replaces the old system-perfmeter tool, which had similar capabilities but more monolithic, with all measurement, calculations and output formatting part of the tool itself. Another difference is that the old system-perfmeter was a singleton object, you could not have multiple tools measuring different things.

The new-system-perfmeter extends the new-probe-monitor, with some added features making it easier to use when measuring Simics performance:

In the first run, only the default probes are used. The -summary switch automatically prints the value of the collected probes when Simics stops. In the second run, the -cpu-exec-modes is added, causing 8 additional probes to be monitored.