2.8 Hardware inspection 2.10 Code Coverage of Target Software
Getting Started  /  2 Tutorials  / 

2.9 Debugging Target Software

Simics comes with a built-in command line debugger for debugging the target software. This section describes how to get started with the Simics debugger. We start of with and example of debugging a Linux kernel module, then go through

To follow the examples in this section it is recommended to start Simics with the following script:

%simics%/targets/qsp-x86/qsp-clear-linux-pcie-demo.simics

2.9.1 Example of debugging a kernel module

This example demonstrates how to debug a custom built kernel module. We use the simics-pcie-demo-driver module, which is included in the QSP-x86 package. The example comes with both a kernel module binary and source code for that.

To locate the installation directory with the example files, run the following command in Simics:

simics> lookup-file "%simics%/targets/qsp-x86/target-source/simics-pcie-demo-driver"

In this example we use the Linux tracker. This tool, which is part of the OS Awareness feature, allows keeping track of Linux tasks and processes. The tracker can provide symbol mappings of kernel modules to the debugger, which is the feature we will use in this example.

2.9.1.1 Configuring the Linux tracker

This section describes the steps needed to configure the Linux tracker for kernel module debugging.

Create a target machine and start simulation:

simics> run-command-file targets/qsp-x86/qsp-clear-linux-pcie-demo.simics
simics> run
running>

Let the simulation run until Linux has booted, and the kernel module has been inserted. The simulation is stopped automatically by the script at this point.

Once the system has booted, we can configure the Linux tracker using symbol information for the current kernel. The tracker accepts symbols in two formats.

In this example we will extract kallsyms from the target system and use that as symbols.

simics> matic0.download /proc/kallsyms cl-kallsyms
matic0:job 5 (download kallsyms)
simics> matic0.run-until-job
matic0:job 5 (download kallsyms) finished
simics> machine.software.tracker.detect-parameters symbol-file = cl-kallsyms -load param-file = cl.params kernel-modules = TRUE
[machine.software.tracker.tracker_obj info] Detecting settings using 2 processor(s)
[machine.software.tracker.tracker_obj info] Successfully detected parameters
Saved autodetected parameters to cl.params
simics> machine.software.enable-tracker
OSA control enabled.

The next time we use the Linux tracker with the same kernel we can re-use the same parameters again, by loading the detected parameters using the load-parameters command followed by enable-tracker.

In order for kernel module debugging to work, a path where to find kernel modules on local disk must be set:

simics> machine.software.tracker.set-kernel-modules-path path = "%simics%/targets/qsp-x86/target-source/simics-pcie-demo-driver"

The kernel module binaries on local disk must match ones running in the target kernel.

2.9.1.2 Adding path maps

The kernel module was built under /root/swbuild, but the source files on local host are located under targets/qsp-x86/target-source. To allow the debugger to find the source files, set the following path map:

simics> add-pathmap-entry /root/swbuild/ "%simics%/targets/qsp-x86/target-source/"

2.9.1.3 Debugging

Enable the debugger:

simics> enable-debugger
Debugger enabled.

Set a breakpoint on a function in the module. This will trigger once the drivers file device gets some input.

simics> bp.source_location.break chari_write -once
Breakpoint 6: 0x6 (planted)

Start the simulation:

simics> run

Then output a string to the file device by writing to it from the Linux shell, using the serial console:

simics@cl-qsp ~ $ sudo bash -c 'echo 01 > /dev/simics_pcie_demo_driver'

The simulation should stop with the following output:

[tcf] Breakpoint 6 on execution in context machine.mb.cpu0.core[0][0]

At this point the current stack frame can be checked using the frame command. We can see that the simulation stopped at chari_write and that the buffer contains "01" as written to the device.

simics> frame
#0 0xffffffffc019459c in chari_write(filep=(struct file *) 0xffff8881f01f8300, buffer=(const char *) 0x21664d0 "01\n", len=3, offset=(loff_t *) 0xffffc90001383ef0) at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:365

simics> list
     363  
     364  #ifdef DRIVER_VERBOSE
->   365       pr_info(KBUILD_MODNAME
     366  	        ": chari_write called, with %d characters of input!\n", (int)len);
     367  #endif

Set a breakpoint on line 370 and advance the simulation to that point:

simics> bp.source_line.break filename = "simics-pcie-demo-driver.c" line-number = 370 -once
Breakpoint 7: 0x7 (planted)
simics> run
[tcf] Breakpoint 7 on execution in context machine.mb.cpu0.core[0][0]
chari_write(filep, buffer=(const char *) 0x21664d0 "01\n", len=3, offset) at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:370
370	    while(charsleft >= 2) {

Stepping in the code is done with the step-into, step-over and step-out commands.

simics> step-into
chari_write(filep, buffer=(const char *) 0x21664d0 "01\n", len=3, offset) at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:372
372	        switch(buffer[0]) {

Note that when step is completed the stack frame and current line will be displayed. As the input was "01" the first switch statement should end up at '0' and the second at '1'.

simics> step-into
373                 case '0': led_no=0; break;
simics> step-into
386             switch(buffer[1]) {
simics> step-into
388                 case '1': brightness=1; break;
simics> step-into
396             hw_update_led_state(led_no,brightness);
simics> step-into
hw_update_led_state(led_no=0, brightness=1)
 at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:146
    146             if( (led_no<0) || (led_no>5)) {

The program enters another stack frame. Run stack-trace to see the current stack trace:

simics> stack-trace maxdepth = 2
#0 0xffffffffc01941a0 in hw_update_led_state(led_no=0, brightness=1) at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:146
#1 0xffffffffc019465d in chari_write(filep, buffer=(const char *) 0x21664d0 "01\n", len=3, offset) at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:396

Inspect symbols in the current frame with the sym-value and sym-type commands:

simics> sym-value brightness
1
simics> sym-type brightness
int

Change symbol values using the sym-write command:

simics> sym-write brightness 0
brightness = 0

Step out to return to the outer function, chari_write:

simics> step-out
chari_write(filep, buffer=(const char *) 0xfab4d0 "01\n", len=3, offset) at /root/swbuild/simics-pcie-demo-driver/simics-pcie-demo-driver.c:400
400	        buffer    += 2;

Now step to the end of chari_write. The while loop should complete after the first iteration.

simics> step-into
370	    while(charsleft >= 2) {
simics> step-into
403	    return len;

We check the return value len, before returning, and see that the function claims that 3 bytes have been consumed.

simics> sym-value len
3

The kernel module used in this example is compiled with some optimizations, meaning that some variables are optimized out and some code paths can appear strange when stepping.

2.9.2 Setting up and enabling the debugger

The first step in order to use the debugger is to enable it:

simics> enable-debugger

Then symbol files, with debug information, for the target software need to be added in order for debugging to work.

simics> add-symbol-file <binary to debug>
simics> add-symbol-file "%simics%/targets/qsp-x86/images/debug_example"

OS Awareness trackers can provide symbol files to the debugger instead of having to add them using the command.

In case the source files are not present in the same location as they were compiled, a path map is needed for the debugger to locate the source:

simics> add-pathmap-entry <compilation path> <local disk path>

When the debugger is enabled and the simulations stops the current stack frame will be displayed, under the condition that there is a valid symbol file added for the current debug context and address.

simics> add-pathmap-entry /tmp .

2.9.3 Debug Contexts

A debug context represents something that can be debugged, such as:

The debug-context command can be used to see current debug context or specify a different one.

simics> debug-context
dbg0 (the x86QSP1 machine.mb.cpu0.core[0][0])
simics> debug-context object = "machine.mb.cpu0.core[1][0]"
dbg1 (the x86QSP1 machine.mb.cpu0.core[1][0])

To see available debug context, use the list-debug-contexts command, which optionally can take context-query as argument.

simics> list-debug-contexts
Fully Qualified Name        | Fully Qualified Name
----------------------------+---------------------------
/machine                    | /machine/mb.cpu0.mem[0][0]
/machine/mb.cpu0.core[0][0] | /machine/mb.cpu0.mem[1][0]
/machine/mb.cpu0.core[1][0] | 

Th context-query argument can be used to match a subset of contexts. To list contexts directly under root:

simics> list-debug-contexts context-query="/*"
Fully Qualified Name
--------------------
/machine

To match specific processor:

simics> list-debug-contexts context-query="/machine/'mb.cpu0.core[1][0]'"
Fully Qualified Name
---------------------------
/machine/mb.cpu0.core[1][0]

A context query is a method to specify a subset of debug contexts, by specifying context properties and what values to match. The default property is name.

The forward slashes in the context names are separators. A query consists of a sequence of parts separated by forward slashes. Two wildcards exist, * and **, where one asterisk is used to match any context and two asterisks matches any sequence of contexts.

Many debugger related commands, such as add-symbol-file and add-pathmap-entry, take context-query as an argument to limit what contexts it applies to. Leaving the argument out, means matching all possible debug contexts.

As example, for a symbol file to only apply for a specific processor then add-symbol-file should be called with the context-query argument to match the processor name.

simics> add-symbol-file context-query="/machine/'mb.cpu0.core[1][0]'" <symbol file>

Extra quotation is added around the processor name. This is needed when the context name contains special characters, such as dots and brackets.

2.9.4 Debugger commands

Simics has several debugger commands for investigating the current state, most of them start with the sym- prefix. Here follows some examples of debugger commands:

Simics provides help for all of its commands. It is a good idea to use the help command with the commands to get details and options for the commands.

2.9.5 Breaking on a source line or function

It is possible to stop the simulation when a specific source line or function is hit. To set a breakpoint on a source line use the following command:

simics> bp.source_line.break filename = foo.c line-number = 10

And in a similar way for a function:

simics> bp.source_location.break foo

After setting such breakpoints, the simulation will stop once the source line foo.c:10 or the function foo is hit.

2.9.6 Debugging Summary

This concludes this tutorial, in which we have learnt how to use the Simics command line debugger to

2.8 Hardware inspection 2.10 Code Coverage of Target Software