This chapter describes how to use gdb-remote, a Simics module that lets you
connect a GDB session running on your host machine to the simulated machine
using GDB’s remote debugging protocol, and use GDB to debug software running on
the target machine.
With the gdb-remote module in Simics, you can use the remote debugging feature
of GDB, the GNU debugger, to connect one or more GDB processes to Simics over
TCP/IP. In order to do this, you need a GDB compiled to support the simulation’s
target architecture on the host you are running. The gdb-remote module has
been tested with several different versions up to at least version 14.
A prebuilt GDB is distributed in the Simics GDB (#1031) package. In a Simics
project, the bin/gdb command (bin\gdb.bat on Windows) can be used to launch
GDB.
The
bin/gdborbin\gdb.batcommands will only work if Simics GDB (#1031) is installed and available as addon package in the Simics project.
To connect a GDB session to Simics, start your Simics session and run the
new-gdb-remote command, optionally followed by a TCP/IP port number, which
defaults to 9123 otherwise. You can specify which processor(s) the remote
debugger should be attached to with the cpu or cpus arguments. If no cpu is
provided the current frontend processor will be used.
When a configuration is loaded, Simics will start listening to incoming TCP/IP connections on the specified port. Run the simulated machine up to the point where you want to connect GDB. To inspect a user process or dynamically loaded parts of the kernel, the easiest solution might be to insert magic instructions at carefully chosen points. For static kernel debugging, a simple breakpoint on a suitable address will solve the problem.
Once Simics is in the desired state, start your GDB session, load any debugging
information into it, and then connect it to Simics using one of the target remote host:port or target extended-remote host:port commands. Here host is
the host Simics is running on, and port is the TCP/IP port number as described
above. Here is a short sample example of debugging a Linux kernel:
(gdb) symbol-file path/to/vmlinux
Reading symbols from path/to/vmlinux...done.
(gdb) set endian little
The target is assumed to be little endian
(gdb) target remote localhost:9123
Remote debugging using localhost:9123
0x000000000000fff0 in unicast_sock ()
(gdb)
The GDB supplied with Simics has builtin support for several target
architectures. The architecture will be set automatically when GDB connects, but
for big-endian architectures (such as PPC), the following GDB command has to be
issued: set endian big.
From this point, you can use GDB to control the target machine by entering
normal GDB commands such as continue, step, stepi, info regs,
breakpoint, etc.
GDB does not expect the target state to change while it is in control and has
asked for the simulation to be stopped. If the simulation is started from
another frontend (e.g., by the run command from the Simics command line) then
GDB will not be notified and will still use cached information about the program
location and register values. To recover from such a situation, issue a GDB
command that makes the simulation run, for example stepi. GDB breakpoints will
not be planted when simulation is started outside of GDB as GDB plants the
breakpoints as part of continuing the target.
To force Simics to close a connection, use the gdb0.disconnect command.
The gdb-remote module does not have any high-level information about the OS
being run inside Simics. This means that in order to examine memory or
disassemble code, the data or code you want to look at has to be in the active
TLB.
When using
gdb-remotewith targets supporting multiple address sizes (such as x86-64), you must have a GDB compiled for the larger address size.
The GDB client can be attached to multiple simulated cpus of the same
architecture. This is done by providing the processors to debug using the cpus
argument to new-gdb-remote and then connecting from the GDB client using
target extended-remote host:port.
Once the gdb-remote object is created each simulated processor will get
assigned a process ID (PID). That PID can be used to attached the cpu from
within the GDB client. The PID for each processor can be obtained using the
info command of the gdb-remote object:
$gdb = (new-gdb-remote cpus = "board.mb.cpu0.core[0][0]" "board.mb.cpu0.core[0][1]")
[gdb0 info] Awaiting GDB connections on port 9123.
[gdb0 info] Connect from GDB using: "target extended-remote localhost:9123"
$gdb.info
...
Process IDs for cpus (for use with attach):
board.mb.cpu0.core[0][0] : 1
board.mb.cpu0.core[0][1] : 2
Or from within the GDB client:
(gdb) target extended-remote localhost:9123
Remote debugging using localhost:9123
(gdb) info os processes
pid cpu
1 board.mb.cpu0.core[0][0]
2 board.mb.cpu0.core[0][1]
When using multicore one should issue the following command in the GDB client:
(gdb) set schedule-multiple on
This will set so that the client continues all attached processes (which matches cpus in the simulation). Simulation normally expects all cpus to advance at the same time, if only one cpu is run the target might not work as expected.
Once connected each cpu can then be attached to an inferior in GDB by issuing
the attach command with the given PID:
(gdb) info inferiors
Num Description Connection Executable
* 1 <null> 1 (extended-remote localhost:9123)
(gdb) attach 1
Attaching to process 1
0x0000000000040000 in ?? ()
(gdb) add-inferior
[New inferior 2]
Added inferior 2 on connection 1 (extended-remote localhost:9123)
(gdb) inferior 2
[Switching to inferior 2 [<null>] (<noexec>)]
(gdb) attach 2
Attaching to process 2
0x000000000000fff0 in ?? ()
(gdb) info inferiors
Num Description Connection Executable
1 process 1 1 (extended-remote localhost:9123)
* 2 process 2 1 (extended-remote localhost:9123)
In the above example, two CPUs have been attached as two processes in different inferiors. Once this is done the cpus can be handled as threads in the GDB client:
(gdb) info threads
Id Target Id Frame
1.1 Thread 1.1 (board.mb.cpu0.core[0][0]) 0x0000000000040000 in ?? ()
* 2.1 Thread 2.1 (board.mb.cpu0.core[0][1]) 0x000000000000fff0 in ?? ()
gdb) thread 1.1
[Switching to thread 1.1 (Thread 1.1)]
#0 0x0000000000040000 in ?? ()
(gdb) info threads
Id Target Id Frame
* 1.1 Thread 1.1 (board.mb.cpu0.core[0][0]) 0x0000000000040000 in ?? ()
2.1 Thread 2.1 (board.mb.cpu0.core[0][1]) 0x000000000000fff0 in ?? ()
Some actions, such as adding symbol files and planting address breakpoints, will only be applied to the currently selected inferior. To add the same symbol file for multiple inferiors the file has to be added for each inferior:
(gdb) inferior 1
[Switching to inferior 1 [process 1] (<noexec>)]
[Switching to thread 1.1 (Thread 1.1)]
#0 0x0000000000040000 in ?? ()
(gdb) add-symbol-file vmlinux
Reading symbols from vmlinux...
(gdb) inferior 2
[Switching to inferior 2 [process 2] (<noexec>)]
[Switching to thread 2.1 (Thread 2.1)]
#0 0x000000000000fff0 in ?? ()
(gdb) add-symbol-file vmlinux
Reading symbols from vmlinux...
When planting a symbolic breakpoint it will be planted on all inferiors that have such a symbol. Planting an address breakpoint will on the other side only apply to the selected inferior. To plant on the same address for multiple cpus one has to plant the breakpoint once for each inferior:
(gdb) inferior 1
[Switching to inferior 1 [process 1] (<noexec>)]
[Switching to thread 1.1 (Thread 1.1)]
#0 0x0000000000040000 in ?? ()
(gdb) b *0x40008
Breakpoint 1 at 0x40008
(gdb) inferior 2
[Switching to inferior 2 [process 2] (<noexec>)]
[Switching to thread 2.1 (Thread 2.1)]
#0 0x000000000000fff0 in ?? ()
(gdb) b *0x40008
Breakpoint 2 at 0x40008
(gdb) info breakpoints
Num Type Disp Enb Address What
1 breakpoint keep y 0x0000000000040008 inf 1
2 breakpoint keep y 0x0000000000040008 inf 2
It takes some work to figure out how to load symbol tables at the correct
offsets for relocatable object modules in GDB. This is done automatically for
normal (non-remote) targets, but for the remote target, you have to do it
yourself. You need to find out the actual address at which the shared module is
mapped in the current context on the simulated machine, and then calculate the
offset to use for GDB’s add-symbol-file command.
Under Linux, the list of memory mappings can be found in the file
/proc/pid/maps (plain text format). The VMA column of the .text line of
the output from objdump -h file contains the start address of the text
segment.
Using these two values, map address and text address, you should use map
address + text address as the offset to add-symbol-file (it has to be done
this way to compensate for how GDB handles symbol loading).
To show you how it works, we will work through a simple example. The example
uses a simple program with a simple shared library. The program can be found in
[qsp-x86]/targets/qsp-x86/images/hello and the shared library is the
libgreeter.so file in the same directory. Here and in the rest of this section
[qsp-x86] refers to the location where the QSP-x86 package is installed.
Start by booting the firststeps machine. Then upload the following files using Simics Agent:
simics> start-agent-manager
simics> agent_manager.connect-to-agent
simics> matic0.upload "%simics%/targets/qsp-x86/images/hello" to=/root
simics> matic0.upload "%simics%/targets/qsp-x86/images/libgreeter.so" to=/root
simics> run
Then run the program in the background. The program will enter the infinite loop in the shared library.
Now we need the map address and the text address of the shared library. To get the map address, look in the process file system to see where it has mapped the shared library:
~ # ./hello &
[1] 13104
~ # cat /proc/13104/maps
00400000-00401000 r-xp 00000000 08:02 9442 /root/hello
00600000-00601000 r--p 00000000 08:02 9442 /root/hello
00601000-00602000 rw-p 00001000 08:02 9442 /root/hello
7f686c5ed000-7f686c786000 r-xp 00000000 08:02 10049 /lib/libc-2.21.so
7f686c786000-7f686c986000 ---p 00199000 08:02 10049 /lib/libc-2.21.so
7f686c986000-7f686c98a000 r--p 00199000 08:02 10049 /lib/libc-2.21.so
7f686c98a000-7f686c98c000 rw-p 0019d000 08:02 10049 /lib/libc-2.21.so
7f686c98c000-7f686c990000 rw-p 00000000 00:00 0
7f686c990000-7f686c991000 r-xp 00000000 08:02 10247 /root/libgreeter.so
7f686c991000-7f686cb90000 ---p 00001000 08:02 10247 /root/libgreeter.so
7f686cb90000-7f686cb91000 r--p 00000000 08:02 10247 /root/libgreeter.so
7f686cb91000-7f686cb92000 rw-p 00001000 08:02 10247 /root/libgreeter.so
7f686cb92000-7f686cbb4000 r-xp 00000000 08:02 9919 /lib/ld-2.21.so
7f686cdad000-7f686cdb0000 rw-p 00000000 00:00 0
7f686cdb2000-7f686cdb3000 rw-p 00000000 00:00 0
7f686cdb3000-7f686cdb4000 r--p 00021000 08:02 9919 /lib/ld-2.21.so
7f686cdb4000-7f686cdb5000 rw-p 00022000 08:02 9919 /lib/ld-2.21.so
7f686cdb5000-7f686cdb6000 rw-p 00000000 00:00 0
7fffde6a7000-7fffde6c8000 rw-p 00000000 00:00 0 [stack]
7fffde7ff000-7fffde800000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
From this output you can see that the program is running with PID 13104 and that the map address is 0x7f686c990000. The exact PID may differ, adapt the commands accordingly.
To get the text address we use objdump. This should be run on a host computer with objdump installed:
> objdump -h [qsp-x86]/targets/qsp-x86/images/libgreeter.so
libgreeter.so: file format elf64-x86-64
Sections:
Idx Name Size VMA LMA File off Algn
:
10 .text 00000134 00000000000005e0 00000000000005e0 000005e0 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
:
The .text symbols starts at address 0x5e0 and this is what we call the text
address, so if we connect GDB to Simics we have to add the symbols with an
offset of 0x7f686c990000 + 0x5e0 = 0x7f686c9905e0.
Now we can set up GDB, connect it to Simics, and debug the program:
(gdb) dir [qsp-x86]/targets/qsp-x86/
Source directories searched: [qsp-x86]/targets/qsp-x86:$cdir:$cwd
(gdb) add-symbol-file [qsp-x86]/targets/qsp-x86/images/libgreeter.so 0x7f686c9905e0
add symbol table from file "[qsp-x86]/targets/qsp-x86/images/libgreeter.so" at
.text_addr = 0x6c9905e0
(y or n) y
Reading symbols from [qsp-x86]/targets/qsp-x86/images/libgreeter.so...done.
(gdb) target remote localhost:9123
Remote debugging using localhost:9123
greet (name=0x4008eb "World") at ../greeter.c:12
14 while (loop); /* Loop until the loop variable is reset by gdb */
(gdb)
This is just a toy program written to make it possible to debug it without any OS awareness. Normally you would use the OS awareness functionality described in Simics Analyzer User’s Guide to debug user space programs.