2.6 Virtual disks 2.8 Hardware inspection

2.7 Getting data into the simulated system

There are several situations when it is necessary and/or useful to let the simulated target machine interact with the host system, or even other systems accessible to the host via networking. Probably the most common situation is the need to transfer files between the target and the host.

This section will describe a few ways of transferring data between the simulation and the real world:

Data transfer using the preferred method, the Simics Agent.
Data transfer by attaching simulated disks, using disk images.
Data transfer using the simulated network and the service node.
Data transfer when connecting the simulated network to a real network.

2.7.1 Transferring data using the Simics Agent

File transfer can be done in several ways. The preferred method is to use the Simics Agent, which is piece of software that must be present on the target machine. The Clear Linux distribution for the QSP-x86 machine has the agent pre-installed. If the agent is not present, it must be transferred to the target machine using one of the methods described later. The Simics Agent is preferred since it is the fastest and least intrusive method.

The Simics Agent manager is a single global manager in Simics, used to manage all connections to all agents on all targets. When a connection is set up to a specific agent on a target, a Simics Agent handle is created. The handle is used to do actual operations. There can be multiple handles in use at once, connected to different agents.

Here is an example of using the Simics Agent for transferring files, when running the

%simics%/targets/qsp-x86/firststeps.simics

target.

Start Simics with this target.

$ bin/simics targets/qsp-x86/firststeps.simics

Start the Simics Agent manager. Here we also give it an explicit name.

simics> $m = (start-agent-manager name = mgr0)

Boot the target machine, log in as simics, and start the simics-agent from the target console prompt (the ClearLinux target has in fact done so already as part of its boot, but we start another one for this example).

simics@cl-qsp ~ $ simics-agent qsp &
qsp connected (1b90f02e10d5a84c)

Now we can connect the agent manager to the simics-agent on the target. A connection between the agent manager and a specific agent is represented by an agent handle. All commands are performed by the user using the agent handle.

simics> $m.connect-to-agent name = agent0 identifier = qsp
agent0 connected to qsp0 (0x1b90f02e10d5a84c)

The agent0 object is the new agent handle.

Using the handle, we can transfer a file from the host to the target.

simics> agent0.upload from = "%simics%/targets/qsp-x86/firststeps.simics" to = /tmp
agent0:job 1 (upload firststeps.simics)
simics> agent0.run-until-job 1
agent0:job 1 (upload firststeps.simics) finished

Similarly, we can use the agent handle to transfer a file from the target to the host.

simics> agent0.download -overwrite from = "/tmp/firststeps.simics" to = /tmp
agent0:job 2 (download firststeps.simics)
simics> agent0.run-until-job 2
agent0:job 2 (download firststeps.simics) finished

Note that the agent operations are necessarily asynchronous, which is why we must wait for them to finish. However, there is no need to wait after each command; several commands can be queued up before a wait is issued.

The Simics Agent can also perform other operations, such as

List files on the target, see agent0.ls
Print target file contents, see agent0.print-file
Run arbitrary target (shell) commands, see agent0.run
Retrieve output of the commands, see agent0.get-captured-output

2.7.2 Transferring files using disk images

There are other methods for transferring data in and out of the simulation. If the Simics Agent is not already present on the target machine, one of them must be used first to copy the agent software.

USB - from a USB disk image, a virtual USB disk can be created and connected to the target machine.
SATA - from a disk image, typically in CRAFF format, a virtual hard disk can be created and connected to the target machine.

Here is an example of transferring the Simics Agent to the target using USB, when running the

%simics%/targets/qsp-x86/firststeps.simics

target machine.

Start Simics with this target.

$ bin/simics targets/qsp-x86/firststeps.simics

Boot the target machine, log in as simics.
Create and connect a virtual USB disk using the Simics Agent disk images provided with Simics.

simics> $usb_img = "%simics%/targets/common/images/simics_agent_binaries.img"

simics> $usb_disk = (new-usb-disk-comp file = $usb_img name = usb_disk)

simics> $usb_disk.connect-to $system
Connecting usb_disk.usb_host to board.mb.sb.usb_port[10]

The new USB disk has been created and detected by the target machine operating system. Now we can mount it, copy and run the agent.

simics@cl-qsp ~ $ sudo mount /dev/sdc1 /media
simics@cl-qsp ~ $ find /media -name simics_agent_x86_linux64
/media/simics_agent_x86_linux64
simics@cl-qsp ~ $ cp /media/simics_agent_x86_linux64 .
simics@cl-qsp ~ $ ./simics_agent_x86_linux64 qsp &

With the agent now running on the target machine, we can now proceed as in the earlier section if we want to transfer other files.

Here is another example of transferring the Simics Agent to the target using a second hard disk, when running the

%simics%/targets/qsp-x86/firststeps.simics

target machine. This machine is already set up to allow easy addition of an extra disk, using the disk1_image parameter.

Start Simics with this target.

$ bin/simics targets/qsp-x86/firststeps.simics disk1_image=targets/common/images/simics_agent_binaries.img

Boot the target machine, log in as simics.

Mount the second hard disk.

simics@cl-qsp ~ $ sudo mount /dev/sdb1 /media
simics@cl-qsp ~ $ find /media -name simics_agent_x86_linux64
/media/simics_agent_x86_linux64

Copy and run the Simics Agent.

simics@cl-qsp ~ $ cp /media/simics_agent_x86_linux64 .
simics@cl-qsp ~ $ ./simics_agent_x86_linux64 qsp &

With the agent now running on the target machine, we can now proceed as in the earlier section if we want to transfer other files.

2.7.3 Network simulation and the service node

Simics is a full system simulator, which can also simulate a network of machines. It includes a feature called the service node which acts like a separate machine on the simulated network, with its own IP address. The service node can expose common useful network services, such as DHCP, DNS, NTP as well as an FTP server that facilitates connecting to the host system and transfer files.

The Clear Linux distribution for the QSP-x86 machine uses the service node DHCP and DNS servers during boot, to set up its network services.

Here is an example of transferring the Simics Agent to the target machine over FTP, when running the

%simics%/targets/qsp-x86/firststeps.simics

target machine.

Start Simics with this target.

$ bin/simics targets/qsp-x86/firststeps.simics

Boot the target machine, log in as simics.
Notice that the service node and the target machine is set up in the service node DNS. The IP and MAC addresses are internal to the simulated network.

simics> $service_node.sn.list-host-info
┌───────────┬───────────────────┬─────────────────┐
│    IP     │    name.domain    │       MAC       │
├───────────┼───────────────────┼─────────────────┤
│10.10.0.1  │simics0.network.sim│20:20:20:20:20:00│
│10.10.0.100│board.network.sim  │00:17:a0:00:00:00│
└───────────┴───────────────────┴─────────────────┘

From the target console, we can also verify that the DNS is working by querying the service node IP.

simics@cl-qsp ~ $ host simics0
simics0.network.sim has address 10.10.0.1

Now we can set up the service node FTP server by specifying its root directory on the host.

simics> $img_dir = "%simics%/targets/common/images"
simics> $service_node.set-ftp-directory dir = $img_dir

Now we can run an FTP client on the target to copy the agent, and then run it.

simics@cl-qsp ~ $ lftpget ftp://simics0/simics_agent_binaries.zip
simics@cl-qsp ~ $ unzip -q simics_agent_binaries.zip
simics@cl-qsp ~ $ ./simics_agent_x86_linux64 qsp &

With the agent now running on the target machine, we can now proceed as in the earlier section if we want to transfer other files.

2.7.4 Connecting to real networks

Simics can let the target machine connect to any network that the host system has access to. This can be achieved in several ways, where NAPT is the simplest option. Simics then acts similarly like a regular home router, performing network address translation between the simulated network and the network that the host is connected to. The service node DNS server can also act as a proxy for real DNS.

The Clear Linux distribution for the QSP-x86 machine is set up to allow such usage.

Here is an example of transferring data from the host to the target using regular networking, when running the

%simics%/targets/qsp-x86/firststeps.simics

target machine.

Start Simics with this target.

$ bin/simics targets/qsp-x86/firststeps.simics

Boot the target machine, log in as simics.
Enable real time mode, to avoid network timeouts. This makes sure that Simics will not run the simulation faster than real time.

simics> enable-real-time-mode

Real-time mode enabled.

Set up a simple HTTP server on the host. This command can naturally also be executed in any host shell, but here we use the shell command from within Simics.

simics> $python = (lookup-file "%simics%/bin/mini-python")
simics> $imgs = (lookup-file "%simics%/targets/common/images")
simics> shell -bg ($python + " -m http.server --directory " + $imgs)

For easier access of the host system from the target, we add the host IP address to the service node DNS.

simics> $service_node.sn.add-host name = "host0" ip = (sim->host_ipv4)

Now we can fetch data from the host using curl in the target console.

simics@cl-qsp ~ $ curl -s http://host0:8000/simics_agent_binaries.zip

On the target machine, we can verify that the file has been transferred.

simics@cl-qsp ~ $ ls simics_agent_binaries.zip
simics_agent_binaries.zip

In this example we only connected to the host itself, but in fact the same technique can be used to connect to any other system that the host can access. Also note that the HTTP server we open on the host will be accessible by any web browser.

Using port forwarding, the target can also be accessed from the host, and potentially from any system that can access the host, so this should be used with care. Here is an example of transferring a file to the target from the host over ssh, when running the

%simics%/targets/qsp-x86/firststeps.simics

target machine. This assumes that the host has ssh installed.

Install ssh on your host, if not already present.
Start Simics with the mentioned target.

$ bin/simics targets/qsp-x86/firststeps.simics

Inspect the service node DNS to find the IP address of the target machine.

simics> $service_node.sn.list-host-info

┌───────────┬───────────────────┬─────────────────┐
│    IP     │    name.domain    │       MAC       │
├───────────┼───────────────────┼─────────────────┤
│10.10.0.1  │simics0.network.sim│20:20:20:20:20:00│
│10.10.0.100│board.network.sim  │00:17:a0:00:00:00│
└───────────┴───────────────────┴─────────────────┘

Set up port forwarding to the target. By default, this opens a few ports on the host corresponding to FTP, SSH, Telnet and HTTP, and connects them to the target. Other ports can also be opened by specifying parameters to the command.

simics> connect-real-network target-ip = 10.10.0.100
NAPT already enabled.
Host TCP port 4021 -> 10.10.0.100:21
Host TCP port 4022 -> 10.10.0.100:22
Host TCP port 4023 -> 10.10.0.100:23
Host TCP port 4080 -> 10.10.0.100:80
Warning: This can expose the target system on the host local network.

Boot the target machine, log in as simics.
Enable real time mode, to avoid network timeouts.

simics> enable-real-time-mode
Real-time mode enabled.

Now we can transfer the Simics Agent to the target machine over ssh. Note that we connect to the host itself and make use of the port forwarding.

simics> $fn = (lookup-file "%simics%/targets/common/images/simics_agent_binaries.zip")
simics> shell -bg "scp -P 4022 " + $fn + " simics@localhost:agent.zip"

On the target machine, we can verify that the file has been transferred.

simics@cl-qsp ~ $ ls agent.zip
agent.zip

In this example we only copied a file, but using port forwarding, we can connect to other network service available on the target, such as web servers.

2.6 Virtual disks 2.8 Hardware inspection