For Simics compatibility, a disk image needs to be either in the
Simics-specific craff file format, or be a raw (linear)
image.
The raw format is what you get when dumping a disk with a utility such
as dd. Keeping the image in raw format is useful if you
want to be able to loopback mount the image, or if you want to update
the image in-place using the read/write option in Simics. For most
other use-cases, compressing the image with the craff
utility will be beneficial as the host disk requirements will often be
dramatically smaller than for raw images. Image file access is usually not
frequent enough during simulation to impact
performance. The craff format is recommended as the best
all-round image file format for Simics. See section 7.2.8 for more information about the craff
utility.
If you modify or create new files on a storage device within Simics, you should remember that by default images are read-only. This means that the alterations made when running Simics are not written to the image, and will last only for the current Simics session. As described in the 6.2.3 section, this behavior has many advantages. You may however want to save your changes to the image, in order to re-use them in future simulations.
The first thing you should do is to make sure that all the changes are actually written to the media you are using. In many cases, the simulated operating system caches operations to disks or floppies. A safe way to ensure that all changes are written back is to shutdown the simulated machine.
When all changes have been written to the media in the simulation, you can save the new contents of the image in different ways:
-save-craff flag
to the command one can create an image file in the compressed Simics-specific
craff format.
Once you have saved the images, you can do the following:
For example, let us suppose that you saved some new files on the disk of the
enterprise machine (started with the
enterprise-common.simics script). You saved the persistent state
of the machine after stopping it to the persistent state file
new-files-added. You can easily create a small script to start
enterprise with the new files:
# enterprise-new-files.simics run-command-file enterprise-common.simics load-persistent-state new-files-added
If you save several persistent states or image diff files that are dependent on each other, it may become cumbersome to take care of all these dependencies and to remember which files are important. You can merge the states or image diff files to create a new independent state:
checkpoint-merge utility to create a persistent state that is
independent of all previous files, including the original images provided with
Simics. This is the recommended way of creating a new independent
image. You can load it with the load-persistent-state
command.craff utility described below to merge the diff files
yourself.
Although images are divided into pages that are only loaded on request, Simics can run out of host memory if very big images are used, or if the footprint of the software running on the simulated system is bigger than the host memory. To prevent these kind of problems, Simics implements a global image memory limitation controlled by the set-image-memory-limit command.
When Simics is started a default memory-limit is automatically set based on the amount of physical memory available on the host. The default memory-limit does not consider if other applications and users are running on the same host, nor what kind of target system that is simulated in Simics. (For example each target processor will allocate additional non-image memory, so for systems with many processors the default limit could be too high).
When the memory limit is reached, Simics will start swapping out pages to disk very much like an operating system would do. The set-image-memory-limit command lets you specify the maximum amount of memory that can be used, and where swapping should occur.
As mentioned in section 6.2.3, images can also work as read-write media, although this is not recommended. It can be useful sometimes when planning to make large changes to an image (like installing an operating system on a disk).
To make an image read-write in your own configurations, simply set the second
parameter (the "read-only" flag) of the files attribute in the
image object to "rw".
In a ready-to-run example like enterprise, you can change this attribute after the configuration is completed:
# read the 'files' attribute simics> @files = conf.disk0_image.files simics> @files [['enterprise3-rh73.craff', 'ro', 0, 20496236544L, 0]] # provide the complete path to the file simics> @files[-1][0] = "[project]/targets/enterprise/images/enterprise3-rh73.craff" # change the second element to make the file read-write simics> @files[-1][1] = "rw" # check the result simics> @conf.disk0_image.files [['[project]/targets/enterprise/images/enterprise3-rh73.craff',
Note that by indexing files with the index -1, the last
element of the array is accessed, which is always the one that should be set
read-write, in case files is a list of several files.
Simics does not look for files in the Simics search path when the files are used in read-write mode. If you do not provide a complete path to a read-write file, a new file (in the uncompressed craff format) will be created in the current directory.
If the read/write file already exists, it must be a raw image or an uncompressed craff file. Simics does not support using compressed craff files in read/write mode.
Use this feature with caution. Make sure to take a copy of the original image before running Simics with the image in read-write mode. Remember to synchronize the storage device within the target OS before exiting Simics, for example by shutting down the simulated machine.
This is a Linux specific chapter.
If you have an image that contains a FAT file system, you can use Mtools
(http://mtools.linux.lu) to get read-write access to the image. You
must have a raw binary dump of the image for Mtools to work. This can be
obtained using the craff utility (see section 7.2.8).
A few wrapper scripts around Mtools are included in the Simics distribution in the scripts directory.
If your image is partitioned (a complete disk for example), you may need to give Mtools special parameters like an offset or a partition. Please see the Mtools documentation for more information.
This is a Linux specific chapter. If the host OS supports loopback devices, like, e.g., Linux, you can mount an image on your host machine and get direct read/write access to the files within the image. If you have root permissions this allows you to easily and quickly copy files.
craff format but you can use the
craff utility to unpack the disk image to a raw image. The
resulting images have the same size as the simulated disk, so you need to have
sufficient free space on your host disk to contain the entire simulated disk
image.
mount <disk_dump> mnt_pnt -o loop=/dev/loopn,offset=m
Example:
# mount /disk1/rh6.2-kde-ws /mnt/loop -o loop=/dev/loop0,offset=17063424 # cd /mnt/loop # ls bin dev home lost+found opt root tmp var boot etc lib mnt proc sbin usr #
As shown in the example, the disk dump containing a Red Hat 6.2 KDE WS is
mounted on the /mnt/loop directory. The file system mounted on
/ starts on the offset 17063424 on the disk. Linux autodetects the
file system type when mounting (ext2 in this example). If you want to access
another kind of file system, use the -t fs option to the
mount command. Once the file system is mounted, you can copy files in and out
of the disk image.
The offset can be calculated by examining the partition table with
fdisk (from within Simics). Use mount to find the
partition you want to edit or examine (e.g., /dev/hda2 is mounted
on /usr which you want to modify). Next, run fdisk on
the device handling this partition (such as fdisk /dev/hda). From
within fdisk, change the display unit to sectors instead of
cylinders with the u command and print the partition table with
p. You will now see the start and end sectors of the partitions; you
can get the offset by taking the start sector multiplied with the sector size
(512).
When you have finished examining or modifying the disk, unmount it and touch the disk image. For example:
cd umount /mnt/loop touch /disk1/rh6.2-kde-ws
The modification date of the disk image does not change when you modify the disk via the loopback device. Thus, if you have run Simics on the disk image earlier, the OS might have cached disk pages from the now modified disk image in RAM. This would cause a new Simics session to still use the old disk pages instead of the newly modified pages. Touching the image file should ensure that the OS rereads each page.
In some cases, you may want to populate a simulated disk from multiple files covering different parts of the disk. For example, the partition table and boot sectors could be stored in a different disk image file than the main contents of the disk. If that is the case, you cannot use the <image>.add-diff-file command: you must set manually the disk image files attribute to put each image file at its appropriate location.
Assume you are simulating a PC and want to build a disk from a main file called
hda1_partition.img and a master boot record image file called
MBR.img. The main partition will start at offset 32256 of the
disk, and the MBR (Master Boot Record) covers the first 512 bytes of the disk
(typically, you would get the contents of these image files from the real disk
as detailed in section 7.8). The following
command in Simics's start-up script will build the disk from these two files.
load-module std-comp
create-ide-disk-comp disk2 size = 2559836160
@image = get_component_object(conf.disk2, 'hd_image')
@image.files = [["hda1_partition.img", "ro", 32256, 1032151040, 0],
["MBR.img", "ro", 0, 512, 0]]
Note that the two image files cover non-overlapping and non-contiguous sections of the disk.
The images distributed with Simics, and in general most of the images created
by Simics, are in the craff file format. The craff
utility can convert files to and from the craff format, and also
merge several craff files into a single file.
In your Simics distribution you will find craff in
Windows:[simics]\bin, Linux:[simics]/bin.
The examples below assume that craff is present in your shell path.
craff format> craff -o mydisk.craff mydisk.img
craff file to a raw file> craff --decompress -o mydisk.img mydisk.craff
craff file
> craff -o merged.craff chkpt1.craff chkpt2.craff chkpt3.craff
craff file to a raw dump, producing a new dump> craff --decompress -o new.img mydisk.img diff.craff
The input files can be any combination of raw and craff files.
> craff --diff -o diff.craff dump1.img dump2.img
The resulting file, diff.craff, will contain only what is needed
to add to dump1.img in order to get dump2.img.
This is useful to save space if little has been changed.
See also the Simics Reference Manual for a full description of the
craff utility and its parameters.
Most large data files in Simics, such as disk images, are stored in
the compressed random access file format (CRAFF). In order to
manipulate the raw data in these files it is necessary to decompress
the file using the craff tool. For disk images these files can be
very large, so decompressing them is not always feasible. Therefore,
we provide an alternative in the experimental craff-fs tool. With
craff-fs you can "mount" the craff file and access it through the
file system as if it is a normal raw file. The craff-fs tool is only
available on Linux and is provided AS IS.
craff-fs requires libfuse version 2.x.x (tested with
2.9.9) and libvtutils.
The synopsis of craff-fs is as follows:
# ./bin/craff-fs craff-fs file mountpt [fuse-opts]
This examples shows how to "mount" a craff image of a disk image in order to expose it as raw data without decompressing it and then mount the resulting file via loop back in order to access the disk's file system. Mounting craff files as raw files does not require root privileges, but mounting loop back mounting the result usually does. In this example, we use the UEFI agent craff image from Simics package 2096.
The first step is to mount the craff file with craff-fs to be able to access the raw data. Note that the mount point directory must always be created first. As explained further below, there are permission issues that one has to get right, and one way is to use administrator privileges everywhere, so in this example we will do that.
$ mkdir -p craff-mnt $ sudo ./bin/craff-fs targets/qsp-x86/images/efi_agent_and_grub_only.craff craff-mnt
The mount point will now be populated with two
files: data and info.
Other users (including root or users with administrator privileges) cannot access the information mounted under craff-mnt mount point:
$ ls craff-mnt ls: cannot access 'craff-mnt': Permission denied
Details about the user, group and file permission cannot be accessed by other users (including root).
$ ls -l d????????? ? ? ? ? ? craff-mnt/
If other users should access the mount point, "allow_others" should be
uncommented from /etc/fuse.conf file. This will enable all users
(including root) to list/view the contents.
The data
file contains the raw data that makes up the craff file,
the info file contains additional information about the
craff file in text format:
$ sudo cat craff-mnt/info Craff mount: 0.1.0 Craff version: 1 Compression: gzip Size: 107374182400 Block bits: 13 Sub bits: 4 Directory bits: 9
Since a disk image will have different partitions at different offsets
it is not possible to mount the "disk image". It is necessary to mount
the individual partitions. There is a tool called parted
that can display this information.
$ sudo parted craff-mnt/data GNU Parted 3.5 Using /disk1/simics-6/craff-mnt/data Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) unit Unit? [compact]? b (parted) print Model: (file) Disk /disk1/jhbaarnh/simics/merge-6/simics-base/craff-mnt/data: 107374182400B Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End Size File system Flags 1 0B 107374182399B 107374182400B fat32 (parted) q
CraffFS allows mounting as read-write and read-only. File systems successfully tested: ext2, ext3, ext4, NTFS, FAT32.
It can be seen from the example above that the first partition has a fat32 file system and it starts at offset 0. This information will be required to mount that partition via the loop back device.
Mount as read-only:
$ mkdir -p mnt-loop $ sudo mount -o ro,loop,offset=0 craff-mnt/data mnt-loop $ ls mnt-loop EFI SimicsAgent.efi $
Mount as read-write:
$ mkdir -p mnt-loop $ sudo mount -o rw,loop,offset=1048576 craff-mnt/data mnt-loop $ ls mnt-loop EFI SimicsAgent.efi $
Mount command used with options (-o) requires administrator privileges. If the craff-fs command is run without administrator privileges, even if mount command is run as root, mounting will fail. There are two options:
/etc/fuse.conf
and uncomment "allow_others". Root (and other users) can mount and
access mounted files.
$ sudo fusermount -u mnt-loop $ sudo fusermount -u craff-mnt
Craff files will increase in size as files and information is written,
but the file will not automatically decrease if same files or others
are removed/deleted. To shrink the craff file size after delete,
create an zero file (full of zeros) until the end of the partition and
use the craff tool. This tool will ignore all zero blocks
and will shrink the craff file.