20 Modeling I2C Devices 22 Representing Network Packets with frags_t
Model Builder User's Guide  /  III Modeling Common Hardware Components  / 

21 Modeling Interconnects

Modeling interconnects is a central part of building a hardware platform in Simics. Sometimes it is as easy as filling in a memory-space map with the devices and the offsets and connecting it to the processor. Other times it is much more complex. Simics provides a rich toolkit to model many variants of interconnects. For modelers it is important to understand the available options and when each of them is suited for the application.

21.1 Interconnects with a Static Memory View

This type of interconnect is best modelled by a memory-space in Simics. The memory-view shall be defined in the Python component code where all devices are created.

phys_mem.map = [
    # Offset    Device                    Fun   Start                 Size
    [0x00001000, boot,                     0,    0,    boot.image.attr.size],
    [0x02000000, clint.bank.regs,          0,    0,                  0xc000],
    [0x10000000, uart.bank.regs,           0,    0,                    0x11],
    [0x10080000, virtio_entropy.bank.mmio, 0,    0,                 64 * KB],
    [0x80000000, ram,                      0,    0,     ram.image.attr.size]
]
# Connect RISC-V HART to interconnect
hart.physical_memory = phys_mem
Figure 13. Memory map taken from the Simics RISC-V Public
simics> memory-map object = board.phys_mem
┌───────────────────────────────────────────────────────┐
│                    board.phys_mem                     │
├──────────┬──────────┬─────────────────────────────────┤
│     Start│       End│Object                           │
├──────────┼──────────┼─────────────────────────────────┤
│0x00001000│0x00040fff│board.boot                       │
│0x02000000│0x0200bfff│board.clint.bank.regs            │
│0x0c000000│0x0fffffff│board.plic.bank.regs             │
│0x10000000│0x10000010│board.uart0.bank.regs            │
│0x10010000│0x1001ffff│board.virtio_net.bank.mmio       │
│0x10020000│0x1002ffff│board.disk0.virtio_disk.bank.mmio│
│0x10030000│0x1003ffff│board.disk1.virtio_disk.bank.mmio│
│0x10040000│0x1004ffff│board.disk2.virtio_disk.bank.mmio│
│0x10080000│0x1008ffff│board.virtio_entropy.bank.mmio   │
│0x80000000│0xffffffff│board.ram                        │
└──────────┴──────────┴─────────────────────────────────┘
Figure 14. Memory map of the HART in Simics RISC-V Public Reference Platform

21.2 Interconnects with a Dynamic Memory View

Memory views that are dynamic must map in devices during runtime. An example would be PCIe where software can:

In all these cases the mapping must now be done at runtime in the device models and not statically in the Python component code. The recommended approach is the usage of the memory-space class and use its map_demap interface for dynamic mapping.


dml 1.4;

device sample_map_demap;
param classname = "sample-map-demap";
param desc = "sample map-demap";

import "utility.dml";
import "simics/devs/map-demap.dml";


connect memory {
    is init_as_subobj;
    interface map_demap;
    param classname = "memory-space";
}

bank regs {
    group BARS[i < 2] {
        param map_obj = i == 0 ? app0.obj : app1.obj;
        register addr size 8 @ i * 0x10 {
            field addr @ [63:12];
        }
        register sz size 4 @ i * 0x10 + 0x8;
        register enable size 1 @ i * 0x10 + 0xC {
            field enable @ [0] is (write) {
                method write(uint64 value) {
                    if (value == this.val)
                        return;

                    /* Enable BAR */
                    if (value == 1) {
                        local map_info_t map_info = { .base = addr.addr.val << 12,
                                                      .length = sz.val,
                                                      ...
                                                    };
                        memory.map_demap.map_simple(map_obj, NULL, map_info);
                    } else {  /* Disable BAR */
                        memory.map_demap.unmap(map_obj, NULL);
                    }
                    this.val = value;
                }
            }
        }
    }
}

/* BAR0 maps internal bank from device model */
bank app0;

/* BAR1 maps an external device */
connect app1 is map_target "application resource" {
    param configuration = "required";
}

Figure 15. Example device mapping objects into an internal memory-space dynamically
simics> @dummy = simics.SIM_create_object("set-memory", "dummy")
simics> @dev = simics.SIM_create_object("sample-map-demap", "dev", app1=dummy)
simics> memory-map dev.memory
┌────────────────┐
│   dev.memory   │
├─────┬───┬──────┤
│Start│End│Object│
├─────┼───┼──────┤
└─────┴───┴──────┘
simics> write-device-reg register = "dev.bank.regs.BARS[0].addr" data = 0x800000000
simics> write-device-reg register = "dev.bank.regs.BARS[0].sz" data = 0x10000
simics> write-device-reg register = "dev.bank.regs.BARS[0].enable" data = 0x1
simics> memory-map dev.memory
┌─────────────────────────────────────┐
│             dev.memory              │
├───────────┬───────────┬─────────────┤
│      Start│        End│Object       │
├───────────┼───────────┼─────────────┤
│0x800000000│0x80000ffff│dev.bank.app0│
└───────────┴───────────┴─────────────┘
simics> write-device-reg register = "dev.bank.regs.BARS[1].addr" data = 0x820000000
simics> write-device-reg register = "dev.bank.regs.BARS[1].sz" data = 0x4000
simics> write-device-reg register = "dev.bank.regs.BARS[1].enable" data = 0x1
simics> memory-map dev.memory
┌─────────────────────────────────────┐
│             dev.memory              │
├───────────┬───────────┬─────────────┤
│      Start│        End│Object       │
├───────────┼───────────┼─────────────┤
│0x800000000│0x80000ffff│dev.bank.app0│
│0x820000000│0x820003fff│dummy        │
└───────────┴───────────┴─────────────┘
Figure 16. Instantiate the sample-map-demap device and program it

Another type of interconnect routes all incoming transactions to either one of two destinations, i.e. a demultiplexer. The routing is decided by the internal state of the interconnect. In Simics this type of interconnect is best modelled by implementing the translator interface.


dml 1.4;

device sample_interconnect_demux;
param classname = "sample-interconnect-demux";
param desc = "sample interconnect";

import "utility.dml";

connect target_dev1 is map_target;
connect target_dev2 is map_target;

port demux_signal is signal_port {
    implement signal {
        method signal_raise() {
            default();
            /* Tell Simics Core that any cached lookup through
             * the demux is no longer valid.
             */
            SIM_translation_changed(dev.obj);
        }
        method signal_lower() {
            default();
            SIM_translation_changed(dev.obj);
        }
    }
}

implement translator {
    method translate(uint64 addr, access_t access,
                     const map_target_t *default_target) -> (translation_t) {
        local translation_t t;

        if (demux_signal.signal.high)
            t.target = target_dev1.map_target;
        else
            t.target = target_dev2.map_target;
        return t;
    }
}

Figure 17. Example Transaction Demultiplexer
simics> @dummy1 = simics.SIM_create_object("set-memory", "dummy1")
simics> @dummy2 = simics.SIM_create_object("set-memory", "dummy2")
simics> @demux = simics.SIM_create_object("sample-interconnect-demux", "demux", target_dev1=dummy1, target_dev2=dummy2)
simics> memory-map demux
┌───────────────────────────────┐
│             demux             │
├─────┬──────────────────┬──────┤
│Start│               End│Object│
├─────┼──────────────────┼──────┤
│  0x0│0xffffffffffffffff│dummy2│
└─────┴──────────────────┴──────┘
simics> @demux.port.demux_signal.iface.signal.signal_raise()
simics> memory-map demux
┌───────────────────────────────┐
│             demux             │
├─────┬──────────────────┬──────┤
│Start│               End│Object│
├─────┼──────────────────┼──────┤
│  0x0│0xffffffffffffffff│dummy1│
└─────┴──────────────────┴──────┘
Figure 18. Interconnect forwarding all transactions to one of two targets

21.3 Initiator Dependent Memory View

Interconnects providing a limited memory view for each initiator are best modelled using atoms which are part of the transaction in Simics. The atoms carry the necessary information about initiator for the interconnect to handle the routing. When the transaction passes through the interconnect it can inspect the atoms and take a routing decision. In Simics an interconnect must implement the transaction_translator interface in order to inspect the atoms of the transaction.

21.3.1 PCIe Memory and Config Routing Example

PCIe transactions are of type Config, Memory, IO or Messages. In the example below we create an interconnect that demuxes incoming transactions depending on the pcie_type


dml 1.4;

device sample_interconnect_pcie_router;
param classname = "sample-interconnect-pcie-router";
param desc = "sample interconnect";

import "utility.dml";
import "simics/devs/pci.dml";

connect downstream_mem is map_target {
    param documentation = "Downstream PCIe memory space";
    param configuration = "required";
}
connect downstream_cfg is map_target {
    param documentation = "Downstream PCIe config space";
    param configuration = "required";
}

implement transaction_translator {
    method translate(uint64 addr,
                     access_t access,
                     transaction_t *t,
                     exception_type_t (*callback)(translation_t txl,
                                                  transaction_t *tx,
                                                  cbdata_call_t cbd),
                     cbdata_register_t cbdata) -> (exception_type_t) {

        local translation_t txl;
        local pcie_type_t type = ATOM_get_transaction_pcie_type(t);
        if (type == PCIE_Type_Not_Set) {
            log info, 1:
                "Downstream PCIe transaction @ 0x%08x is missing type", addr;
            return callback(txl, t, cbdata);
        }

        if (type == PCIE_Type_Cfg) {
            txl.target = downstream_cfg.map_target;
        } else if (type == PCIE_Type_Mem) {
            txl.target = downstream_mem.map_target;
        } else {
            log info, 1: "Unsupported PCIe Type atom %d"
                + " terminating downstream transaction", type;
        }
        return callback(txl, t, cbdata);
    }
}
Figure 19. Example Transaction Translator
simics> @pcie_mem = simics.SIM_create_object("set-memory", "pcie_mem")
simics> @pcie_cfg = simics.SIM_create_object("set-memory", "pcie_cfg")
simics> @dev = simics.SIM_create_object("sample-interconnect-pcie-router", "dev", downstream_mem=pcie_mem, downstream_cfg=pcie_cfg)

simics> probe-address obj = dev -add-atoms ATOM_pcie_type = 1 address = 0x1000
Translating virtual address to physical: 0x1000 -> p:0x1000
┌────────┬──────────┬─────┬───────────────┐
│ Target │  Offset  │Notes│Inspected Atoms│
├────────┼──────────┼─────┼───────────────┤
│dev     │0x00001000│~    │pcie_type      │
│        │          │     │               │
├────────┼──────────┼─────┼───────────────┤
│pcie_mem│0x00001000│     │               │
└────────┴──────────┴─────┴───────────────┘
'~' - Translator implementing 'transaction_translator' interface
Destination: pcie_mem offset 0x1000 - no register information available
simics> probe-address obj = dev -add-atoms ATOM_pcie_type = 3 address = 0x1000
Translating virtual address to physical: 0x1000 -> p:0x1000
┌────────┬──────────┬─────┬───────────────┐
│ Target │  Offset  │Notes│Inspected Atoms│
├────────┼──────────┼─────┼───────────────┤
│dev     │0x00001000│~    │pcie_type      │
│        │          │     │               │
├────────┼──────────┼─────┼───────────────┤
│pcie_cfg│0x00001000│     │               │
└────────┴──────────┴─────┴───────────────┘
'~' - Translator implementing 'transaction_translator' interface
Destination: pcie_cfg offset 0x1000 - no register information available
Figure 20. Interconnect forwarding transactions base on PCIe Type atom

21.3.2 PCIe PASID Routing Example

In PCIe the Process Address Space ID (PASID), is an example of an identifier for the transaction initiator which are used by the Address Translations Services (ATS) in PCIe. In Simics the PASID has the corresponding atom pcie_pasid which are added by PCIe Endpoints when issuing ATS related PCIe Transactions. An IOMMU can then inspect the pcie_pasid atom to decide the routing of the incoming transaction.


dml 1.4;

device sample_interconnect_pcie_pasid;
param classname = "sample-interconnect-pcie-pasid";
param desc = "sample interconnect";

import "utility.dml";
import "simics/devs/pci.dml";

bank regs {
    register allowed_pasid size 4 @ 0x0 {
        field pasid @ [19:0] "Allowed PASID";
    }
}

connect host_memory is map_target {
    param documentation = "Host memory";
    param configuration = "required";
}

implement transaction_translator {
    session bool emitted_warning;
    method translate(uint64 addr,
                     access_t access,
                     transaction_t *t,
                     exception_type_t (*callback)(translation_t txl,
                                                  transaction_t *tx,
                                                  cbdata_call_t cbd),
                     cbdata_register_t cbdata) -> (exception_type_t) {

        local translation_t txl;
        local const uint32 *pasid_val = ATOM_transaction_pcie_pasid(t);
        if (pasid_val == NULL) {
            if (!emitted_warning) {
                log warning:
                    "Denying request: AT translated request @ 0x%08x is missing PASID", addr;
                emitted_warning = true;
            }
            return callback(txl, t, cbdata);
        }

        local pcie_pasid_info_t pasid = {
            .u32 = *pasid_val,
            ...
        };
        if (pasid.field.pasid == regs.allowed_pasid.pasid.val) {
            txl.target = host_memory.map_target;
        } else {
            log info, 1 then 3:
                "Denying request @ 0x%08x for PASID %d", addr, pasid.field.pasid;
        }
        return callback(txl, t, cbdata);
    }
}
Figure 21. Example Transaction Translator
simics> @host_mem = simics.SIM_create_object("set-memory", "host_mem")
simics> @dev = simics.SIM_create_object("sample-interconnect-pcie-pasid", "dev", host_memory=host_mem)
simics> write-device-reg register = "dev.bank.regs.allowed_pasid" data = 0x100
simics> memory-map object = dev
[dev warning] Denying request: AT translated request @ 0x00000000 is missing PASID
┌────────────────┐
│      dev       │
├─────┬───┬──────┤
│Start│End│Object│
├─────┼───┼──────┤
└─────┴───┴──────┘
simics> memory-map object = dev -add-atoms ATOM_pcie_pasid = 1
[dev info] Denying request @ 0x00000000 for PASID 1
┌────────────────┐
│      dev       │
├─────┬───┬──────┤
│Start│End│Object│
├─────┼───┼──────┤
└─────┴───┴──────┘
simics> memory-map object = dev -add-atoms ATOM_pcie_pasid = 0x100
┌─────────────────────────────────┐
│               dev               │
├─────┬──────────────────┬────────┤
│Start│               End│Object  │
├─────┼──────────────────┼────────┤
│  0x0│0xffffffffffffffff│host_mem│
└─────┴──────────────────┴────────┘
Figure 22. Interconnect blocking transactions with wrong PASID

21.4 Interconnects Adding Metadata to the Transaction

21.4.1 Interconnects that are Bridging One Domain with Another

This type of interconnect needs to convert a transaction from the source type to the destination type before passing it through.

One such example is the PCIe Host Bridge connecting the Host interconnect with a PCIe bus.

Downstream memory transactions from a host interconnect routed into the PCIe subsystem must be converted to PCIe packets by the PCIe Host Bridge. Simics does not operate on the packet level, but some metadata are important for functional simulation.

In Simics the PCIe atom pcie_type has to be set for all transactions operating in the PCIe domain, and it can hold the values: PCIE_Type_Mem, PCIE_Type_Cfg, PCIE_Type_IO and PCIE_Type_Msg.

To automatically add the pcie_type atom to all Host transactions one should use the transaction_translator interface and chain the original transaction_t with a new one that appends the relevant atoms. Details about transaction chaining can be found here


dml 1.4;

device sample_interconnect_pcie_bridge;
param classname = "sample-interconnect-pcie-bridge";
param desc = "sample interconnect";

import "utility.dml";
import "simics/devs/pci.dml";

connect host_memory is map_target {
    param documentation = "Host memory";
    param configuration = "required";
}

connect pcie_downstream is map_target {
    param documentation = "PCIe downstream";
    param configuration = "required";
}

template host_to_pcie {
    param pcie_type : pcie_type_t;
    implement transaction_translator {
        method translate(uint64 addr,
                         access_t access,
                         transaction_t *prev,
                         exception_type_t (*callback)(translation_t translation,
                                                      transaction_t *transaction,
                                                      cbdata_call_t cbdata),
                         cbdata_register_t cbdata) -> (exception_type_t) {
            local atom_t atoms[2] = {
                ATOM_pcie_type(pcie_type),
                ATOM_LIST_END,
            };
            local transaction_t t = { .atoms = atoms, .prev = prev, ... };
            local translation_t txl = { .target = pcie_downstream.map_target, ... };
            return callback(txl, &t, cbdata);
        }
    }
}

port host_to_pcie_cfg is host_to_pcie {
    param documentation = "Host downstream PCIe Config transactions";
    param pcie_type = PCIE_Type_Cfg;
}
port host_to_pcie_mem is host_to_pcie {
    param documentation = "Host downstream PCIe Memory transactions";
    param pcie_type = PCIE_Type_Mem;
}

port pcie_upstream {
    implement transaction_translator {
        method translate(uint64 addr,
                         access_t access,
                         transaction_t *t,
                         exception_type_t (*callback)(translation_t txl,
                                                      transaction_t *tx,
                                                      cbdata_call_t cbd),
                         cbdata_register_t cbdata) -> (exception_type_t) {

            local translation_t txl;
            if (ATOM_get_transaction_pcie_type(t) != PCIE_Type_Mem) {
                log info, 1:
                    "Upstream transaction @ 0x%08x, only forwarding PCIE_MEM"
                  + " transactions to host memory", addr;
                return callback(txl, t, cbdata);
            }

            txl.target = host_memory.map_target;
            return callback(txl, t, cbdata);
        }
    }
}
Figure 23. Dummy PCIe Host Bridge

A real Simics PCIe Host bridge shall use the Simics PCIe Modeling Library which does this automatically. See manual PCIe Modeling Library for documentation of the library.

The downstream transaction_translators, (host_to_pcie_mem and host_to_pcie_cfg), do not support deferred transactions. The necessary code to support deferred transactions is outside the scope of this example.
simics> @host_mem = simics.SIM_create_object("set-memory", "host_mem")
simics> @pcie_downstream = simics.SIM_create_object("set-memory", "pcie_downstream")
simics> @dev = simics.SIM_create_object("sample-interconnect-pcie-bridge", "dev", host_memory=host_mem, pcie_downstream=pcie_downstream)
simics> probe-address obj = dev.port.host_to_pcie_mem address = 0x8010001000
Translating virtual address to physical: 0x8010001000 -> p:0x8010001000
┌─────────────────────────┬──────────────────┬─────┬───────────────────────────────────┐
│         Target          │      Offset      │Notes│            Added Atoms            │
├─────────────────────────┼──────────────────┼─────┼───────────────────────────────────┤
│dev.port.host_to_pcie_mem│0x0000008010001000│~    │pcie_type=pcie_type_t.PCIE_Type_Mem│
│                         │                  │     │                                   │
├─────────────────────────┼──────────────────┼─────┼───────────────────────────────────┤
│pcie_downstream          │0x0000008010001000│     │                                   │
└─────────────────────────┴──────────────────┴─────┴───────────────────────────────────┘
'~' - Translator implementing 'transaction_translator' interface
Destination: pcie_downstream offset 0x8010001000 - no register information available
simics> probe-address obj = dev.port.host_to_pcie_cfg address = 0x1000000
Translating virtual address to physical: 0x1000000 -> p:0x1000000
┌─────────────────────────┬──────────┬─────┬───────────────────────────────────┐
│         Target          │  Offset  │Notes│            Added Atoms            │
├─────────────────────────┼──────────┼─────┼───────────────────────────────────┤
│dev.port.host_to_pcie_cfg│0x01000000│~    │pcie_type=pcie_type_t.PCIE_Type_Cfg│
│                         │          │     │                                   │
├─────────────────────────┼──────────┼─────┼───────────────────────────────────┤
│pcie_downstream          │0x01000000│     │                                   │
└─────────────────────────┴──────────┴─────┴───────────────────────────────────┘
'~' - Translator implementing 'transaction_translator' interface
Destination: pcie_downstream offset 0x1000000 - no register information available
Figure 24. Dummy PCIe bridge device inspected using `probe-address` command

21.4.2 Combining Interconnect Elements into a Hierarchy

Because every sample interconnect above utilize the standard memory interfaces part of the Simics product it is possible to connect them together and build a hierarchical memory view. Each interconnect in the path impacts the memory view of the initiator.

In the example below the hierarchy starts with the pcie_bridge. It will add the PCIe Type atom and then forward the transactions to the pcie_router. The pcie_router then forwards the PCIe memory access to the map_demap1 device and PCIe config access to the map_demap2 device. The dummy endpoints dummy_endpoint1 and dummy_endpoint2 sits behind these two devices.

simics> probe-address obj = pcie_bridge.port.host_to_pcie_mem address = 0x820000000
Translating virtual address to physical: 0x820000000 -> p:0x820000000
┌─────────────────────────────────┬──────────────────┬─────┬───────────────────────────────────┬───────────────┐
│             Target              │      Offset      │Notes│            Added Atoms            │Inspected Atoms│
├─────────────────────────────────┼──────────────────┼─────┼───────────────────────────────────┼───────────────┤
│pcie_bridge.port.host_to_pcie_mem│0x0000000820000000│~    │pcie_type=pcie_type_t.PCIE_Type_Mem│               │
│                                 │                  │     │                                   │               │
├─────────────────────────────────┼──────────────────┼─────┼───────────────────────────────────┼───────────────┤
│pcie_router                      │0x0000000820000000│~    │                                   │pcie_type      │
│                                 │                  │     │                                   │               │
├─────────────────────────────────┼──────────────────┼─────┼───────────────────────────────────┼───────────────┤
│map_demap1.memory                │0x0000000820000000│     │                                   │               │
├─────────────────────────────────┼──────────────────┼─────┼───────────────────────────────────┼───────────────┤
│dummy_endpoint1                  │0x0000000000000000│     │                                   │               │
└─────────────────────────────────┴──────────────────┴─────┴───────────────────────────────────┴───────────────┘
simics> memory-map object = pcie_bridge.port.host_to_pcie_mem
┌────────────────────────────────────────────┐
│     pcie_bridge.port.host_to_pcie_mem      │
├───────────┬───────────┬────────────────────┤
│      Start│        End│Object              │
├───────────┼───────────┼────────────────────┤
│0x800000000│0x80000ffff│map_demap1.bank.app0│
│0x820000000│0x820003fff│dummy_endpoint1     │
└───────────┴───────────┴────────────────────┘
simics> memory-map object = pcie_bridge.port.host_to_pcie_cfg
┌────────────────────────────────────────────┐
│     pcie_bridge.port.host_to_pcie_cfg      │
├───────────┬───────────┬────────────────────┤
│      Start│        End│Object              │
├───────────┼───────────┼────────────────────┤
│0x900000000│0x90000ffff│map_demap2.bank.app0│
│0x920000000│0x920003fff│dummy_endpoint2     │
└───────────┴───────────┴────────────────────┘
Figure 25. Combining interconnect elements

21.4.3 Interconnects Adding Security Information Depending on Access Path

Interconnects can carry security information used by ports or receivers to limit the access rights of the initiator. An interconnect could for instance have two access ports one for secure and one for nonsecure accesses. All transactions entering the interconnect on the nonsecure port will have their atom lists amended with a nonsecure tag. Then as the transaction is routed across the system another port, or the receiver itself, can inspect the nonsecure tag and act on it.


dml 1.4;

device sample_interconnect_arm_nonsecure;
param classname = "sample-interconnect-arm-nonsecure";
param desc = "sample interconnect";

import "utility.dml";
import "simics/arch/arm.dml";

connect memory is map_target {
    param documentation = "Host memory";
    param configuration = "required";
}

template nonsecure_translator {
    param nonsecure : bool;
    implement transaction_translator {
        method translate(uint64 addr,
                         access_t access,
                         transaction_t *prev,
                         exception_type_t (*callback)(translation_t translation,
                                                      transaction_t *transaction,
                                                      cbdata_call_t cbdata),
                         cbdata_register_t cbdata) -> (exception_type_t) {
            local atom_t atoms[2] = {
                ATOM_arm_nonsecure(nonsecure),
                ATOM_LIST_END,
            };
            local transaction_t t = { .atoms = atoms, .prev = prev, ... };
            local translation_t txl = { .target = memory.map_target, ... };
            return callback(txl, &t, cbdata);
        }
    }
}

port nonsecure is nonsecure_translator {
    param nonsecure = true;
}
port secure is nonsecure_translator {
    param nonsecure = false;
}

Figure 26. Sample device adding the ARM security atom
The transaction_translators, (secure and nonsecure), do not support deferred transactions. The necessary code to support deferred transactions is outside the scope of this example.

21.4.4 Interconnects Supporting Deferred Transactions

Interconnects passing through transactions only has to take deferred transactions into account when it adds atoms within transaction_translator interface calls. The previous sample devices sample-interconnect-pcie-bridge and sample-interconnect-arm-nonsecure are such examples. See details about deferred transaction in Transactions.

When a transaction is deferred the atoms can no longer reside on the stack but must instead be moved to the heap so the endpoint can access them asynchronously outside the transaction interface call stack. Secondly to support checkpointing the heap allocated atoms must be serialized and deserialized during storing and loading of a checkpoint.

To support deferred transactions in the sample-interconnect-arm-nonsecure device the VECT library is used to store the deferred transactions and its atoms in a linked list. Attribute chained_transactions, is added that implements get/set to serialize and deserialize the linked list of deferred transactions.


dml 1.4;

device sample_interconnect_arm_nonsecure_deferred;
param classname = "sample-interconnect-arm-nonsecure-deferred";

param desc = "sample interconnect deferred transactions";

import "utility.dml";
import "simics/arch/arm.dml";

extern void VADD(...);
extern void VREMOVE_FIRST_MATCH(...);
extern void VFREE(...);
extern int VLEN(...);

connect memory is map_target {
    param documentation = "Host memory";
    param configuration = "required";
}

typedef struct {
    transaction_t t;
    atom_t atoms[4];
} sample_transaction_t;

template security_translator {
    param nonsecure : bool;
    implement transaction_translator {
        method translate(uint64 addr,
                         access_t access,
                         transaction_t *prev,
                         exception_type_t (*callback)(translation_t translation,
                                                      transaction_t *transaction,
                                                      cbdata_call_t cbdata),
                         cbdata_register_t cbdata) -> (exception_type_t) {
            local translation_t txl = { .target = memory.map_target, ... };
            local sample_transaction_t st;
            st.t.atoms = st.atoms;
            st.t.prev = prev;

            st.atoms[0] = ATOM_arm_nonsecure(nonsecure);
            st.atoms[1] = ATOM_owner(dev.obj);
            st.atoms[2] = ATOM_completion(&empty_completion);
            st.atoms[3] = ATOM_LIST_END;
            local exception_type_t exc = callback(txl, &st.t, cbdata);
            /* Transaction got deferred, atoms now have to be moved to the heap */
            if (exc == Sim_PE_Deferred) {
                local sample_transaction_t *new_st = new sample_transaction_t;
                memcpy(new_st->atoms, st.atoms, sizeof(st.atoms));
                new_st->t.atoms = new_st->atoms;
                new_st->t.atoms[2] = ATOM_completion(&completion);
                new_st->t.prev = prev;

                SIM_replace_transaction(&st.t, &new_st->t);
                VADD(chained_transactions.val, new_st);
                return SIM_monitor_chained_transaction(&new_st->t, exc);
            } else {
                // Cannot return exc directly here. If the parent transaction_t
                // has a completion atom that it expects to be called,
                // it will only be called if SIM_monitor_chained_transaction
                // is called by this transaction.
                return SIM_monitor_chained_transaction(&st.t, exc);
            }
        }
    }
    method empty_completion(transaction_t *t, exception_type_t e)
        -> (exception_type_t) {
        return e;
    }
    method completion(transaction_t *t,
                      exception_type_t e) -> (exception_type_t) {
        local sample_transaction_t *st = cast(t, sample_transaction_t *);
        VREMOVE_FIRST_MATCH(this.chained_transactions.val, st);
        delete st;
        return e;
    }
    /* Internal attribute to store all deferred outstanding transactions for the port.
     * The set()/get() methods serializes and deserializes the atom list
     * and the transaction ID. The ID is used in function SIM_reconnect_transaction()
     * during checkpoint load to reconstruct the deferred transaction.
     */
    attribute chained_transactions {
        param type = "[[ib]*]";
        param internal = true;
        session sample_transaction_t * vect val;

        method set(attr_value_t value) throws {
            if (!SIM_is_restoring_state(obj)) {
                SIM_attribute_error("must only be set during checkpoint restore");
                throw;
            }

            for (local int i = 0; i < VLEN(this.val); i++)
                delete this.val[i];

            VFREE(this.val);

            for (local int i = 0; i < SIM_attr_list_size(value); i++) {
                local attr_value_t li = SIM_attr_list_item(value, i);
                local int64 id = SIM_attr_integer(SIM_attr_list_item(li, 0));

                local sample_transaction_t *st = new sample_transaction_t;
                st->t.atoms = st->atoms;
                st->atoms[0] =
                    ATOM_arm_nonsecure(SIM_attr_boolean(SIM_attr_list_item(li,
                                                                           1)));
                st->atoms[1] = ATOM_owner(dev.obj);
                st->atoms[2] = ATOM_completion(&completion);
                st->atoms[3] = ATOM_LIST_END;

                VADD(this.val, st);
                SIM_reconnect_transaction(&st->t, id);
            }
        }

        method get() -> (attr_value_t) {
            local attr_value_t value = SIM_alloc_attr_list(VLEN(this.val));
            for (local int i = 0; i < VLEN(this.val); i++) {
                local transaction_t *t = &this.val[i]->t;
                local attr_value_t li = SIM_make_attr_list(
                    2,
                    SIM_make_attr_uint64(SIM_get_transaction_id(t)),
                    SIM_make_attr_boolean(ATOM_get_transaction_arm_nonsecure(t)));
                SIM_attr_list_set_item(&value, i, li);
            }
            return value;
        }

        method destroy() {
            // Deferred transactions are completed by Simics Core,
            // we just need to free the vect.
            VFREE(this.val);
        }
    }
}

port secure is security_translator {
    param nonsecure = false;
}

port nonsecure is security_translator {
    param nonsecure = true;
}

method destroy() {
    secure.chained_transactions.destroy();
    nonsecure.chained_transactions.destroy();
}

Figure 27. Sample device adding the ARM security atom with deferred transaction support
20 Modeling I2C Devices 22 Representing Network Packets with frags_t