The bank-coverage-tool provides register coverage for DML devices.
Device register coverage can be thought of as code coverage for modeled registers. The tool can aggregate register accesses over several Simics runs by saving intermediate results to a file and summarize the total coverage figures for each device through the command line interface. The tool can also export results to CSV.
The tool makes use of the bank instrumentation framework to collect register accesses and uses the general instrumentation framework and its familiar interfaces, documented in this section, to connect to register banks. Please see the API Reference Manual for more details about the bank instrumentation framework, which the tool also relies on, and how to develop your own tools using this framework.
By connecting the tool to banks of interest, or using the -connect-all flag when instantiating the tool, the tool will track all accesses to connected banks and summarize the number of accessed registers in a device as a percentage to the number of its registers in total. We call this the 'coverage' of a device. For further inquiry, the tool may also present the accrued number of accesses to individual registers in a connected bank. However the register accesses are presented, they may be filtered on read accesses, write accesses, or both. Both commands may be interacted with in this way and the output may be sorted and mutated in a variety of ways.
In order to instantiate the tool, use the new-bank-coverage-tool command and provide any banks you want to connect to the tool. Alternatively, you can use the -connect-all flag to connect it to any supporting banks in the system. Additional banks can be connected using the add-instrumentation command on the tool.
When the configuration is loaded, instantiate and connect the bank coverage tool to all banks in the system like so:
simics> new-bank-coverage-tool -connect-all
Created coverage_tool0 (connected to 75 banks)
Once the tool is instantiated we may run the workload, which in this case is booting Linux. At any time we may stop the simulation and use the coverage command to present the coverage so far, which is ordered by the coverage percentage by default:
simics> coverage_tool0.coverage -write max = 10
┌─────────┬─────────────────────────┬──────────────────────┬─────────────┬─────────┐
│ Row # │ Device │No. accessed registers│No. registers│Coverage%│
├─────────┼─────────────────────────┼──────────────────────┼─────────────┼─────────┤
│ 1│board.mb.sb.spi_flash_obj│ 0│ 0│ 100.00%│
│ 2│board.mb.sb.com[0] │ 5│ 8│ 62.50%│
│ 3│board.mb.sb.com[2] │ 5│ 8│ 62.50%│
│ 4│board.mb.sb.com[1] │ 5│ 8│ 62.50%│
│ 5│board.mb.sb.com[3] │ 5│ 8│ 62.50%│
│ 6│board.mb.sb.hpet │ 15│ 28│ 53.57%│
│ 7│board.mb.sb.bridge │ 17│ 38│ 44.74%│
│ 8│board.mb.sb.uhci[4] │ 19│ 50│ 38.00%│
│ 9│board.mb.sb.uhci[2] │ 19│ 50│ 38.00%│
│ 10│board.mb.sb.uhci[0] │ 19│ 50│ 38.00%│
├─────────┼─────────────────────────┼──────────────────────┼─────────────┼─────────┤
│# 10/44 │ │ │ │ │
│Sum (all)│ │ 830│ 3391│ │
│Sum shown│ │ 109│ 248│ │
└─────────┴─────────────────────────┴──────────────────────┴─────────────┴─────────┘
Note that we're limiting the output by using the max parameter. The output of the bank coverage tool commands may be morphed like this in many ways. It's left as an exercise to the user to explore these possibilities further.
A user may also export the attained coverage to CSV by using the to-file parameter:
simics> coverage_tool0.coverage -write max = 10 to-file = ~/coverage.csv
This will output a CSV file like so:
Device,No. accessed registers,No. registers,Coverage%
board.mb.sb.spi_flash_obj,0,0,1.0
board.mb.sb.com[0],5,8,0.625
board.mb.sb.com[2],5,8,0.625
board.mb.sb.com[1],5,8,0.625
board.mb.sb.com[3],5,8,0.625
board.mb.sb.hpet,15,28,0.5357142857142857
board.mb.sb.bridge,17,38,0.4473684210526316
board.mb.sb.uhci[4],19,50,0.38
board.mb.sb.uhci[2],19,50,0.38
board.mb.sb.uhci[0],19,50,0.38
...
Furthermore, we may inspect the register accesses, of any connected bank, of every device presented in the coverage output by using the access-count command:
simics> coverage_tool0.access-count board.mb.sb.hpet.bank.regs -write sort-on-column = "Count" max = 10
┌─────────┬───────────────┬──────┬────┬──────┐
│ Row # │ Name │Offset│Size│Count │
├─────────┼───────────────┼──────┼────┼──────┤
│ 1│tim_comp[2] │ 0x148│ 8│529647│
│ 2│tim_comp[3] │ 0x168│ 8│ 19140│
│ 3│gen_conf │ 0x10│ 8│ 6│
│ 4│fsb_int_rout[2]│ 0x150│ 8│ 6│
│ 5│tim_conf[1] │ 0x120│ 8│ 5│
│ 6│tim_conf[0] │ 0x100│ 8│ 5│
│ 7│tim_conf[2] │ 0x140│ 8│ 5│
│ 8│tim_comp[0] │ 0x108│ 8│ 2│
│ 9│tim_comp[1] │ 0x128│ 8│ 2│
│ 10│main_cnt │ 0xf0│ 8│ 2│
├─────────┼───────────────┼──────┼────┼──────┤
│# 10/15 │ │ │ │ │
│Sum (all)│ │ │ │548825│
│Sum shown│ │ │ │548820│
└─────────┴───────────────┴──────┴────┴──────┘
Note that we're sorting the table based on the number of accesses rather than offset, which is the default.
If the tool were to be used to compile the total coverage figures over a set of runs, such as over several tests, a user may utilize the save command to temporarily save the results to disk:
simics> coverage_tool0.save ~/coverage_0.bin
...and subsequently use the load command to aggregate the results:
simics> coverage_tool0.load ~/coverage_0.bin
simics> coverage_tool0.load ~/coverage_1.bin
simics> coverage_tool0.load ~/coverage_2.bin
...
Once loaded, the coverage and access-count commands will act as if all the accesses had happened during the same run and present the combined results accordingly. In this way, a user may compile the aggregated results of an entire test suite, which should prove useful.