appnote-teaching hardware-software integration with Simics 2008-9-something v0.9.5

8/7/2019 appnote-teaching hardware-software integration with Simics 2008-9-something v0.9.5

1/48

VIRTUTECH APPLICATION NOTE

TEACHING HARDWARE-

SOFTWARE

INTEGRATION WITH

SIMICS

VERSION 0.9.5

JAKOBENGBLOM

WWW.VIRTUTECH.COM
mailto:[email protected]:[email protected]://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/http://www.virtutech.com/mailto:[email protected]:[email protected]


2/48

TEACHING HARDWARE-SOFTWARE INTEGRATION WITH SIMICS

INTRODUCTION

This application note documents a teaching setup for Virtutech Simicsintended to teach hardware-software integration at a system level. Thesetup consists of an MPC8641D-based virtual platform, running Linux,into which students add a custom hardware unit, a Linux driver for thehardware unit, and user-level software that makes use of the customhardware unit.

The setup consists of the following pieces:

The target machine itself, an MPC8641D virtual platform. Theversion used is mpc8641d-simple in Simics 4.0.

U-Boot version 1.3.0 for booting the target machine. A Linux kernel version 2.6.23, configured for the target machine. A ramdisk-based target file system containing a Busybox version

1.12.1 setup.

An example Simics simple_device device model. A Linux char driver for simple_device, configured to be built as

a loadable kernel module.

A test program activating the simple device over the device driver. Simics scripts to automate the loading of software and hardwareinto the virtual target platform. A cross-compiler for building target software, kernel, busybox, and

u-boot.

An example target application for which to create hardwareacceleration. Currently, this is a rule30 cellular automatonimplemented in C.

PROJECT IDEAS

The following is a set of suggested projects that can be done using this

setup, teaching various aspects of hardware-software integration.

Designing a Hardware AcceleratorStudents should use the simple_device and its driver as the starting point forthe creation of a memory-mapped device that accelerates the execution ofthe target application rule30 (or some other application of choice).

The memory-mapped device should be created at the default Simicstransaction-level of detail. Timing should be handled by assuming orprescribing some kind of delay from when data is entered until it is

Page 2 / http://www.virtutech.com 2008 Virtutech


3/48


4/48


Teaching Virtual Platform ModelingThe setup could also to teach hardware modeling for virtual platforms.Students might be given the specification for some real block in theMPC8641D, and tasked with implementing a functional model for theblock. With complete driver code in source, debugging the hardware-software interface and understanding the requirements of the software isgreatly facilitated. The example simple_device is really too simple to beuseful for this purpose, the students should work on some real hardwarecomponent.

AVAILABILITYThe software setup will be made available from Virtutech as adownloadable archive. It should be noted that it is fairly extensive, since itincludes source code for all relevant components. The complete filearchive is on the order of 1 GB.

Host MachineThe setup requires a Linux host, due to the need to cross-compile theLinux kernel and related software, it is very hard to port to Windows in areasonable way. It has been tested on a Kubuntu 7.04 setup.

Host CompilerThe host machine needs to have the tools available to build Simicsmodules, which means that it needs to have a host C compiler installed.Please refer to the Simics installation guide for more information.

Simics InstallationVirtutech Simics is supposed to be installed in the default way, whichwould put it at /opt/virtutech/simics-4.0/. This does not matter onceworkspace setup for the Simics workspace has been performed. All workis done in the user workspace.

The Simics installation needs to at least have the following packagesinstalled:

Simics base 4.0.x Simics model builder 4.0.x Simics MPC8641D-simple 4.0.x

Please refer to the Simics installation guide for more information on howto install packages.



5/48


OVERVIEW

The complete setup is fairly complex, as shown in Figure 1, but the coursefocus is just on three parts, all the other parts are really just scaffoldingnecessary to provide a realistic system environment:

The device being developed The device driver for the device The program using the device via the device driver

Figure 1. The complete virtual system setup

SimicsSimics

VirtualMPC8641DBoardMPC8641DSoC

CPUcores

0

and

1

Linux2.6.23kernel

The areas of primary interest for teaching are shown with bold outlines inFigure 1. In all likelihood, the virtual network will not be used, but it isthere if needed. The simicsfs file system provides an easy way to getsoftware and data into the target machine, without having to rebuild theinitial ramdisk file system (which is easy with the provided scripts, butfairly time-consuming anyway).

Busybox

Devicedrivers

Hfsdevice

UART

Simpledevicedriver

Programusingsimpledevice

Ethernet

Simicsfs

driver

MPIC

Simple_device

MemCtrl

RAM

PCI PCIe

Timers

Other

program

HostcomputerHostoperatingsystem

Hostdisk

Hostfilesystem

Virtualserialconsole

Virtualnetwork

Simicsscripting



6/48


All compilation of software for the target will be done outside of theSimics target machine, using cross-compilation tools. This is how mostembedded software is developed in practice today.

DIRECTORY LAYOUT

The setup contains a complete embedded system, and as such it is fairlycomplex. The file set is supposed to be put in some main directory, whichwe call base in the following discussion.

The relevant directories in base are shown in Figure 2.

Figure 2. Directories in the base directory

base/cross-compiler/ -- the cross-compiler to be used to compile all code base/documentation/ -- documentation about the target machine and Simics. base/initrd-busybox/ -- the ramdisk construction and busybox build environment. The top-

level contains the files needed to build and modify the ramdisk (initrd). base/initrd-busybox/busybox-1.12.1/ the build environment for the busybox binary

base/linux-kernel/linux-2.6.23/ the Linux kernel build area base/linux-kernel/linux-2.6.23/drivers/char/simple_example_driver our example

driver for the simple device base/linux-kernel/linux-2.6.23/fs/simicsfs the Simicsfs file system driver that

allows the target Linux system to mount the host file system. You should not needto care about this, but it is good to know that it exists.

base/simics-workspace/ the Simics 4.0 workspace used to run Simics and build models. base/simics-workspace/modules/simple_device/ the source code for the simple

device model base/simics-workspace/targets/mpc8641-simple/ the scripts to run and start the

MPC8641D machine used in this setup, as well as scripts to load new hardwareand drivers and test software onto the machine

base/simics-workspace/targets/mpc8641-simple/images/ the bootable U-Boot andLinux and initrd images.

base/simics-workspace/targets/mpc8641-simple/target-code/ here is where targetcode is located and built

base/simics-workspace/targets/mpc8641-simple/target-code/simple_device_test/ atest program that uses the Linux driver to use the simple example device

base/simics-workspace/targets/mpc8641-simple/target-code/rule30/ one example ofa program for which to design a hardware accelerator

base/u-boot/u-boot-1.3.0/ the U-Boot build setupFigure 3 shows a screenshot of a file browser looking at the file structure.



7/48


Figure 3. Browsing the file structure

HOST ENVIRONMENT VARIABLES

To use the provided software with minimum hassle, please add thefollowing definitions to your logon file, like .bashrc:

Figure 4. Environment Variable Setup## Assume that the HSI file structure is under ~/export ARCH=powerpcexport CROSS_COMPILE=~/cross-compiler/powerpc-linux-gnu/bin/powerpc-linux-gnu-export PATH=$PATH:~/u-boot/u-boot-1.3.0/tools/

The paths might of course be different on your machine. The path to thecross-compiler is automatically picked up from the environment variableby all parts that need to use it.



8/48


RUNNING SIMICS

To give a feel for how things work with this setup, we should start byrunning Simics with some scripts.

On a Linux host, Simics is easiest to start using scripts provided in thesimics-workspace. To start Simics in GUI mode, do the following:

Figure 5. Starting Simics in GUI mode

base$ cd simics-workspace

base/simics-workspace$ ./simics-gui &

Once the Simics GUI has started, you should then select New Session fromScript and select a start script. We suggest you start with the script calledmpc8641-simple/mpc8641-hsi-course-setup.simics, as shown in Figure 6.

Figure 6. Starting a Simics simulation session from the GUI

Once this script has started, you will see a new window on the screen,which is the serial console of the target machine. Nothing has appeared yetthere, since the simulation is not yet started. Figure 7 shows the state atthis point, and also identifies some key windows.



9/48


Figure 7. Simics has started a new simulation session

Press the start button in the GUI to start running the target. Note that youcan pause the simulation at any point in time and resume it.

If you wait a while, the target system will boot into U-Boot, load thekernel and ramdisk into memory, boot the Linux kernel, mount theramdisk, and get to a login prompt.

The Simics command-line window will be issuing some warning aboutbad accesses to various places in memory. This is completely normal, andmost of it is hardware probing where device drivers try various things tosee how the hardware reacts. Also, hardware is quite commonly veryforgiving about minor mistakes in accesses (such as attempting to changethe value of read-only bits in registers), while Simics presents warnings.On physical hardware, these kinds of issues were never noticed, and thedrivers therefore have not been patched to correct them.



10/48


Simics has scripted the bootup and login process, and the boot will finishwith you logged into the Linux running on the virtual MPC8641D target.

You can try various simple commands as shown in Figure 8 to familiarizeyourself with the target system.

Figure 8. Example commands on target system

~ # cat /proc/interrupts~ # ls /bin~ # cat /proc/cpuinfo~ # cat /proc/devices~ # ls l /dev~ # ping 10.10.0.1~ # df

Checkpoint and restoreA key feature of Simics is checkpointing. To try this, do the following:

Pause the simulation Select Save Checkpoint from the file menu, and save the

checkpoint anywhere you like. Select New Empty Session from the file menu, which will remove

the current simulation. Select Open Checkpoint from the file menu, and select the

checkpoint you save previously. You should now be back at the exact same place.

Note that there is already a checkpoint provided in the course materialsthat is used by other scripts to start simulation from the point where thetarget system is booted.

Reverse executionAnother interesting Simics feature is reverse execution. To try this, do the

following:

Enable reverse execution by clicking the right-most button in controlwindow ( ). You should see reverse execution enabled in thestatus bar.

Start simulation again. In the target console, do something really destructive like rm /bin/*.

Let this run its course, and note that the system becomes prettybroken. Nothing works.



11/48


Figure 9. Breaking the target system

Stop the simulation and click the reverse button. Simics will now run the action backwards. You can click forward to replay it. To actually do something different, reverse back to the state before

the rm command, and then select Clear Input Recording from the Runmenu. This will make Simics forget all you did, and you can now dosomething different.

Note that Simics does not save the changes you make to the target systemdisk, so there is no risk that you can wreck future runs by making mistakesinside a simulation run. The reason for this is both that you need toexplicitly ask Simics to save any changes to target system disks in orderkeep them (and they would only be saved as disk differences from theoriginal state), and that the kind of initial ramdisk disk used in this setupcannot be saved. It is loaded into memory and uncompressed during theboot, and its state in ram is not possible to compress back down to a filesystem for future reference.



12/48


Simicsfs host disk accessThe target system we are using has been configured with a special Simicsfeature that allows the target Linux to access the disk on the host machine.This is very handy to copy code and data into the target.

To try this, start Simics from the script mpc8641-hsi-course-booted.simics,which will open a checkpoint of a booted system. Then, in the targetconsole, do mount /host and then ls to see the host file system.

If you want to change where the starting point for the simicsfs file systemon the host file system is, you need to use the Simics command-line

window and the command hfs.root.

Figure 10 shows an example session using simicsfs.



13/48


Figure 10. Use of simicfs to mount the host

Once you have copied the files you need into the target system, you shouldunmount the host file system using umount /host. The reason is that thispreserves the deterministic and repeatable semantics of Simics mountingthe host introduces a non-controllable factor into the simulation.

Try a complete integrated test runSimics has a very powerful script environment, and this is used in thisteaching setup to create short turn-around times. The most complete suchscript is hsi-course-booted-load-driver-and-test.simics. Start a newsession using this script, and Simics will do the following:

Start from a checkpoint of a booted machine Add in the simple_device device model Mount the host and copy the driver code from the host insmod the driver into the Linux kernel



14/48


Create the /dev/ node needed to access the driver Copy the test program from the host Run the test program Stop the simulation run

The script is shown in Figure 11. Note how it works by calling otherscripts, running the simulation for a while (using continue, the scriptscalled will call stop to stop the simulation).

Figure 11. Script to run a complete test run

## Start from the checkpoint, and then load the driver,# and then run the test program that exercises the driver#run-command-file "%script%/mpc8641d-hsi-course-booted.simics"

## Load the driver# -- Provide argument here to give it an address#$insmod_arguments = "phys_addr=0xf80f0000"run-command-file "%script%/hsi-course-load-driver.simics"continue

## The load driver script finishes once the driver is loaded

#run-command-file "%script%/hsi-course-load-test-program.simics"continue

In this way, testing a new set of device model, device driver, and testprogram is really very simple. There is no manual work involved otherthan running the script.

Note that all of the steps above can be run from the Simics command-lineinteractively.

For more on Simics scripting, please see the Advanced Features of

Simics document in the Simics documentation. In particular, look intohow to script the serial console and the use of script branches.



15/48


THE SIMPLE_DEVICE MODEL

The simple_device is written in DML, the Virtutech Device ModelingLanguage. As shown in Figure 12, DML is compiled into C code, which isthen compiled by a host C compiler into a dynamically loadable library(shared object on Linux hosts) that can be loaded into the Simicssimulator.

Figure 12. DML compilation process

Simics target machine

Target operating system

Target firmware

User program

DevicesDevicesDevicesDevices

IO,

Links,Inter-

connects

DevicesMemories DevicesProcessors

DMLmodelsource files

DMLtemplates& library

DML Compiler

C files

C Compiler

Simicsheader files

Simicsmodule DLL

Since you will be using DML in this course, you should look into thedocumentation provided about DML.

In the documentation directory, you will find a whitepaper on DML togive an introduction.

There is also a whitepaper on modeling that presents the Simicsmodeling philosophy in some detail.

After this, you should refer to the Simics documentation. To do this,you can start Simics in GUI mode, and then refer to the help menu(see below for how to start Simics, and Figure 13 for how this canlook). You can also find the documentation at /opt/virtutech/simics-4.0/simics-model-builder-4.0.x/doc, for example in order to print it out.The most important manuals are:

Modeling your System in Simics DML 1.2 Reference Manual



16/48


Figure 13. Help from the Simics GUI

Building the Device ModelTo rebuild the device model, you go to base/simics-workspace, and issuemake. Note that you do not need to go down into modules directory to dothis. Figure 14shows an example session in an xterm on Linux.



17/48


Figure 14. Building the Simics device model

Once the device model has been built, it can be loaded into Simics usingthe script hsi-course-add-simple-device.simics. This script creates a newdevice object in Simics and sets the crucial parameters for its integrationinto Simics: the name of the processor it is attached to for simulationpurposes, and the target for its interrupts. Figure 15 shows the Simicscommand-line code needed to create a device object and map intomemory. The line starting with @ is inline Python code, which is oftenused in Simics scripts to do complicated things.

Figure 15. Device mapping script

load-module simple_device@SIM_create_object("simple_device","sd0",[["queue",conf.cpu0],["irq_dev",[conf.pic,'internal_interrupts']],["irq_level",23], ["time_to_result", 1e-3]])ccsr_space.add-map sd0:regs 0xf0000 0x100

The device model in the target memory mapThe device model is mapped into the target memory at address 0xf80f_0000as seen from the processors. How this is done is shown in Figure 15. Thisaddress is in the range of the on-chip simple peripherals of the MPC8641D



18/48


SoC, and the device is Simics-technically mapped into a memory spacecalled ccsr_space, which is at offset 0xf800_0000 in the main physicalmemory space map. Inside this space, it is located at 0xf_0000. There isroom until 0xf_ff00, where the pseudo-device for the simicsfs system ismapped, which should be plenty of room for memory-mapped registers foran accelerator.

Note that you can map into any other free location, there is nothingmandatory about this. But it works. To see all mappings in Simics, use theMemory Mappings Browser found on the Tools menu. Figure 16 showshow this looks after the device has been mapped. Our simple device object

is called sd0, keeping with Simics style of using fairly short objectnames, and the :regs bit indicates that the register bank in the device thatis being mapped here is called regs. Look inside the source code insimple_device.dml and you will find the declaration of this register bank.



19/48


Figure 16. Memory map

To further look into the device, you can either look at the source code oruse the Simics Device Registers window you find on the Debug menu.Actually, look at both and see how they relate. There is a also an ObjectBrowser available that presents the Simics implementation view of theobjects in the simulation, as shown in Figure 21.

Note that mapping has a defined length, and this is set in the script creatingthe device, in the map command that maps it into ccsr_space as shown inFigure 15.



20/48


Figure 17. Device register view

Interrupting the CPUThe simple device model is setup to generate interrupts to the processorwhen an operation is complete. To do this, it currently uses interruptnumber 23 to the MPIC main interrupt controller. This is a numberborrowed from some unused on-chip devices, since the Linux kernel washard to convince to accept any IRQ number. Note that the Linux kernelconsiders this to be IRQ 39, since the first 16 IRQ numbers are dedicated

to the external i8259 interrupt controller also found on the system.

You should not need to change this part of the device model or its setup.Just use the facilities provided in the simple device code to raise and lowerinterrupts. How to do this should be clear from the source code.

Understanding the device functionalityThe simple device code is intended as a starting point for your ownaccelerator device. Its current functionality is very simple: it performs aconfigurable (using the control register) operation combining all valueswritten to its input register.

The result is put into the accumulator register.

The target software is expected to check the operation completed (OC) bit inthe status register before reading the results. This bit is set with a delayfrom the last write to input (note that input accepts data at any rate). Onlyafter this delay has passed will the device model set the OC bit and send aninterrupt signaling completion.



21/48


Note that the result in this simple model is computed immediately, but thatit is not considered valid. You could also model this by not setting thevalue of the accumulator register until the final result is due, but that wouldrequire some kind of internal register to hold the result until then. Thistype of simplified modeling works well most of the time, especially whenwe know that the target software is well-behaved.

The delay is configured using the time_to_result attribute of the devicemodel, and is set from the Simics command-line (for experiments, you canset it using the Simics command sd0->time_to_result = TIME at any point intime from the command-line window),

The delay is implemented by using Simics events, posted into the future sothat the device model gets a callback to event() method inside the eventoperation_complete object in the DML file.

The version register is used by the device driver to check that the device isreally what it expects and to report the version number of the devicehardware to the software.

The irq_num register is a bit of cheat, as it communicates the interruptconfiguration from the hardware device to the software driver, by peeking

at the system setup in Simics. In physical hardware, this might be hard todo depending on the nature of the interrupt controller and system.

Device info and status commandsSimics has a very powerful command-line interpreter, which is fullyextendible by users like you. For the benefit of users, all devices aresupposed to implement two custom commands to expose its state andsetup.

These commands are defined in the file base/simics-workspace/modules/simple_device/commands.py, which is a Python-language

source file. If you do not know Python, there is no reason to worry. It is avery simple language to pick up if you know any language in the C family,and the structure of the commands.py file makes it very simple to extend it.Basically, what is done in the info and status commands is to readvalues of attributes in your Simics device model, and present these to theuser with an explanatory text, and there is support for this in Simics core.All you supply is a list of heading-attribute name pairs.

The output is viewed both in the Simics object browser (as shown inFigure 18), and from the command-line using sd0.info/sd0.statuscommands. Note that there is no static checking of the command code, so



22/48


if you use a bad name for an attribute, it will not show up as an error untilSimics runtime when you try to use the commands.

Figure 18. Simics Object Browser: Status Command



23/48


THE DEVICE DRIVER

The next part of the puzzle is the device driver. This is a Linux kernelmodule that talks directly to the hardware driver, and exposes a chardevice interface to the software.

A good starting point to understand Linux device drivers is the book calledLinux Device Drivers, 3

rdedition. It can be bought in paper form, or

downloaded from http://lwn.net/Kernel/LDD3/ . It is highly recommendedto have some such reference handy when working with the device driver.

The code is found at:

base/linux-kernel/linux-2.6.23/drivers/char/simple_example_driver/

It consists of several files:

simple.h is the common header simple.c contains the initialization code simple_hw.h/.c contains the hardware mappings and hardware-

interface of the driver, including a set of convenience functions. simple_file_ops.h/.c contains the file operations facing the user-level

software. README provides more details and suggestions for improvements to the

driver.

Building the driverThe driver is built by going to the base/linux-kernel/linux-2.6.23/directoryand issuing make (after making some change to the driver). This willresult in a new simple_example_driver.ko loadable kernel object that can beused on the target machine.

http://lwn.net/Kernel/LDD3/http://lwn.net/Kernel/LDD3/


24/48


Figure 19. Building the device driver

An example of how to build is shown in Figure 19.

Verbose buildIf you want to check on the detailed invocation of the cross-compiler onthe device driver code, you use the command make V=1 instead. V=1 tellsthe Linux kernel make system to be verbose and tell you all it does and thecomplete command-line invocation of compilers.

Loading the driver onto the target

To load the driver onto the target, you need to first have a bootedMPC8641D target machine configured with the simple_device in place.You get this from the mpc8641d-hsi-course-booted.simics script.

The script hsi-course-load-driver.simics can then be used to load thedriver on the machine. It automates several steps on the target machine:

Mounting host, copying the most recent driver in, and unmounting thehost.

Doing an insmod to make the kernel load the driver, and providing andargument to insmod to tell the driver where to find the hardware.



25/48


Creating the device node /dev/simd so that programs can access thedevice, using mknod.

Linux uses device numbers to call the right driver for the right node in/dev/. For this setup, we use the major number 240 and minor 100, since itwill only be used in this system, and thus it is fine to use one of thenumbers set aside for local use. There is no need to complicate things withautomatic assignment of dynamic device numbers at this point in time.

Using the device via the Linux file systemThe way to use the device is via regular Linux file system calls. Our

device is used in the following way:

Do open() on /dev/simd to get a file handle to the device Use write() to send data to the device Use read() to get results back. Read will block until the results are

available, as indicated by the status.OC bit in the hardware. Finally, close() the file.

The test program described below does exactly this.

Modifying the device driver

The device driver will need to be modified as the device hardware designchanges. It is suggested that new hardware registers are declared in theheader file simple_hw.h, and that all interaction with the hardware is keptlocal to simple_hw.c. The file operations in simple_file_ops.c should callfunctions in simple_hw.c, unless it turns out that performance requires somecode to be inlined in the file operation code.

The driver code is extensively commented to help you find your wayaround.

The example code for the Linux Device Drivers book is a good source of

snippets of code to extend the driver.

Note that as noted in the README file, the device driver has plenty of roomfor added functionality.



26/48


THE TARGET PROGRAMS

The test programs intended to run on the target machine are found in theSimics workspace, at targets/mpc8641-simple/target-code/. There are twoprograms currently:

rule30_cellular_automaton is a program that should be accelerated. simple_device_test is a test program that uses the device driver and

example simple device.

Building target programs

To build them, go to their respective directory and issue make. Figure 20shows how a build looks. The programs are cross-compiled for theMPC8641D target using the cross compiler provided.

Figure 20. Building a target program

The target programs are built to have an extension .elf, to make it easy tosee that they are not host programs (since they are cross-compiled).



27/48


The file called Makefile.generic in the target-code/ directory contains thegeneric rules for building target programs, and the Makefile files in eachprogram directory lists the files which are part of each program.

Running target programs on the targetTo run a target program on the target, you need to have a booted targetmachine. If you want to use the device and its device driver, you shouldalso make sure that these have been setup. You then use simicsfs to copythe target software from you host to your target, and run it on the target.

Note that once a program has been copied into the target disk, it can be

rerun any number of times. There is no need to recopy it unless you havechange the code in some way.

The provided programs all use command-line parameters for configurationand flexibility. The driver test program accepts the /dev/ node to use, andthe rule30 program the number of generations to run.

There are Simics scripts provided to automate the loading of programs:

hsi-course-load-rule30.simics loads the rule30 program into a bootedtarget and runs it.

hsi-course-load-test-program.simics loads the test program into amachine that has configured the device driver. hsi-course-booted-load-driver-and-test.simics does a complete test run

from a checkpoint, loading the driver, and the test program.

Note that you can run scripts like hsi-course-load-rule30.simics from theSimics GUI using the command Append from Script on the File menu,after the machine has been booted or a checkpoint opened.

Modifying the target programsThe target programs are standard Unix command-line programs. The

cross-compiler and target file system contain the standard C libraryfunctions, so there are no obvious limitations due to the nature of thetarget.



28/48


THE LINUX KERNEL

In most cases, students should not need to rebuild or change the Linuxkernel used in the course setup. However, all the source code is provided ifit would prove needed. It can also be quite instructive to actually do somereconfiguration and recompilation of the kernel, just to see how it is done.

The present kernel configuration has been configured for use on theparticular simulation setup we have in place only, with certain devicedrivers that generated warnings about accesses to non-existing memoryand similar deactivated.

All device drivers and features being used are configured as staticallyloaded modules, i.e., part of the compiled kernel binary. This is typical forhow embedded systems are usually configured, simply because you knowthe hardware you are running on and the applications it is supposed to run.Desktop and server setups tend to use loadable modules far more.

Note that since we are using dynamic module loading for our devicedriver, all the support for loading modules dynamically is configured to bepresent.

Building the KernelTo build the kernel, go to base/linux-kernel/linux-2.6.23/ and enter thecommand make, just like when building the device driver.

To reconfigure the kernel, issue make menuconfig, and you should get theinteractive setup shown in Figure 21.



29/48


Figure 21. Make menuconfig for Linux kernel

To use the new kernel in the Simics simulation setup, you need to copytwo files to the Simics workspace file tree:

uImage, which is the U-Boot compressed image used in the actualtarget system. This file is found in base/linux-kernel/linux-2.6.23/arch/powerpc/boot/.

vmlinux, which is the ELF-format file for the kernel that is used todebug the kernel (it contains symbols and is readable for debuggerslike Simics and gdb). This file is found on the top-level Linux builddirectory, at base/linux-kernel/linux-2.6.23/

The two files need to be copied to base/simics-workspace/targets/mpc8641-simple/images/ to be picked-up and used by the Simics scripts. Note thatany checkpoints created using an earlier kernel version should bediscarded and regenerated these have already loaded the kernel intomemory, and will not change their kernel setup just because a file in thehost file system changed.



30/48


Enabling Debug Information in the KernelThe Linux kernel per default does not generate debug line numberinformation, only symbols (this means that you can tell which function youare in, but not where in that function).

To get more information for symbolic debug, you need to explicitly enabledebugging information in kernel setup. This is done under Kernel Hackingin the setup system, as shown in Figure 22.

Figure 22. Kernel hacking: Enable debug support

Enabling Kernel debugging brings up a large set of new options, manyof which add instrumentation support and extra checks to the kernelbehavior. Figure 23 shows some of the debug options available for use tohelp debug system behavior. In particular, enabling debug information to

be generated in the kernel build.



31/48


Figure 23. Kernel debugging options



32/48


Note that if you build the kernel with debug information and ask Simics toload that debug information, the Simics process will get quite large due tothe need to represent the line-number information in Simics.

The kernel is a fairly large piece of software, and just the compiled debuginformation is on the order of 35MB. Loading that into Simics will requirea few hundred MB of host memory to represent, so make sure that youhost has plenty of memory. In particular, running Simics inside a VmWarevirtual machine might make this operation swap like crazy. Therefore, it isno recommended to use a full debug-information kernel unless absolutely

necessary. Note that the default build does have symbol information thattells you which function you are in, which is often sufficient.



33/48


HARDWARE-SOFTWARE INTERFACE DESIGN

There are some recurring patterns in hardware-software interface design,some of which are already present in our simple example device.

Status and Control Registers

You always need one register to set basic controls for a device, and oneregister to read out its status. Sometimes, one register is used for eachaspect, but quite commonly several flags or settings are combined into asingle register. This makes for some fiddling when getting out individual

bits in a driver, but also makes it poss ible to set many settings and getmany status details with a single hardware access. This has advantages forconvenience and consistency of the state. In DML, such register fields isvery simple to program, as shown in Figure 24.

Figure 24. Register fields for control register of simple device in DML

register control @ 0x04 "Device control register" {field OP [1:0] "Operation select" {

// read-write semanticsmethod write(value) {

$this = value;inline $reset();log "info", 2 : "Operation selected: %d", $this;

}}field IE [2] "Interrupt enable" {

method write(value) {$this = value;if(value==1) {

log "info", 2 : "Enabling interrupts";} else {

// Lower our flag about having raised an IRQ// and lower IRQ if it was raisedlog "info", 2 : "Disabling interrupts";inline $clear_interrupt_if_raised();

}}

}}

Writes to the control register will usually have side-effects, while readingshould not so that software can check the device settings without changinganything.

A status register should be readable without side-effects. An olderhardware pattern that used to be quite common was read clears, so thatonce you had read a register all information flags and raised interrupts andsimilar was cleared as a side-effect of just reading it. This is a bit brittle ina complex software stack, and the recommended way to clear states andflag is to use a write. A common idiom is write 1 clears, which means



34/48


that a flag is cleared if a bit value of one is written to it. This makes it easyto clear a range of flags with a single write, and also to leave certain flagsalone by just writing them as zero bits.

PollingThe simplest way to drive most hardware is to use polling, where thesoftware keeps reading some status register bit to tell if the hardware hascompleted its operation. This means that you should always haveoperation complete indicators in your hardware status registers.

The input registers are designed in such a way that there is no need to have

a flag for input accepted. Essentially, the hardware device will notcomplete the write operation from the processor until it has put the data itreceived in the right place internally.

InterruptsIf a device can raise interrupts, it needs a few facilities in place:

A way to tell the device to generate interrupts, usually a singleinterrupt enable bit in a control register. By default, a device shouldnot raise interrupts until told to.

A way to see if an interrupt has been raised, and why. This is one bitor more in a status register.

A way to lower the interrupt once the software has serviced it.Usually, by writing to the status register to clear the interrupt flag.

Some way to forcibly reset the interrupt state of the device, to force aclearing of raised interrupts and all interrupt status flags. This isneeded so that a device driver can force the device into a known-goodstate when initializing it, for example.

The reason you want to have explicit status bits for interrupt status is thatseveral devices could share a single interrupt line to the processor, and thedevice drivers would then need to look at the device status registers and

tell which device or devices raised the interrupt.

In our simple device example, the interrupt status bit is the same as thegeneral operation complete bit.

Starting operationsStarting a hardware operation can either be explicit or implicit.

Our simple device has an implicit design, where writing to the single inputregisters begins an operation, and the point of operation complete is



35/48


pushed out each time. This is a bit simplistic, since it basically assumesthat the hardware has an infinitely long pipeline that can accept anyamount of input.

A more common pattern is an explicit design where the hardwareessentially has a start button. Here, you write input to a large number ofinput registers to setup the desired operation, and then a final write to acontrol register begins the operation. After this, the software goes off towait for an interrupt or polls the operation complete flag.

To get an example of such a design, the easiest is to copy some of the

example code provided with Simics. At a Linux command-line prompt atthe Simics workspace top level, issue the command:

bin/workspace-setup --copy-device=sample-dma-device

This will copy the device called sample-dma-device into base/simics-workspace/modules/sample-dma-device/. In this code, you have severalDMA settings registers that you setup before starting a transfer by writingboth the EN and SWT bits in the control register shown in Figure 25.

Figure 25. Register fields for control of sample-dma-device

register DMA_control size 4 @ 0x00 "DMA control register" {

field EN [31] "Enable DMA";field SWT [30] "Software Transfer Trigger";field ECI [29] "Enable Completion Interrupt";field TS [15:0] "Transfer size (words)";method after_write(memop) {

inline $do_dma_transfer();}

}

This code requires some additional explanation. Note how it does anafter_write() operation rather than the straight write() used previously.The reason for this is that the effect of a write to the control registerdepends on the value of several bits. The DML write() methods for bits in

a register are called in order from low to high bits to generate the newvalue of the register, which means that a write() method for a particularfield cannot assume that the effect of writing other fields has taken placewhen it is being called. The solution to this is to have an after_write()method that is called once the entire register has been updated.

More readingThe number of sources on this topic is fairly limited, but here are a fewlinks:



36/48


Gary Stringham and associates has a newsletter addressing varioushardware-software integration issues, which can be found athttp://www.garystringham.com/newsletter.shtml



37/48


DML PROGRAMMING TIPS

This chapter will address some additional techniques for programming inDML.

Device-internal registers

DML supports the generation of registers that are internal to a device andthat have software-accessible interface. This is useful to model internalstate registers, for example, in a way that is consistent with other registers.The syntax is simple: apply the template unmapped to the register, as

shown in Figure 26. Such registers are checkpointed like other registers,and are visible in the Simics register viewer. It is the best way to codeinternal registers in a device.

Figure 26. Unmapped registers in DML

bank regs {register op_state size 4 is (unmapped);register to_state_1 size 4 @ 0x00 {method write(value) {// do work...$op_state = 1;// ...

}}

method state_dependent_op() {if($op_state == 1) {...

}}

}

Note that if the register represents a configuration parameter to the device,like the IRQ level in our example device, it is most appropriate torepresent it as an attribute.

Large input buffers

If your device interface design needs a larger input buffer that can accept alarge block of data, there are two ways to code this in DML.

The simplest solution is shown in Figure 27. You essentially allocate a setof registers in an array of registers, and have the software write this. Thedevice model can then access the written data using array notation. Notethat this solution does not really create a memory buffer that can beaccessed using accesses of any size, but really just an array of registers. Toread out the contents, you have to work on units the size of the registers.



38/48


Figure 27. An input buffer represented as an array of registers

bank regs {// Give this parameter for an entire bank to allow all registers// to be accessed as bytes and subwords as well as in fullparameter partial=true;...// Put down 200 registers as offset 0x100, 0x104, ..., 0x41c// They have to be read as 32-bit values in the device model// Note the explicit multiplication of the index by four to get// a correct offset.register buf[200] size 4 @ 0x100 + $i*4;...

}...method do_work() {

...local int i;local uint32 s;for(i=0;i


39/48


For example, for ease of computation, you might want to expand a tersebit-based input register into an array of bytes in your device model to beused from several separate functions during a single simulation step. Thisis just an internal optimization, and the true state of the device is stillrepresented by the value of the register. An outline code is shown in Figure28, note that both the write and set methods have to be overridden for thisto work correctly. This example is also checkpoint-safe.

Figure 28. Expanding input data to an internal buffer

data uint8 expanded_buffer[8];

bank e {register input size 1 @ 0x00 {

method write(value) {// Expand data to the buffer$this = value;inline $do_expand();

}method set(value) {

// expand data on attribute set, to support checkpoint$this = value;inline $do_expand();

}// get and read do not need changing, they are just reporting// the byte value in the register

}

method do_expand() {// iterate across bits in $input// set the bytes in expanded_buffer// ... implementation abbreviated ...

}}

Note that if you can, you should try to use a local variable inside a singlemethod or function instead, as shown above in the second part of Figure27. This avoids any checkpointing issues and makes it clear that data istemporary.

Including C code in a DML device

DML does support putting in arbitrary C code in a device. This isdocumented in the Simics documentation, but the short version is that youhave optional header and footer sections in the DML source file that insertsarbitrary C code before and after the code generated from the DML code.Typically, header is used to include the C header files needed for the codein the C code to work, and footer contains the actual implementation code.The best way to communicate data from the DML code to the C code is touse arguments to the C functions, not to try to peek into the device datastructure generated by DML (even if that can be done and is well-definedin how it works).



40/48


Modeling configuration pinsSometimes a chip or a function block inside a chip has a set of pins that arewired to 1 or 0 (pull-up or pull-down) by the surrounding system. Thiscould be, for example, jumper switches on a board, or simply constantpull-up/down connections inside a chip. In Simics, such pins are bestmodeled as attributes, so that they can be set in the configuration (andchanged from the command-line).

The code in Figure 29 implements three such configuration pins, using acouple of DML templates. Note the use ofconfiguration=pseudo to make theregister not be checkpointed (as its value is derived from the existingattributes which are checkpointed), and allocate=false to remove the needfor storing any data for the register in the device data structure, as that isalready stored in the attributes. This code works with the device viewerand the software view.



41/48


Figure 29. Modeling configuration pins

// Template for the attributestemplate pin_attribute {

// "b" means bool and restricts setting the attribute to true/falseparameter type = "b";// The data is stored in integer form due to a pecularity// of the Simics API that returns integers valued 0 or 1// for boolean attributes.data int value;method get() -> (result) {

result = SIM_make_attr_boolean($value);}method set(attr_value) {

$value = SIM_attr_boolean(attr_value);}

}

// The attributes get the good easy to understand names,// as they are accessed in the configuration.// dev0->pin_1 etc. is the way to access themattribute pin_1 is (pin_attribute);attribute pin_2 is (pin_attribute);attribute pin_3 is (pin_attribute);

template wired_pin_field {// Set pin_attribute to tell us which attribute to readparameter pin_attribute default undefined;// Make this not stored in the device data structureparameter allocate = false;method write(value) {

log "spec_violation" : "Writing hard-wired bit %s", $qname;}method read() -> (result) {

result = $pin_attribute.value;}

// read current value for checkpointing and device viewermethod get -> (v) {

inline $read -> (v);}method set(v) {

// Should not get called, but ignore values written}

}

bank wired_config {register values size 4 @ 0x00 {

// To not checkpoint this value, as it is redundantparameter configuration = "pseudo";// Then the fieldsfield p1 [0] is (wired_pin_field) {

parameter pin_attribute = $pin_1;}field p2 [1] is (wired_pin_field) {

parameter pin_attribute = $pin_2;}field p3 [2] is (wired_pin_field) {

parameter pin_attribute = $pin_3;}

}}

More device example in SimicsNote that there are several more example devices available in the Simicsinstallation, just do bin/workspace-setup --copy-device=badname to get a list



42/48


43/48


cat /proc/modules: lists all dynamically loaded modules, which shouldonly be our simple device driver, as all other drivers are staticallylinked to the kernel.

dmesg: print the contents of the the printk kernel message bufferDevice LoggingIt is a very good idea to pepper your device code with log statements asyou initially figure out how to do things. They can be removed later, or leftin at a high level of logging (they do not hurt performance if notactivated). SeeModeling Your System in Simics, Section 3.7.

The standard log levels used in Virtutech devices are the following:

1. Important: Error messages and very important messages, for exampleEthernet controller receive functionality not implemented

2. Informative: High-level information like mode changes and events, forexample Ethernet port status changed to link-up

3. Debugging: Medium level of information. More detailed than level 2.For example, Ethernet controller saved received frame at0x1000BEA0

4. Detailed debugging: Detailed messages about what is happening in thedevice, for example Ethernet controller received an ARP frame with

correct CRC and moved it queue 0x01 to be handled in 100 cycles

Device Attributes and StateAll registers and attributes in a DML source file automatically have Simicsattributes generated for them. This provides back-door access to the stateof a device, for both reading and changing it. To read an attribute of aSimics object, you can either:

Use the command sd0->ATTRIBUTENAME on the Simics command-line.Note that all registers in the bankregs have attributes generated withnames starting with regs_. There is no hierarchical namespace. See

the bottom ofFigure 31 for an example attribute access. Define sd0.info and status commands, as discussed above. Use the Simics GUI Device Registers window shown in Figure 17.

However, that only shows the registers, not other attributes. Use the Simics GUI Object Browser to look at all the attributes of the

device, as shown in Figure 30.



44/48


Figure 30. Simics Object Browser

Writing a Device Memory Map from the Simics CLIYou can read and write the memory-mapped registers of a device directlyfrom the Simics command-line, to see that memory writes have theintended effect (and often that appropriate log messages are seen). To dothis, you use the command write or read on a memory space in Simics. Inour setup, the relevant memory spaces are ccsr_space, where the device ismapped at 0xf0000 (in the default configuration) and phys_mem, where thedevice is mapped at 0xf80f0000, as seen from the processor.

Figure 31 shows an example session using read and write. Note that the

reads and writes seem to come from cpu0 for the device, and by designthere is no way to differentiate them from regular processor reads andwrites. If you want to see or change values without looking like aprocessor, use the attribute mechanism (or the device viewer shown inFigure 17).

The write command works like this: MEMORYSPACE.write ADDRESS VALUE[SIZE], where SIZE is optional and defaults to 4 bytes. Note that the ADDRESSis local in the memory space. MEMORYSPACE.read ADDRESS [SIZE] reads a valuefrom memory, with the same argument types as write. You can also forcethe endianness of the access to be little-endian or big-endian, but that is not



45/48


an issue in the completely big-endian Power Architecture system that weare working with here.

Note that write and read have side-effects, and that is what you want fordevice testing. There are also MEMORYSPACE.get and set commands thatgenerate inquiry accesses which do not have side-effects. These are notthat useful for device model testing.

Figure 31. Device read and write from the command-line

simics> output-radix 16 2simics> ccsr_space.read 0xf0000

[sd0 trace-io] Read from cpu0: 0x0f_00_00 4 0xab_cd_00_01 (BE)0xab_cd_00_01simics> ccsr_space.write 0xf0000 0xdeadbeef[sd0 spec-viol] Write to read-only field regs.version.V_MINOR (value written = 0xef, contents = 0x1).[sd0 spec-viol] Write to read-only field regs.version.V_MAJOR (value written = 0xbe, contents = 0x0).[sd0 spec-viol] Write to read-only field regs.version.COOKIE (value written = 0xdead, contents = 0xabcd).[sd0 trace-io] Write from cpu0: 0x0f_00_00 4 0xde_ad_be_ef (BE)simics> phys_mem.write 0xf80f_0000 0xdeadbeef[sd0 spec-viol] Write to read-only field regs.version.V_MINOR (value written = 0xef, contents = 0x1).[sd0 spec-viol] Write to read-only field regs.version.V_MAJOR (value written = 0xbe, contents = 0x0).[sd0 spec-viol] Write to read-only field regs.version.COOKIE (value written = 0xdead, contents = 0xabcd).[sd0 trace-io] Write from cpu0: 0x0f_00_00 4 0xde_ad_be_ef (BE)simics> simics> sd0->regs_version0xab_cd_00_01

If you want

Driver LoggingUsing printk() in your driver code offers a way to have it tell you what it isdoing. Note that it is not good to put this on the common calls like write()as the target console will mix the printk() and application-level printoutson the same line. When the driver detects errors, however, printk() candefinitely be a very useful way to signal driver state.

Use the kernel various defined KERN_x printk levels. To remove debug codewhen not used, use local DEBUG_SOMETHING defines in the device driver. Thereis currently a few of these defined in simple.h and used in the various .cfiles.

Linux Kernel Debug FeaturesThe linux kernel used can be rebuilt to include more debug features andrun-time checks. Currently, the default setup is intended for run-time use,and does not enable these hooks. See Enabling Debug Information in theKernel above for more information.



46/48


Application Program LoggingOutput in the application program is natural to do to see the flow ofexecution, but it should not be part of tight inner loops that you want tobenchmark or accelerate. The core problem here is that doing serial outputis very expensive in terms of execution time, and will likely skew orcompletely swamp performance results.

Simics DebuggerSimics has a built-in debugger that can handle both kernel-level and user-level code. We will get back with a description on how to use these.

PERFORMANCE MEASUREMENT

More to come.



47/48


SUMMARY

This application note has presented a complete teaching setup based onVirtutech Simics, the Freescale MPC8641D virtual platform, and theLinux operating system. This setup can be used to teach a variety of topicsrelated to how hardware and software interact. It provides students andteachers a complete software stack for a realistic target, with the flexibilityto change any part, including the hardware.

CONTACT INFORMATION

Northand South America

[email protected]

Europe, Middle-East, Africa

[email protected]

Asia-Pacific

[email protected]

Japan

[email protected]

http://www.virtutech.com

Copyright 2008 Virtutech, Inc. All Rights Reserved.

TRADEMARKS. Virtutech, Simics, Hindsight, and the logos thereof, are trademarks or registered trademarks of Virtutech, Inc.

and/or its subsidiaries, in the United States and/or other countries.

mailto:[email protected]:[email protected]:[email protected]:[email protected]://www.virtutech.com/http://www.virtutech.com/mailto:[email protected]:[email protected]


48/48


THIS PUBLICATION IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A

PARTICULAR PURPOSE, OR NON-INFRINGEMENT.

THIS PUBLICATION COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES

ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN

NEW EDITIONS OF THE PUBLICATION. VIRTUTECH MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE

PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS PUBLICATION AT ANY TIME.

Virtutech, Inc., 2001 Gateway Place, Suite 201E, San Jose, CA 95110, USA

Documents

appnote-teaching hardware-software integration with Simics 2008-9-something v0.9.5