View
328
Download
5
Category
Tags:
Preview:
DESCRIPTION
In this talk I gave at EHSM 2014 event ( http://ehsm.eu ) I am explaining what a MMU is and how it works. I then explain how I ported NetBSD (and EdgeBSD which is a fork of NetBSD) on this open source LM32 CPU in which I added an MMU.
Citation preview
Porting NetBSDon
the open source LatticeMico32 CPU
Yann SionneauM-Labs
@ EHSM 2014
About me
• Yann Sionneau• Embedded software developer• Working at Sequans Communication• M-Labs contributor• @yannsionneau on twitter• Email: yann.sionneau@gmail.com
I’m going to talk about…
How to run NetBSD and EdgeBSD on theMilkymist One
Agenda
• I) The hardware part: the MMU–What is a MMU and how it works
• II) The software part–How to port NetBSD to a new CPU
Milkymist One?!
Milkymist One?!
Milkymist One?!
The Milkymist One uses an FPGA
What’s an FPGA??
• A chip
FPGA internals
Milkymist System-on-Chip
LatticeMico32 CPU
• 32 bits Harvard Architecture RISC• Big Endian• 6 stages• Fully bypassed• Optional configurable I/D caches– Direct mapped or– 2-way set associative
• Wishbone on-chip bus
LatticeMico32 , Good points
• Small• Portable (works with several FPGA vendors)• Fast (~100 MHz on Slowtanpartan 6)• Actually works• GCC/Binutils/GDB/Qemu/uCLinux/OpenWRT
support• OPEN SOURCE
LatticeMico32, Bad points
• No Memory Management Unit… yet!
LatticeMico32, Bad points
• No Memory Management Unit… yet! Done
Used in…
• Closed source commercial ASICs• Open source projects
• Can achieve 800 MHz in TSMC 90nm standard cell process
LatticeMico32 pipeline
What’s a pipeline?
• « In computing, a pipeline is a set of data processing elements connected in series, where the output of one element is the input of the next one. »
-- Pipeline (computing), Wikipedia
What’s a pipeline?
Data processing element 1
Data processing element 2
Data processing element 3
ININ INOUTOUT
OUT
What’s a pipeline?
$ cat .bash_history | grep 'cat' | wc -l 6
What’s a CPU pipeline?
Address
instruction Fetch
instruction Decode
instruction eXecute
Memory load/store
register Write Back
What’s a CPU pipeline?
A
F
D
X
M
W
Pipelined instruction executionInstr.
numberPipeline Stage
1 A
2
3
4
Clock cycle 1 2 3 4 5 6 7
Pipelined instruction executionInstr.
numberPipeline Stage
1 A F
2 A
3
4
Clock cycle 1 2 3 4 5 6 7
Pipelined instruction executionInstr.
numberPipeline Stage
1 A F D
2 A F
3 A
4
Clock cycle 1 2 3 4 5 6 7
Pipelined instruction executionInstr.
numberPipeline Stage
1 A F D X
2 A F D
3 A F
4 A
Clock cycle 1 2 3 4 5 6 7
Pipelined instruction executionInstr.
numberPipeline Stage
1 A F D X M
2 A F D X
3 A F D
4 A F
Clock cycle 1 2 3 4 5 6 7
Pipelined instruction executionInstr.
numberPipeline Stage
1 A F D X M W
2 A F D X M
3 A F D X
4 A F D
Clock cycle 1 2 3 4 5 6 7
Pipelined instruction executionInstr.
numberPipeline Stage
1 A F D X M W
2 A F D X M W
3 A F D X M
4 A F D X
Clock cycle 1 2 3 4 5 6 7
Address
instruction Fetch
instruction Decode
instruction eXecute
Memory load/store
register Write back
InstructionCache
DataCache
Main Memory
CPU Internal
Before
PHYSICAL
ADDRESS
PHYSICAL
ADDRESS
PA
PA
Address
instruction Fetch
instruction Decode
instruction eXecute
Memory load/store
register Write back
InstructionCache
DataCache
Main Memory
CPU Internal
Memory Management Unit (MMU)
Raising exception
MM
U lo
okup
Cache lookup
After
VIRTUAL ADDRESSES PHYSICAL ADDRESSES
What’s the MMU’s job?
• Translate « virtual addresses » into « physical addresses »
• Memory protection against unwanted execution of code or data write (e.g. software bug or security issue)– Memory right access management
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
How does the MMU know the VA->PA translation ?
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
Page Table
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
Page TableWhy « PAGE »?
Why « Page »?
• 0x00000004 -> 0x10000000• 0x00000005 -> 0x10000001• 0x00000006 -> 0x10000002Etc…
Why « Page »?
• 0x00000004 -> 0x10000000• 0x00000005 -> 0x10000001• 0x00000006 -> 0x10000002Etc…
This is WRONG!!!
Why « Page »?
• 0x00000*** -> 0x10000***• 0x00001*** -> 0x10001***• 0x00002*** -> 0x10002***Etc…
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
Page Table
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
Page Table
TLBTLB : Translation Lookaside Buffer
Main Memory
Memory Management Unit (MMU)
CPU pipelineVA PA
VA : Virtual AddressPA : Physical Address
Page Table
TLBOperating System
Updates the
Gets information from the
Updates the
Features?
• Page size–Only 4 kB
32 bits physical address : xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx
How many bits of an address indicate the offset within a given page?
Features?
• Page size–Only 4 kB
32 bits physical address : xxxxxxxx xxxxxxxx xxxx xxxx xxxxxxxx
Page number [31:12]
20 bits
Offset [11:0]
12 bits
Features?
• 2 TLB (Translation Lookaside Buffer)– ITLB–DTLB
• Each TLB contains 1024 entries–How many bits needed to index the TLB?
Features?
• 2 TLB (Translation Lookaside Buffer)– ITLB–DTLB
• Each TLB contains 1024 entries–How many bits needed to index the TLB?
10 bits!
Features?
• No hardware page-tree walker– i.e. TLB is software assisted
Memory Management Unit
(MMU)
Virtual address
Load or store?
Instruction or Data?
Physical address
Accessgranted/denied
Memory Management Unit
(MMU)
Virtual address
Load or store?
Instruction or Data?
Physical address
Accessgranted/denied
I don’t know!
Let’s have a look inside
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001004
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004
Page number
Offset in the page
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001
VPN = 0xA0001 1010 0000 0000 0000 0001
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 1010 0000 00 00 0000 0001
TLB index, used to select a TLB line
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 1010 0000 00 00 0000 0001
TLB index, used to select a TLB line
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 1010 0000 00 00 0000 0001
Tag = 0x280 1010 0000 00=
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 1010 0000 00 00 0000 0001
Tag = 0x280 1010 0000 00=
Physical page number = 0xB0001
Tag [10] Physical page number [20] Read-only [1] Valid [1]
0xABC 0xABC00 0 0
0x280 0xB0001 1 1
0x300 0x00001 0 1
The TLBVA = 0xA0001 004Page offset = 4Virtual Page number = 0xA0001 TLB index = 1
VPN = 0xA0001 1010 0000 00 00 0000 0001
Tag = 0x280 1010 0000 00=
Physical page number = 0xB0001Physical Address = 0xB0001004
Porting NetBSD
• 1°) NetBSD cross compilation toolchain– build.sh– Makefiles here and there– Arch-specific directories
Allows to do:$ ./build.sh -U -m lm32
tools
Porting NetBSD
• 2°) Support for built-ins in libkern– NetBSD kernel is• Not linked against libgcc• Linked against libkern
– Need to implement basic arithmetic functions emitted by gcc in object code
– Implementation in sys/lib/libkern/arch/lm32
Porting NetBSD
• 3°) Building my first kernel– Create sys/arch/lm32 and sys/arch/milkymist– Populate• sys/arch/<cpu|soc>/include• sys/arch/<cpu|soc>/conf
– Stub, stub, stub…
Allows to do:$ ./build.sh -m milkymist -U kernel=GENERIC
Porting NetBSD
• 4°) Write basic console driver for early prints
struct consdev milkymist_com_cons = { […] milkymist_com_cngetc, /* cn_getc: kernel getchar interface */ milkymist_com_cnputc, /* cn_putc: kernel putchar interface */ […]};
Porting NetBSD
• 5°) Implement exception handlers• 6°) Call milkymist_startup() C code– Initialize console driver • -> consinit() -> milkymist_uart_cnattach()• cn_tab = &milkymist_com_cons;
– Initialiaze virtual memory subsystem• Call MD pmap_bootstrap()
– Let the kernel boot• Call NetBSD MI main()
Porting NetBSD
• 7°) Implement pmap.9pmap -- machine-dependent portion of the virtual memory system– pmap_bootstrap()– pmap_init, pmap_create, pmap_destroy …– SW managed TLB? -> sys/uvm/pmap/
– used in (PowerPC Booke and LM32)
Porting NetBSD
• 8°) Implement copyin/copyout• 9°) Implement atomic operations– No atomic instruction RAS (Restartable Atomic
Sequence) CAS (Compare And Swap)– Other atomic ops built around this CAS
RAS CASint _atomic_cas_32(volatile uint32_t *val, uint32_t old, uint32_t new);_atomic_cas_32:_atomic_cas_ras_start:
lw r4, (r1+0) /* load *val into r4 */bne r4, r2, 1f /* compare r4 (*val) and old (r2) */ sw (r1+0), r3
_atomic_cas_ras_end: 1:
mv r1, r4 /* return (*val) */ret
Porting NetBSD
• 10°) Add support for interrupts– Write a function to register interrupt handlers
• 11°) Have a running system clock– Write cpu_initclocks()– Write clock irq handler• Call hardclock()
Other functions to write
• Switch context from one thread to another– cpu_switchto(9)
• Copy data and abort on page fault– kcopy(9)
• Save current context– setfault()
• Low level code to finish up fork() operation– cpu_lwp_fork(9)
Other functions to write
• Block interrupts to protect critical sections– spl(9)
• Init CPU and print copyright message– cpu_startup(9)
• Determine the root file system device– cpu_rootconf(9)
• Etc…
Porting NetBSD
• To boot user space– Create dummy ramdisk with /sbin/init– Build kernel with MFS– Insert ramdisk with mdsetimage– Boot it!
Porting NetBSD
DEMO
Thank you!
Sébastien Bourdeauducq, Michael Walle, Robert Swindells, Stefan Kristiansson, Lars-Peter Clausen, Pierre Pronchery, Radoslaw Kujawa, Youri Mouton, Matt Thomas, tech-kern@, M-Labs mailing list, and many more
Questions?
NetBSD/milkymist Memory Layout
Kernel spaceUser space
0 0xffffffff
0xc0000000
0xc8000000Ram window
User stack
Kernelstack
DDR SDRAM : 128 MB
Recommended