Detecting PCI devices

On identifying the peripheral equipment installed in our PC

Early PCs

• Peripheral devices in the early PCs used fixed i/o-ports and fixed memory-addresses, e.g.:– Video memory address-range: 0xA0000-0xBFFFF – Programmable timer i/o-ports: 0x40-0x43– Keyboard and mouse i/o-ports: 0x60-0x64– Real-Time Clock’s i/o-ports: 0x70-0x71– Hard Disk controller’s i/o-ports: 0x01F0-01F7– Graphics controller’s i/o-ports: 0x03C0-0x3CF– Serial-port controller’s i/o-ports: 0x03F8-0x03FF– Parallel-port controller’s i/o-ports: 0x0378-0x037A

The PC’s evolution

• It became clear in the 1990s that there would be contention among equipment vendors for ‘fixed’ resource-addresses, which of course were in limited supply

• Among the goals that motivated the PCI Specification was the creation of a more flexible scheme for allocating addresses that future peripheral devices could use

PCI Configuration Space

PCI Configuration Space Body(48 doublewords – variable format)

64doublewords

PCI Configuration Space Header(16 doublewords – fixed format)

A non-volatile parameter-storage area for each PCI device-function

PCI Configuration Header

StatusRegister

CommandRegister

DeviceID

VendorID

BISTCacheLineSize

Class CodeClass/SubClass/ProgIF

RevisionID

Base Address 0

SubsystemDevice ID

SubsystemVendor ID

CardBus CIS Pointer

reservedcapabilities

pointer Expansion ROM Base Address

MinimumGrant

InterruptPin

reserved

LatencyTimer

HeaderType

Base Address 1

Base Address 2Base Address 3

Base Address 4Base Address 5

InterruptLine

MaximumLatency

31 0 31 0

16 doublewords

Dwords

1 - 0

3 - 2

5 - 4

7 - 6

9 - 8

11 - 10

13 - 12

15 - 14

Three IA-32 address-spaces

memoryspace(4GB)

i/o space(64KB)

PCIconfiguration

space(16MB)

accessed using a large variety of processor instructions (mov, add, or, shr, push, etc.) and virtual-to-physical address-translation

accessed only by using the processor’s special ‘in’ and ‘out’ instructions (without any translation of port-addresses)

i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space

reserved

Interface to PCI Configuration Space

CONFADD( 0x0CF8)

CONFDAT( 0x0CFC)

31 23 16 15 11 10 8 7 2 0

EN

bus(8-bits)

device(5-bits)

doubleword (6-bits)

function(3-bits) 00

PCI Configuration Space Address Port (32-bits)

PCI Configuration Space Data Port (32-bits)

31 0

Enable Configuration Space Mapping (1=yes, 0=no)

Reading PCI Configuration Data

• Step one: Output the desired longword’s address (bus, device, function, and dword) with bit 31 set to 1 (to enable access) to the Configuration-Space Address-Port

• Step two: Read the designated data from the Configuration-Space Data-Port:# read the PCI Header-Type field (byte 2 of dword 3) for bus=0, device=0, function=0

movl $0x8000000C, %eax # setup address in EAXmovw $0x0CF8, %dx # setup port-number in DX outl %eax, %dx # output address to port

mov $0x0CFC, %dx # setup port-number in DXinl %dx, %eax # input configuration longwordshr $16, %eax # shift word 2 into AL registermovb %al, header_type # store Header Type in variable

Demo Program

• We created a short Linux utility that searches for and reports all of your system’s PCI devices

• It’s named “pciprobe.cpp” on our CS635 website• It uses some C++ macros that expand to Intel

input/output instructions -- which normally are ‘privileged’ instructions that a Linux application-program is not allowed to execute (segfault!)

• Our system administrator (Alex Fedosov) has created a utility (named “iopl3”) that will allow your command-shell to acquire I/O privileges

Example: network interface

• We identify the network interface controller in our classroom PC’s by class-code 0x02

• The subclass-code 0x00 is for ‘ethernet’• We can identify the NIC from its VENDOR and

DEVICE identification-numbers:• VENDOR_ID = 0x14E4• DEVICE_ID = 0x1677

• You can use the ‘grep’ command to search for these numbers in this header-file:

</usr/src/linux/include/linux/pci_ids.h>

Vendor’s identity

• The VENDOR-ID 0x14E4 belongs to the Broadcom Corporation (headquarters in Irvine, California)

• Information about this firm may be learned from the corporation’s website:

<http://www.broadcom.com>

• The DEVICE-ID 0x1677 is used to signify Broadcom’s BCM5751 ethernet product

nic

Typical NIC

TX FIFO

RX FIFO

transceiver LANcableB

US

main memory

packet

buffer

CPU

Packet filtering capability

• Network Interface’s hardware needs to implement ‘filtering’ of network packets

• Otherwise the PC’s memory-usage and processor-time will be wasted handling packets not meant for this PC to receive network packet’s layout

Destination-address (6-bytes) Source-address (6-bytes)

Each data-packet begins with the 6-byte device-address of the network interface which is intended to receive it

Your NIC’s unique address

• You can see the Hardware Address of the ethernet controller on your PC by typing:

$ /sbin/ifconfig

• Look for it in the first line of screen-output that is labeled ‘eth0’, for example:

• (The NIC’s filter-register stores this value)

eth0 Link encap: Ethernet HWaddr 00:11:43:C9:50:3A

Our ‘tigon3.c demo

• We wrote a kernel module that lets users see certain register-values which pertain to the BCM5751 network interface in your classroom workstation:– (1) the PCI Configuration Space registers– (2) the Media Access Controller’s address

• It also shows your machine’s node-name (in case you want to save the information)

How we got the MAC-address

• We do not have Broadcom’s programming datasheet -- but we do have Linux source code for the ‘tigon3’ device-driver, which includes a header-file ‘tg3.h’ found here:

</usr/src/linux/drivers/net/>

• If you scroll through the #define directives you will see the offset where the hardware address is stored in the memory-mapped register-space of the ‘tigon3’ interface

Driver’s authors

• The Linux kernel’s open-source driver for the Broadcom ‘tigon3’ network controller was jointly written by David S. Miller (see photo below) and Jeff Garzik

David Miller’s announcement in Feb 2002of their driver’s BETA version is online. It includes his candid comments aboutthe challenge of writing such a driver whenthe vendor does not make available itsdevice’s programming documentation.

How we got tigon3 registers

StatusRegister

CommandRegister

DeviceID0x1677

VendorID0x14E4

BISTCacheLineSize

Class CodeClass/SubClass/ProgIF

RevisionID

Base Address 0

SubsystemDevice ID

SubsystemVendor ID

CardBus CIS Pointer

reservedcapabilities

pointer Expansion ROM Base Address

MinimumGrant

InterruptPin

reserved

LatencyTimer

HeaderType

Base Address 1

Base Address 2Base Address 3

Base Address 4Base Address 5

InterruptLine

MaximumLatency

31 0 31 0

16 doublewords

Dwords

1 - 0

3 - 2

5 - 4

7 - 6

9 - 8

11 - 10

13 - 12

15 - 14

Linux helper-functions#include <linux/pci.h>

struct pci_dev *devp;unsigned int iomem_base, iomem_size;void *io;

devp = pci_get_device( 0x14E4, 0x1677, NULL );if ( !devp ) return –ENODEV;

iomem_base = pci_resource_start( devp, 0 );iomem_size = pci_resource_len( devp, 0 );

io = ioremap( iomem_base, iomem_size );if ( !io ) return -EBUSY;

Big-Endian to Little-Endian

mac1

mac0

mac5

mac4

mac3

mac2

0x0410 0x0411 0x0412 0x0413 0x0414 0x0415 0x0416 0x0417

Broadcom network interface storage-addresses

Intel IA-32 character-array storage

mac0

mac1

mac2

mac3

mac4

mac5

In-class exercise

• Copy the ‘tigon3.c’ source-module to your own directory, then rename it ‘anchor.c’

• Your assignment is to modify it so that it will show information about the Intel NICs in our ‘anchor’ cluster’s machines:

#define VENDOR_ID 0x8086 // Intel Corp#define DEVICE_ID 0x109A // 82573L NIC

• Intel’s filter-register at offset 0x5400 uses the ‘little endian’ storage-convention

Little-Endian to Little-Endian

mac0

mac1

mac2

mac3

mac4

mac5

0x5400 0x5401 0x5402 0x5403 0x5404 0x5405 0x5406 0x5407

Intel network interface storage-addresses

Intel IA-32 character-array storage

mac0

mac1

mac2

mac3

mac4

mac5