New ECE 4750 Computer Architecture Course Overview · 2017. 8. 14. · What is Computer Architecture? Activity 1 • Trends in Computer Architecture • Activity 2 Computer Architecture

ECE 4750 Computer ArchitectureCourse OverviewChristopher Batten

School of Electrical and Computer EngineeringCornell University

http://www.csl.cornell.edu/courses/ece4750

• What is Computer Architecture? • Activity 1 Trends in Computer Architecture Activity 2 Computer Architecture Design

The Computer Systems Stack

Technology

Application

Gap too large to bridge in one step(but there are exceptions, e.g., a magnetic compass)

In its broadest definition, computer architecture is thedevelopment of the abstraction/implementation layers that allow us to

execute information processing applications efficientlyusing available manufacturing technologies

ECE 4750 Course Overview 2 / 37



Register-Transfer Level

CircuitsDevices

Instruction Set Architecture

Programming LanguageAlgorithm

Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

Sort an array of numbers2,6,3,8,4,5 -> 2,3,4,5,6,8

1. Find minimum number in array2. Move minimum number into output array3. Repeat steps 1 and 2 until finished

Out-of-place selection sort algorithm

void sort( int b[], int a[], int n ) { for ( int idx, k = 0; k < n; k++ ) { int min = 100; for ( int i = 0; i < n; i++ ) { if ( a[i] < min ) { min = a[i]; idx = i; } } b[k] = min; a[idx] = 100; }}

C implementation of selection sort





CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

Mac OS X, Windows, LinuxHandles low-level hardware management

blez $a2, donemove $a7, $zeroli $t4, 99move $a4, $a1move $v1, $zeroli $a3, 99lw $a5, 0($a4)addiu $a4, $a4, 4slt $a6, $a5, $a3movn $v0, $v1, $a6addiu $v1, $v1, 1movn $a3, $a5, $a6

MIPS32 Instruction SetInstructions that machine executes





CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

Silicon processtechnology

Si Si Si

SiSiSi

Transistors and wires

Combining devicesto do useful work

Boolean logic gatesand functions

How data flowsthrough system



Application Requirements vs. Technology Constraints

In its broadest definition, computer architecture is thedevelopment of the abstraction/implementation layers that allow us to

execute information processing applications efficientlyusing available manufacturing technologies


CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

Application Requirements • Suggest how to improve architecture • Provide revenue to fund development

Technology Constraints • Restrict what can be done efficiently • New technologies make new arch possible

Computer engineers provide feedback to guideapplication and technology research directions



Computer Architecture in the ECE/CS Curriculum


CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level ECE 2300 Digital Logic & Computer Org

ECE 3140 Embedded Systems

ECE 4750 Computer Architecture

ECE 4740 Digital VLSI Design

CS 4420 Compilers

Related Graduate Courses • ECE 5760 Advanced Microcontroller Design • ECE 5750 Advanced Computer Architecture • ECE 5730 Memory Systems • ECE 5770 Resilient Computer Systems • ECE 5745 Complex Digital ASIC Design • ECE 5775 High-Level Design Automation

CS 4410 Operating Systems



Logic, State, and Interconnect


CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

Interconnect

StateLogic StateLogic Logic

StateLogic StateLogicState

Digital systems are implemented with three basic building blocks • Logic to process data • State to store data • Interconnect to move data



Processors, Memories, and Networks


CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

Computer engineering basic building blocks • Processors for computation • Memories for storage • Networks for communication

Network

Processor

Memory

InputData

OutputData

Computedata

Movedata

Storedata



Computer Architecture Artifacts


What is Computer Architecture? • Activity 1 • Trends in Computer Architecture Activity 2 Computer Architecture Design

RTL

Devices

ISA

PL

Algorithm

�Arch

Technology

Application

OS

Gates

Circuits

Agenda

What is Computer Architecture?

Activity 1

Trends in Computer Architecture

Activity 2

Computer Architecture Design


What is Computer Architecture? • Activity 1 • Trends in Computer Architecture Activity 2 Computer Architecture Design

Activity #1: Sorting with a Sequential Processor

Network

Processor

Memory

I Application: Sort 32 numbers

I Simulated Sequential Computing System. Processor: You!. Memory: Worksheet, read input data, write output data. Network: Passing/collecting the worksheets

I Activity Steps. 1. Discuss strategy with neighbors. 2. When instructor starts timer, flip over worksheet. 3. Sort 32 numbers as fast as possible. 4. Lookup when completed and write time on worksheet. 5. Raise hand. 6. When everyone is finished, then analyze data


What is Computer Architecture? Activity 1 • Trends in Computer Architecture • Activity 2 Computer Architecture Design

RTL

Devices

ISA

PL

Algorithm

�Arch

Technology

Application

OS

Gates

Circuits

Agenda


Activity 1


Activity 2




Application Requirements vs. Technology Constraints


CircuitsDevices



Microarchitecture

Com

pute

r Arc

hite

ctur

e

Technology

Application

Operating System

Gate Level

TraditionalApplication Requirements • As much processor compute as possible • As much memory capacity as possible • As much network bandwidth as possible

TraditionalTechnology Constraints • Exponential scaling of resources



Exponential Scaling for Processor Computation

Transistors(Thousands)

MIPSR2K

IntelP4

1975 1980 1985 1990 1995 2000 2005 2010 2015

100

101

102

103

104

105

106

107

DECAlpha 21264

Frequency(MHz)

SPECintPerformance~50%/ye

ar

~38%/year~50%/year

Data collected by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, C. Batten



Exponential Scaling for Memory Capacity

1970 1980 1990 1995 20001975 1985 2005

1K1K

16G

1G

64M

4M

256K

16KMem

ory

Cap

acity

per

Chi

p

DRAM

SRAM

FLASH

First presentation at ISSCC or Symp. VLSI Circuits

Adapted from K. Itoh et al. “Ultra-Low Voltage Nano-Scale Memories.” Spring 2007.



Exponential Scaling for Network BandwidthN

etw

ork

Peak

Bis

ectio

n Ba

ndw

idth

(MB/

s)

1985 1990 1995 2000 2005

103

104

105

106

107

102

Bus-BasedInterconnection

Networks(10's Nodes)

Multi-StageInterconnection

Networks(100-1000's Nodes)

Data from Hennessy & Patterson, Morgan Kaufmann, 2nd & 5th eds., 1996 & 2011; D.E. Culler et al., Morgan Kaufmann, 1999.



Key trends in application requirements andtechnology constraints over the past decade have

resulted in a radical rethinking of theprocessors, memories, and networksused in modern computing systems

Five Key Trends in Computer Architecture

1. Growing diversity in application requirements motivate growingdiversity in computing systems pushing towards the cloud and IoT

2. Energy & power constrain systems across the computing spectrum

3. Transition to multiple cores integrated onto a single chip

4. Transition to heterogeneous systems-on-chip

5. Technology scaling challenges motivate new emerging compute,storage, and communication device technologies



Trend 1: Bell’s Law

Roughly every decade a new, smaller, lower priced computer class formsbased on a new programming platform resulting in entire new industries

103

104

105

106

107

108

102

Pric

e in

Dol

lars

1950 1960 1970 1980 2000 2010

G. Bell. "Bell's Law for the Birth and Death of Computer Classes."CACM, Jan 2008.

1990

Volu

me

in c

m3

2020 1950 1960 1970 1980 2000 20101990 2020

Y. Lee et al. "Modular 1mm3 Die-Stacked Sensing Platform ..."JSSC, Jan 2013.

104

106

108

102

100

107

105

103

101

Mainframes

Mainframes

Minicomputers

Minicomputers

Workstations

WorkstationsPersonalComputers

PersonalComputers

Laptops

LaptopsHandhelds

Handhelds

Supercomputers

ScalableClusters

CloudComputing

Internet of ThingsIoT



Trend 1: Growing Diversity in Apps & Systems

Data Centers

MedicalDevices

Internet Routers

Humanoid RobotsUnmanned Vehicles

GPS Devicesand Satellites

Automobiles

DigitalCameras

Wearable Activity MonitorsSmart Home

WearableComputing

Sensor Networks

GameConsoles Computing: From Handhelds to Servers



Trend 2: Energy and Power Constraints

Power = EnergySecond

EnergyOp= Ops

Second

Battery LifeElectricity BillMobile Device Weight

Chip PackagingChip CoolingSystem NoiseCase TemperatureData-Center Air Conditioning

Power Energy

Performance (Ops/Second)

Ener

gy p

er O

pera

tion 100W Workstation

Power Constraint

1W Handheld Power Constraint



Trend 2: Energy and Performance of Single Processor

Performance (Millions of Instructions per Second)

En

erg

y (P

ico-J

oule

pe

r In

stru

ctio

n)

200 400 600 800 1000 1200 1400 1600 1800 2000

50

100

150

200

250

500

Incr

easing P

ower

in-order, 1-issue

Based on analytical models of90nm technology with joint optimization ofmicroarchitectural and circuit parameters

in-order, 2-issuein-order, 3-issueout-of-order, 1-issue

out-of-order, 3-issueout-of-order, 2-issue

IO/3

OOO/2

OOO/3

IO/1

Adpated from O. Azizi et al. “Energy-Performance Tradeoffs ...” ISCA, 2010.



Trend 2: Power Constrains Single-Processor Scaling


MIPSR2K

IntelP4

1975 1980 1985 1990 1995 2000 2005 2010 2015

100

101

102

103

104

105

106

107

DECAlpha 21264

TypicalPower (W)

Frequency(MHz)

SPECintPerformance

~9%/year

~15%/year



Trend 3: Transition to Multicore Processors

P

M

N

P

M

N

P

M

P

M

Intel Pentium 4Single monolithic processor

Cray XT3 Supercomputer1024 single-core processors

P

M

P

M

P

M

P

M

N

AMD Quad-Core Opteron Four cores on the same die

P

M

P

M

N

IBM Blue Gene QSupercomputer Thousands of 18-core processors

P

M

P

M



Trend 3: Energy and Performance of Multicores

Performance (Tasks per Second)

(Joulesper

Task)

SimpleSingle Proc

Processor PowerConstraint

A

Increasin

g Power

AAA

Energy

B

AAAA

Superscalarw/ Deeper

Pipelines

COut-of-Order Superscalar

Superpipelined

B

D

EEMulticore FFMulticore E

General-PurposeManycores

F



Trend 3: The Multicore “Hail Mary Pass”


MIPSR2K

IntelP4

1975 1980 1985 1990 1995 2000 2005 2010 2015

100

101

102

103

104

105

106

107

DECAlpha 21264

TypicalPower (W)

Frequency(MHz)

SPECintPerformance

~9%/year

~15%/year

Numberof Cores

Intel 48-Core Prototype

Parallelism?

AMD 4-Core Opteron



Trend 4: Heterogeneous Systems-on-Chip

OMAP'4'SoC'

Today’s&SoC&

ARM&Cores& GPU&DSP& DSP&

System&Bus&

Secondary&&Bus&

Secondary&&Bus&

Ter=ary&&Bus&

DMA&

DMA& SD&USB&Audio& Video& Face& Imaging&

USB&

26&Adapted from D. Brooks Keynote at NSF XPS Workshop, May 2015.



Trend 5: Emerging Device Technologies

`90`80`70`60`50`40`30`20`10 `10

?Vertical MOSFETsGrapheneCarbon NanotubesNanorelaysQuantum ComputingMolecular ComputingMemristersPhase-Change MemSpintronics3D IntegrationNanophotonics

Adapted from R. Kurzweil. “The Singularity is Near.” Penguin Books, 2006.



Key trends in application requirements andtechnology constraints over the past decade have

resulted in a radical rethinking of theprocessors, memories, and networksused in modern computing systems

Five Key Trends in Computer Architecture

1. Growing diversity in application requirements motivate growingdiversity in computing systems pushing towards the cloud and IoT

2. Energy & power constrain systems across the computing spectrum

3. Transition to multiple cores integrated onto a single chip

4. Transition to heterogeneous systems-on-chip

5. Technology scaling challenges motivate new emerging compute,storage, and communication device technologies


What is Computer Architecture? Activity 1 Trends in Computer Architecture • Activity 2 • Computer Architecture Design

RTL

Devices

ISA

PL

Algorithm

�Arch

Technology

Application

OS

Gates

Circuits

Agenda


Activity 1


Activity 2




Activity #2: Sorting with a Parallel Processor

P

M

P

M

P

M

P

M

N

I Application: Sort 32 numbers

I Simulated Parallel Computing System. Processor: Group of 2–8 students. Memory: Worksheet, scratch paper. Network: Communicating between students

I Activity Steps. 1. Discuss strategy with group. 2. When instructor starts timer, master processor flips over worksheet. 3. Sort 32 numbers as fast as possible. 4. Lookup when completed and write time on worksheet. 5. Master processor only raises hand. 6. When everyone is finished, then analyze data



Activity #2: Discussion

unsorted

sorted

Distribute

Sort 4 Numbers

Merge Phase 1 > merge 4+4 = 8



Proc/Mem

Proc/Mem

Network

Network

Network

Proc/Mem

Proc/Mem

Network

Network

AlgorithmCommunicationLoad BalancingFault Tolerance

Dataset Size


What is Computer Architecture? Activity 1 Trends in Computer Architecture Activity 2 • Computer Architecture Design •

RTL

Devices

ISA

PL

Algorithm

�Arch

Technology

Application

OS

Gates

Circuits

Agenda


Activity 1


Activity 2




What do computer architects actually do?

Ask questionabout nature

Constructhypothesis

Test withexperiment

Analyze results anddraw conclusions

General ScienceDiscover truthsabout nature

Design and modelbaseline system

Ask questionabout system

Design and modelalternative system

Test withexperiment


Computer EngineeringExplore design space

for a new system

Build prototypeor real system



Modeling in Computer Architecture




Test withexperiment



for a new system


// rdy is OR of the AND of reqs and grants assign in_rdy = | (reqs & grants);

reg [2:0] reqs; always @(*) begin if ( in_val ) begin

// eject packet if it is for this tile if ( dest == p_router_id ) reqs = 3'b010;

// otherwise, just pass it along ring else reqs = 3'b001;

end else begin // if !val, don't request any ports reqs = 3'b000; end end

Verilog • SystemVerilog • VHDLC++ • SystemC

Bluespec • Chisel • Python



How do we design something so incredibly complex?




Test withexperiment



for a new system


Fighter Airplane: ~100,000 partsIntel Sandy Bridge E:2.27 Billion transistors



I Design Principles. Modularity – Decompose into components with well-defined interfaces. Hierarchy – Recursively apply modularity principle. Encapsulation – Hide implementation details from interfaces. Regularity – Leverage structure at various levels of abstraction. Extensibility – Include mechanisms/hooks to simplify future changes

I Design Patterns. Processors, Memories, Networks. Control/Datapath Split. Single-Cycle, FSM, Pipelined Control. Raw Port, Message, Method Interfaces

I Design Methodologies. Agile Hardware Development. Test-driven Development. Incremental Development



Final Goal for Lab Assignments

P

I$

D$

Network

Network

Network

P

I$

P

I$

P

I$

D$ D$ D$

Mai

n M

emor

y

Quad-core processor with privateL1 instruction caches and a shared,banked L1 data cacheinterconnected through various ringnetworks implemented at theregister-transfer-level and capablerunning real parallel programs

Lab assignments will use an agilehardware developmentmethodology based on a Pythonhardware modeling framework, theVerilog hardware descriptionlanguage (optional), the GitHubrepository hosting site, and and theTravisCI continuous integrationservice



imemreq_msg_addr

imemresp_msg

P

I$

D$

Network

Network

Network

P

I$

P

I$

P

I$

D$ D$ D$

Mai

n M

emor

y

b_regb_mux_sel

32b

a_rega_mux_sel

32b

result_reg

result_mux_sel

result_en

32b

32b

32b

>>

<<

add_mux_sel

32b

b_lsb

0

IDLE

CALC

DONE

!resp_rdy

!req_val

req_val

resp_rdy

counter = 32

(counter < 32)&& (b[0] == 1)

/add,shift

(counter < 32)&& (b[0] == 0)/shift

Lab 1: Iterative Multiplier

sext

ir[20:16]

ir[15:0]

BrTarget

ir[25:21]

op0_reg_X

ex_result_reg_M

regfile(read)

regfile(write)alu

alu_fn_X

wb_result_sel_M rf_wen_W

rf_waddr_W

br_cond_zero_Xbr_target_reg_X

op1_sel_D

reg_en_D

reg_en_X

op1_reg_X

dmemreq_msg_addr

dmemreq_msgto Memory

dmemresp_msg_data

reg_en_M reg_en_W

to_mngr_data

pc_sel_F

reg_en_F

pc_reg_Fpc_plus4_reg_D

inst_D_reg

unpacked using inst_unpack unpacked using inst_unpack unpacked using inst_unpack unpacked using inst_unpack

wb_result_reg_W

JTargetir[25:0]

dmem_write_data_reg_X

dmemreq_msg_data

16

op0_sel_D

inst_shamt_zext_Dir[10:6]

zextir[15:0]

imul

ex_result_sel_X

j_target_Djr_target_Dbr_target_X

imul_req_val_Dimul_req_rdy_D

imul_resp_val_Ximul_resp_rdy_X

+4pc_incr_F

pc_plus4_F

pc_next_F

pc_F

br_cond_neg_X

br_cond_eq_X

zext

imemresp_msg_datamsg

unpack

imemresp_msg_unpack

P Lab 2: Pipelined Processor

tagcheck

readdata

access

wait

writedata

access

idle

evictprepare

evictrequest

refillrequest

evictwait

refillupdate

refillwait

readhit

writehit

write

read

!cacheresp_rdy

!cachereq_val

miss & dirty

!memreq_rdy

!memreq_rdy

!memresp_val

miss & !dirty

initdata

access

inittransaction

memreq_rdy

memresp_val

memreq_rdy

mem

resp

_val

read_addr

write_addr

write_data

write_en

read_en

write_byte_en

read_data

TagArray

tag_array_wen

tag_array

_ren

3'b111

read_addr

write_addr

write_data

write_en

read_en

write_byte_en

read_data

DataArray

data_array_wen

data_array

_ren

data_array_wben

memresp_en

repl

128b

memresp_data_reg

cachereq_type_reg

write_data_mux_sel

cachereq_en

cachereq_addr_reg

cachereq_data_regv

c_M

emR

eqM

sgU

np

ack

dat

aad

dr

typ

e

vc_

Mem

Req

Msg

Un

pac

k

dat

aad

dr

typ

e

cach

ereq

_msg

vc_

Mem

Res

pM

sgU

np

ack

dat

aig

ore

oth

er f

ield

s

mem

resp

_msg 128b

32b

tagidx

cmp

tagidx

read_data_reg_en

read_data_reg

evict_addr_reg_en

evict_addr_reg

mkaddr

memreq_addr_mux_sel

mkaddr

cachereq_type

cachereq_addr

vc_

Mem

Res

pM

sgP

ack

dat

aty

pe

cach

eres

p_m

sg

vc_

Mem

Req

Msg

Pac

k

dat

a mem

req

_msg

len

len

127:96

95:64

63:32

31:0

read_word_mux_sel

2'b0

2'b0

cacheresp_type

memreq_type

tag_match

128b

128b

32b

test

opaq

ue

opaq

ue

8'b0

cachereq_opaque_reg

opaq

ue

hit

add

r

Lab 3:BlockingCache

I$

in0_deq_valin0_deq_rdy



out0_valout0_rdy

out1_valout1_rdy

out2_valout2_rdy

num_free_west

num_free_east

xbar_sel0

xbar_sel1

xbar_sel2

dest0size: 4

size: 4

size: 4

in0_queue

in1_queue

in2_queue

in0_deq_msg

in1_deq_msg

in2_deq_msg

xbar

dest1

dest2

R

chan

nel

Rch

anne

lR

R channelR

channel

Rchannel

R

channel

Rchannel

channel

RchannelR

Rchannel

Rchan

nel

Rch

anne

lR

chan

nel

chan

nel

R channelR

channel

Lab 4:Ring Network

Network

Lab 5: Muliticore


What is Computer Architecture? Activity 1 Trends in Computer Architecture Activity 2 Computer Architecture Design

RTL

Devices

ISA

PL

Algorithm

�Arch

Technology

Application

OS

Gates

Circuits

Take-Away Points

I Computer architecture is the process of buildingcomputing systems to meet given applicationrequirements within physical technology constraints

I We are entering an exciting new era of computerarchitecture with growing diversity in applicationsand systems, a remarkable industrial shift towardsmainstream parallel processing and SoCs, andsignificant technology scaling challenges

I This era offers tremendous challenges andopportunities, which makes it a wonderful time tostudy and contribute to the field of computerarchitecture


Documents

New ECE 4750 Computer Architecture Course Overview · 2017. 8. 14. · What is Computer Architecture? Activity 1 • Trends in Computer Architecture • Activity 2 Computer Architecture