EECC551 - ShaabanEECC551 - Shaaban#1 lec # 6 Winter 2005 1-11-2006
Evolution of Microprocessor Evolution of Microprocessor PerformancePerformance
Source: John P. Chen, Intel Labs
CPI > 10 1.1-10 0.5 - 1.1 .35 - .5 (?)
Pipelined(single issue)
Multi-cycle Multiple Issue (CPI <1) Superscalar/VLIW/SMT
So far we examined static & dynamic techniques to improve the performance of single-issue (scalar) pipelined CPU designs including: static & dynamic scheduling, static & dynamic branch predication. Even with these improvements, the restriction of issuing a single instruction per cycle still limits the ideal CPI = 1
(Chapter 3.6, 3.7, 4.3, 4.5)
We next examine the two approaches to achieve a CPI < 1by issuing multiple instructions per cycle:
• Superscalar CPUs• Very Long Instruction Word (VLIW) CPUs.
Single-issue Processor = Scalar ProcessorInstructions Per Cycle (IPC) = 1/CPI
IPC
?1 GHzto ???? GHz
Original(2002)IntelPredictions15 GHz
EECC551 - ShaabanEECC551 - Shaaban#2 lec # 6 Winter 2005 1-11-2006
Tran
sist
ors
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000
1970 1975 1980 1985 1990 1995 2000 2005
Bit-level parallelism Instruction-level Thread-level (?)
i4004
i8008i8080
i8086
i80286
i80386
R2000
Pentium
R10000
R3000
Parallelism in Microprocessor VLSI GenerationsParallelism in Microprocessor VLSI Generations
Simultaneous Multithreading SMT:e.g. Intel’s Hyper-threading
Chip-Multiprocessors (CMPs)e.g IBM Power 4, 5 AMD Athlon64 X2 Intel Pentium D
Multiple micro-operations per cycle(multi-cycle non-pipelined)
Superscalar/VLIWCPI <1Single-issue
PipelinedCPI =1
Not PipelinedCPI >> 1
Instruction Level Parallelism (ILP)
Thread Level Parallelism (TLP)
Pipeliningsingle issue
MultipleInstructionIssue
Superscalar ProcessorsVery Long Instruction Word (VLIW) ISA/ProcessorsCPI < 1
(ILP)
(TLP)
EECC551 - ShaabanEECC551 - Shaaban#3 lec # 6 Winter 2005 1-11-2006
Multiple Instruction Issue: CPI < 1Multiple Instruction Issue: CPI < 1 • To improve a pipeline’s CPI to be better [less] than one, and to better exploit
Instruction Level Parallelism (ILP), a number of instructions have to be issued in the same cycle.
• Multiple instruction issue processors are of two types:
– Superscalar: A number of instructions (2-8) is issued in the same cycle, scheduled statically by the compiler or -more commonly- dynamically (Tomasulo).
• PowerPC, Sun UltraSparc, Alpha, HP 8000, Intel PII, III, 4 ...
– VLIW (Very Long Instruction Word): A fixed number of instructions (3-6) are formatted as one long
instruction word or packet (statically scheduled by the compiler). – Example: Explicitly Parallel Instruction Computer (EPIC)
• Originally a joint HP/Intel effort.• ISA: Intel Architecture-64 (IA-64) 64-bit address:• First CPU: Itanium, Q1 2001. Itanium 2 (2003)
• Limitations of the approaches:– Available ILP in the program (both).– Specific hardware implementation difficulties (superscalar).– VLIW optimal compiler design issues.
CPI < 1 or CPI < 1 or Instructions Per Cycle (IPC) > 1
Most common = 4 instructions/cyclecalled 4-way superscalar processor
Chapter 3.6, 3.7, 4.3, 4.5
EECC551 - ShaabanEECC551 - Shaaban#4 lec # 6 Winter 2005 1-11-2006
Multiple Instruction Issue:Multiple Instruction Issue:
SuperscalarSuperscalar Vs. Vs. VLIWVLIW• Usually dynamically
scheduled (Tomasulo)
• Complex scheduling hardware.
• Smaller code size.– Fewer registers
– No template field
• Binary compatibility across generations of hardware.
• Statically scheduled.
• Complex compiler design.
• Simplified Hardware for decoding, issuing instructions.
• Usually no Interlock Hardware (compiler checks?)
• More ISA registers (no register renaming) , but simplified hardware for register ports.
(Hardware-based) (Software-based)
EECC551 - ShaabanEECC551 - Shaaban#5 lec # 6 Winter 2005 1-11-2006
• Two instructions can be issued per cycle (static two-issue or 2-way superscalar).• One of the instructions is integer (including load/store, branch). The other instruction is a floating-point operation.
– This restriction reduces the complexity of hazard checking. • Hardware must fetch and decode two instructions per cycle.• Then it determines whether zero (a stall), one or two instructions can be issued (in decode stage) per cycle.
Simple Statically Scheduled Superscalar PipelineSimple Statically Scheduled Superscalar Pipeline
MEM
EX
EX
EX
ID
ID
IF
IF
EX
EX
ID
ID
IF
IF
WB
WB
EX
MEM
EX
EX
EX
WB
WB
EX
MEM
EX
WB
WB
EX
ID
ID
IF
IF
WB
EX
MEM
EX
EX
EX
ID
ID
IF
IF
Integer Instruction
Integer Instruction
Integer Instruction
Integer Instruction
FP Instruction
FP Instruction
FP Instruction
FP Instruction
1 2 3 4 5 6 7 8Instruction Type
Two-issue statically scheduled pipeline in operationTwo-issue statically scheduled pipeline in operationFP instructions assumed to be adds (EX takes 3 cycles)FP instructions assumed to be adds (EX takes 3 cycles)
(Ch. 3.6) Ideal CPI = 0.5 Ideal Instructions Per Cycle (IPC) = 2
Instructions assumed independent (no stalls)
EECC551 - ShaabanEECC551 - Shaaban#6 lec # 6 Winter 2005 1-11-2006
Intel IA-64: VLIW “Explicitly Parallel Intel IA-64: VLIW “Explicitly Parallel Instruction Computing (EPIC)”Instruction Computing (EPIC)”
• Three 41-bit instructions in 128 bit “Groups” or bundles; an instruction bundle template field (5-bits) determines if instructions are dependent or independent and statically specifies the functional units to used by the instructions:– Smaller code size than old VLIW, larger than x86/RISC– Groups can be linked to show dependencies of more than three
instructions.
• 128 integer registers + 128 floating point registers• Hardware checks dependencies
(interlocks binary compatibility over time)
• Predicated execution: An implementation of conditional instructions used to reduce the number of conditional branches used in the generated code larger basic block size
• IA-64 : Name given to instruction set architecture (ISA).• Itanium : Name of the first implementation (2001).
In VLIW dependency analysis is done statically by the compilernot dynamically in hardware (Tomasulo)
EECC551 - ShaabanEECC551 - Shaaban#7 lec # 6 Winter 2005 1-11-2006
Intel/HP EPIC VLIW ApproachIntel/HP EPIC VLIW Approachoriginal sourceoriginal source
codecode
ExposeExposeInstructionInstructionParallelismParallelism(dependency(dependency
analysis)analysis)OptimizeOptimize
Exploit Exploit InstructionInstructionParallelism:Parallelism:
GenerateGenerateVLIWsVLIWs
compilercompiler
Instruction DependencyInstruction DependencyAnalysisAnalysis
Instruction 2Instruction 2 Instruction 1Instruction 1 Instruction 0Instruction 0 TemplateTemplate
128-bit bundle128-bit bundle
00127127
41 bits 41 bits 5 bits41 bits
SequentialCode
Dependency Graph
Template field has static assignment/scheduling information
EECC551 - ShaabanEECC551 - Shaaban#8 lec # 6 Winter 2005 1-11-2006
IA-64 Instruction TypesIA-64 Instruction Types
I-unit/B-unitExtendedL+X
B-unitBranchB
F-unitFloating PointF
M-unitMemoryM
I-unitNon-integer ALUI
I-unit or M-unitInteger ALUA
Execution Unit TypeDescriptionInstruction Type
Information on static assignment of instructions to functional units and instruction typed in a bundle contained in the template field
EECC551 - ShaabanEECC551 - Shaaban#9 lec # 6 Winter 2005 1-11-2006
IA-64 Template UseIA-64 Template Use• The template specifies the functional units for the three
operations (instructions) in the instruction bundle.
– Part of static scheduling
• Possible instruction combinations:– M-unit, I-unit, I-unit
– M-unit, L-unit, X-unit
– M-unit, M-unit, I-unit
– M-unit, F-unit, I-unit
– M-unit, M-unit, F-unit
– M-unit, I-unit, B-unit
– M-unit, B-unit, B-unit
– B-unit, B-unit, B-unit
– M-unit, M-unit, B-unit
– M-unit, F-unit, B-unit
EECC551 - ShaabanEECC551 - Shaaban#10 lec # 6 Winter 2005 1-11-2006
6777241214
qpR1R2R3X2bX4VeX2a8
056121319202627282932333435363740
8 is the major opcode for this instruction type.
X2a, X2b, Ve, and X4 are opcode extensions.
qp is the predicate register (or predication flag) assigned to this operation (64 such flags)
Register-register format:
IA-64 Instruction Format Example:IA-64 Instruction Format Example:
Type A (Integer ALU) Instruction Format
Predication: Any instruction can be cancelled (turned into a no-op) based on the value of one of 64 predication flags (qp)
EECC551 - ShaabanEECC551 - Shaaban#11 lec # 6 Winter 2005 1-11-2006
677761214
qpR1imm7bR3imm6dVeX2as8
05612131920262732333435363740
X2a and Ve are opcode extensions.
qp is the predicate register assigned to this operation.
The immediate value is made up of s, imm6d, and imm7b.
Other formats available for 8-bit and 22-bit immediates.
14-bit Immediate format:
IA-64 Instruction Format Example:IA-64 Instruction Format Example:
Type A (Integer ALU) Instruction Format
EECC551 - ShaabanEECC551 - Shaaban#12 lec # 6 Winter 2005 1-11-2006
1
x
27
67772214
qpR1R3hintX6m4
056121319202628293035363740
m and x determine the subset of load/store operations.
Bits 35:32 give special directives for LAT, cache, etc.
Bits 31:30 tell the size of the load (1, 2, 4, or 8 bytes).
Hint allows for system pre-fetching of data by specifying if temporal locality exists for this data.
Integer Load/Store operations:
IA-64 Instruction Format Example:IA-64 Instruction Format Example:
Type M (Memory) Instruction Format
EECC551 - ShaabanEECC551 - Shaaban#13 lec # 6 Winter 2005 1-11-2006
1
p
12
1
s
36
63320214
qpbtypeimm20bwhdopcode
056891120263334353740
Opcode 0- indirect branch, 1- indirect call, 4- IP-relative branch,
5- IP-relative call btype sub-category of branch type, qp- branch condition d- cache de-allocation, wh- branch hint (static branch prediction), s & imm20b give offset p pre-fetch 0-few, or 1-many instructions following the target
IP-relative branch:
IA-64 Instruction Format Example:IA-64 Instruction Format Example:
Type B (Branch) Instruction Format
{0 00 1
1 01 1
Not Taken
Taken
IP = Instruction Pointer (e.g. PC)
EECC551 - ShaabanEECC551 - Shaaban#14 lec # 6 Winter 2005 1-11-2006
Unrolled Loop Example for Unrolled Loop Example for Scalar (single-issue) PipelineScalar (single-issue) Pipeline
1 Loop: L.D F0,0(R1)2 L.D F6,-8(R1)3 L.D F10,-16(R1)4 L.D F14,-24(R1)5 ADD.D F4,F0,F26 ADD.D F8,F6,F27 ADD.D F12,F10,F28 ADD.D F16,F14,F29 S.D F4,0(R1)10 S.D F8,-8(R1)11 DADDUI R1,R1,#-3212 S.D F12,16(R1)13 BNE R1,R2,LOOP14 S.D F16,8(R1) ; 8-32 = -24
14 clock cycles, or 3.5 per original iteration (result)(unrolled four times)
Latency:L.D to ADD.D: 1 CycleADD.D to S.D: 2 Cycles
Unrolled and scheduled loopfrom loop unrolling examplein lecture # 3 (slide 10)
No stalls in code above: CPI = 1 (ignoring initial pipeline fill cycles)
EECC551 - ShaabanEECC551 - Shaaban#15 lec # 6 Winter 2005 1-11-2006
Loop Unrolling in 2-way Superscalar Pipeline: Loop Unrolling in 2-way Superscalar Pipeline:
(1 Integer, 1 FP/Cycle)(1 Integer, 1 FP/Cycle)Integer instruction FP instruction Clock cycle
Loop: L.D F0,0(R1) 1
L.D F6,-8(R1) 2
L.D F10,-16(R1) ADD.D F4,F0,F2 3
L.D F14,-24(R1) ADD.D F8,F6,F2 4
L.D F18,-32(R1) ADD.D F12,F10,F2 5
S.D F4,0(R1) ADD.D F16,F14,F2 6
S.D F8,-8(R1) ADD.D F20,F18,F2 7
S.D F12,-16(R1) 8
DADDUI R1,R1,#-40 9
S.D F16,-24(R1) 10
BNE R1,R2,LOOP 11
SD -32(R1),F20 12• Unrolled 5 times to avoid delays and expose more ILP (unrolled one more time)• 12 cycles, or 12/5 = 2.4 cycles per iteration (3.5/2.4= 1.5X faster than scalar)• CPI = 12/ 17 = .7 worse than ideal CPI = .5 because 7 issue slots are wasted
Empty or wastedissue slot
Recall that loop unrolling exposes more ILP by increasing basic block size
EECC551 - ShaabanEECC551 - Shaaban#16 lec # 6 Winter 2005 1-11-2006
Loop Unrolling in VLIW PipelineLoop Unrolling in VLIW Pipeline(2 Memory, 2 FP, 1 Integer / Cycle)(2 Memory, 2 FP, 1 Integer / Cycle)
Memory Memory FP FP Int. op/ Clockreference 1 reference 2 operation 1 op. 2 branchL.D F0,0(R1) L.D F6,-8(R1) 1L.D F10,-16(R1) L.D F14,-24(R1) 2L.D F18,-32(R1) L.D F22,-40(R1) ADD.D F4,F0,F2 ADD.D F8,F6,F2 3L.D F26,-48(R1) ADD.D F12,F10,F2 ADD.D F16,F14,F2 4
ADD.D F20,F18,F2 ADD.D F24,F22,F2 5S.D F4,0(R1) S.D F8, -8(R1) ADD.D F28,F26,F2 6S.D F12, -16(R1) S.D F16,-24(R1) DADDUI R1,R1,#-56 7S.D F20, 24(R1) S.D F24,16(R1) 8S.D F28, 8(R1) BNE R1,R2,LOOP 9
Unrolled 7 times to avoid delays and expose more ILP 7 results in 9 cycles, or 1.3 cycles per iteration (2.4/1.3 =1.8X faster than 2-issue superscalar, 3.5/1.3 = 2.7X faster than scalar) Average: about 23/9 = 2.55 IPC (instructions per clock cycle) Ideal IPC =5, CPI = .39 Ideal CPI = .2 thus about 50% efficiency, 22 issue slots are wasted Note: Needs more registers in VLIW (15 vs. 6 in Superscalar)
(In chapter 4.3 pages 317-318)
Empty or wastedissue slot
EECC551 - ShaabanEECC551 - Shaaban#17 lec # 6 Winter 2005 1-11-2006
Superscalar Dynamic SchedulingSuperscalar Dynamic Scheduling• The Tomasulo dynamic scheduling algorithm is extended to issue more than one instruction per
cycle.
• However the restriction that instructions must issue in program order still holds to avoid violating instruction dependencies (construct correct dependency graph dynamically).
– The result of issuing multiple instructions in one cycle should be the same as if they were single-issued, one instruction per cycle.
• How to issue two instructions and keep in-order instruction issue for Tomasulo?
• Simplest Method: Restrict Type of Instructions Issued Per Cycle• To simplify the issue logic, issue one one integer + one floating-point instruction per cycle (for
a 2-way superscalar). – 1 Tomasulo control for integer, 1 for floating point.
• FP loads/stores might cause a dependency between integer and FP issue:– Replace load reservation stations with a load queue; operands must be read in the order they are
fetched (program order).– Replace store reservation stations with a store queue; operands must be written in the order they
are fetched.
• Load checks addresses in Store Queue to avoid RAW violation– (get load value from store queue if memory address matches)
• Store checks addresses in Load Queue to avoid WAR, and checks Store Queue to avoid WAW.
(the above load/store queue checking is also applicable to single-issue Tomasulo to take care of memory RAW, WAR, WAW). More on this next ..
EECC551 - ShaabanEECC551 - Shaaban#18 lec # 6 Winter 2005 1-11-2006
Data Memory Dependency Checking/Handling In Dynamically Scheduled Processors
(Related discussion in textbook page 195)
• Renaming in Tomasolu-based dynamically scheduled processors eliminates name dependence for register access but not for data memory access• Thus both true data dependencies and name dependencies must be detected to ensure correct ordering of data memory accesses for correct
execution.• One possible solution:• For Loads: Check store queue/buffers to ensure no data dependence violation (RAW hazard)• For Stores:
– Check store queue/buffers to ensure no output name dependence violation (WAW hazard)– Check load queue/buffers to ensure no anti-dependence violation (WAR hazard)
.
.
From CPU
To Data Memory
Store Queue/Buffers
.
.
To CPU
From Data Memory
Load Queue/Buffers
For Store Instructions
For Load Instructions
Check Store Queuefor possible anti-dependence(WAR hazard) In case of anaddress match delay the currentstore until all pending loads are completed
Check Store Queuefor possible outputdependence(WAW hazard)In case of addressmatch ensurethis store will occurlast
Check Store Queue for possible true data dependence(RAW hazard) In case of an address match get the value of the that store
Note: Since instructions issue in program order, all pending load/store instructions in load/store queues are before the current load/store instruction in program order.
EECC551 - ShaabanEECC551 - Shaaban#19 lec # 6 Winter 2005 1-11-2006
Three techniques can be used to support multiple instruction issue in Tomasulo
without putting restrictions on the type of instructions issued per cycle:
1 Issue at a higher clock rate so that issue remains in order.– For example for a 2-Issue supercalar issue at 2X Clock Rate.
2 Widen the issue logic to handle multiple instruction issue– All possible dependencies between instructions to be issues are detected at
once and the result of the multiple issue matches in-order issue
Superscalar Dynamic SchedulingSuperscalar Dynamic Scheduling
IssueFirstInstruction
IssueSecondInstruction
One Cycle
CheckInstructionDependencies
IssueBothInstructions
One Cycle
0, 1 or 2 instructions issued per cyclefor either method
2-Issue superscalar
For correct dynamic construction of dependency graph:The result of issuing multiple instructions in one cycle should be the same as if they were single-issued, one instruction per cycle.
EECC551 - ShaabanEECC551 - Shaaban#20 lec # 6 Winter 2005 1-11-2006
3 To avoid increasing the CPU clock cycle time in the last two approaches, multiple instruction issue can be spilt into two pipelined issue stages:
– Issue Stage One: Decide how many instructions can issue simultaneously checking dependencies within the group of instructions to be issued + available RSs, ignoring instructions already issued.
– Issue Stage Two: Examine dependencies among the selected instructions from the group and the those already issued.
• This approach is usually used in dynamically-scheduled wide superscalars that can issue four or more instructions per cycle.
• Splitting the issue into two pipelined staged increases the CPU pipeline depth and increases branch penalties
– This increases the importance of accurate dynamic branch prediction methods.
• Further pipelining of issue stages beyond two stages may be necessary as CPU clock rates are increased.
• The dynamic scheduling/issue control logic for superscalars is generally very complex growing at least quadratically with issue width.
– e.g 4 wide superscalar -> 4x4 = 16 times complexity of single issue CPU
Superscalar Dynamic SchedulingSuperscalar Dynamic Scheduling
EECC551 - ShaabanEECC551 - Shaaban#21 lec # 6 Winter 2005 1-11-2006
Multiple Instruction Issue with Dynamic Scheduling Example
Example on page 221
Assumptions:
Restricted 2-way superscalar:1 integer, 1 FP Issue Per Cycle
A sufficient number of reservation stations is available.
Total two integer units available: One integer unit (for ALU, effective address)One integer unit for branch condition
2 CDBs
Execution cycles:Integer: 1 cycleLoad: 2 cycles (1 ex + 1 mem)FP add: 3 cycles
Any instruction followinga branch cannot start executionuntil after branch condition is evaluated in EX (resolved)
Branches are single issued, no delayed branch,perfect branch prediction
EECC551 - ShaabanEECC551 - Shaaban#22 lec # 6 Winter 2005 1-11-2006
Multiple Instruction Issue with Dynamic Scheduling Example
EECC551 - ShaabanEECC551 - Shaaban#23 lec # 6 Winter 2005 1-11-2006
Only one CDB is actually needed in this case.
Three Loop Iterations on Restricted 2-way Superscalar Tomasulo
(Start)
FP ADD has 3 execution cyclesBranches single issue
EECC551 - ShaabanEECC551 - Shaaban#24 lec # 6 Winter 2005 1-11-2006
Resource Usage Table for Example:
Only one CDB is actually needed in this case.
EECC551 - ShaabanEECC551 - Shaaban#25 lec # 6 Winter 2005 1-11-2006
Multiple Instruction Issue with Dynamic Scheduling Example
Example on page 223
Assumptions:The same loop in previous exampleOn restricted 2-way superscalar:1 integer, 1 FP Issue Per Cycle
A sufficient number of reservation stations is available.
Total three integer unitsone for ALU, one for effective addressOne integer unit for branch condition2 CDBs
Execution cycles:Integer: 1 cycleLoad: 2 cycles (1 ex + 1 mem)FP add: 3 cycles
Any instruction followinga branch cannot start executionuntil after branch condition is evaluated
Branches are single issued, no delayed branch,perfect branch prediction
Previous example repeated with one more integer ALU (3 total)
EECC551 - ShaabanEECC551 - Shaaban#26 lec # 6 Winter 2005 1-11-2006
Same three loop Iterations on Restricted 2-way Superscalar Tomasulo but with Three integer units (one for ALU, one for effective address calculation, one for branch condition)
(page 224)
(Start)
For instructions after a branch: Execution starts after branch is resolved
Both CDBs are used here (in cycles 4, 8)
16 vs. 19 cycles
EECC551 - ShaabanEECC551 - Shaaban#27 lec # 6 Winter 2005 1-11-2006
(page 225)
Resource Usage Table for Example:
Both CDBs are used here (in cycles 4, 8)
EECC551 - ShaabanEECC551 - Shaaban#28 lec # 6 Winter 2005 1-11-2006
Limits to Multiple Instruction Issue Limits to Multiple Instruction Issue MachinesMachines• Inherent limitations of ILP:
– If 1 branch exist for every 5 instruction : How to keep a 5-way superscalar/VLIW processor busy?
– Latencies of unit adds complexity to the many operations that must be scheduled every cycle.
– For maximum performance multiple instruction issue requires about: Pipeline Depth x No. Functional Units
active instructions at any given cycle.
• Hardware implementation complexities:– Duplicate FUs for parallel execution are needed, more CDBs.
– More instruction bandwidth is essential.
– Increased number of ports to Register File (datapath bandwidth):• VLIW example needs 7 read and 3 write for Int. Reg.
& 5 read and 3 write for FP reg
– Increased ports to memory (to improve memory bandwidth).– Superscalar issue/decoding complexity may impact pipeline clock rate, depth.
i.e to achieve 100% Functional unit utilization
EECC551 - ShaabanEECC551 - Shaaban#29 lec # 6 Winter 2005 1-11-2006
Conditional Instructions and Speculation Conditional Instructions and Speculation • Compiler ILP techniques (loop-unrolling, software Pipelining etc.) are
not effective to uncover maximum ILP when branch behavior is not well known at compile time.
• Techniques to further reduce the impact of branches on performance:
– Conditional or Predicted Instructions: An extension to the instruction set with instructions that turn into no-ops if a condition is not valid at run time (e.g. canceling branch delay instruction).
– Speculation: An instruction is executed before the processor knows that the instruction should execute to avoid control dependence stalls (i.e. branch not resolved yet):
• Static Speculation by the compiler with hardware support:– The compiler labels an instruction as speculative and the hardware helps
by ignoring the outcome of incorrectly speculated instructions.
– Conditional instructions provide limited speculation.
• Dynamic Hardware-based Speculation: (Ch. 3.7)– Uses dynamic branch-prediction to guide the speculation process.– Dynamic scheduling and execution continued passed a conditional branch
in the predicted branch direction.No ISAor CompilerSupport Needed
ISA/CompilerSupport Needed
ISA Support Needed
- (Speculative Tomasulo)
e.g dynamic speculative execution
EECC551 - ShaabanEECC551 - Shaaban#30 lec # 6 Winter 2005 1-11-2006
Conditional or Predicted InstructionsConditional or Predicted Instructions• Avoid branch prediction by turning branches into
conditionally-executed instructions (helps increase average size of basic blocks by reducing the number of branches):
if (x) then (A = B op C) else NOP– If false, then neither store result nor cause exception:
instruction is annulled (turned into NOP) .– Expanded ISA of Alpha, MIPS, PowerPC, SPARC have
conditional move.– HP PA-RISC can annul any following instruction.– IA-64: 64 1-bit condition fields (flags) selected so conditional execution of any instruction (Predication).
• Drawbacks of conditional instructions– Still takes a clock cycle even if “annulled”.– Must stall if condition is evaluated late.– Complex conditions reduce effectiveness;
condition becomes known late in pipeline.
x
A = B op C
NOP
(Ch. 4.5) ISA/Compiler Support Needed
ISA Based:
EECC551 - ShaabanEECC551 - Shaaban#31 lec # 6 Winter 2005 1-11-2006
IA-64 Predication ExampleIA-64 Predication Example
Taken Direction Not Taken Direction
Predication Flag evaluated
Predication in IA-64 turns control dependence into data dependence on predication flags
64 Predication flags in IA-64
TakenDirection
Not TakenDirection
Predication: Any instruction can be cancelled (turned into a no-op) based on the value of one of 64 predication flags (qp)
ISA Based: Conditional Execution in IA-64 VLIW (Predication)
1 0
EECC551 - ShaabanEECC551 - Shaaban#32 lec # 6 Winter 2005 1-11-2006
Dynamic Hardware-Based SpeculationDynamic Hardware-Based Speculation• Combines:Combines:
– Dynamic hardware-based branch prediction– Dynamic Scheduling: issue multiple instructions in order and
execute out of order. (Tomasulo)
• Continue to dynamically issue, and execute instructions passed a conditional branch in the dynamically predicted branch direction, before control dependencies are resolved.– This overcomes the ILP limitations of the basic block size.– Creates dynamically speculated instructions at run-time with no
ISA/compiler support at all.– If a branch turns out as mispredicted all such dynamically
speculated instructions must be prevented from changing the state of the machine (registers, memory).
• Addition of commit (retire, completion, or re-ordering) stage and forcing instructions to commit in their order in the code (i.e to write results to registers or memory in program order).
• Precise exceptions are possible since instructions must commit in order.
(Ch. 3.7)
How?
(Speculative Execution Processors, Speculative Tomasulo)
i.e speculated instructions must be cancelled
EECC551 - ShaabanEECC551 - Shaaban#33 lec # 6 Winter 2005 1-11-2006
Hardware-Based Hardware-Based SpeculationSpeculation
Speculative Execution +Speculative Execution + Tomasulo’s AlgorithmTomasulo’s Algorithm
Usuallyimplemented as a circularbuffer
StoreResults
Commit or Retirement
= Speculative Tomasulo
EECC551 - ShaabanEECC551 - Shaaban#34 lec # 6 Winter 2005 1-11-2006
Four Steps of Speculative Tomasulo AlgorithmFour Steps of Speculative Tomasulo Algorithm1. Issue — (In-order) Get an instruction from Instruction Queue
If a reservation station and a reorder buffer slot are free, issue instruction & send operands & reorder buffer number for destination (this stage is sometimes called “dispatch”)
2. Execution — (out-of-order) Operate on operands (EX) When both operands are ready then execute; if not ready, watch CDB for
result; when both operands are in reservation station, execute; checks RAW (sometimes called “issue”)
3. Write result — (out-of-order) Finish execution (WB) Write on Common Data Bus (CDB) to all awaiting FUs & reorder
buffer; mark reservation station available.4. Commit — (In-order) Update registers, memory with reorder buffer result
– When an instruction is at head of reorder buffer & the result is present, update register with result (or store to memory) and remove instruction from reorder buffer.
– A mispredicted branch at the head of the reorder buffer flushes the reorder buffer (cancels speculated instructions after the branch)
Instructions issue in order, execute (EX), write result (WB) out of order, but must commit in order.
EECC551 - ShaabanEECC551 - Shaaban#35 lec # 6 Winter 2005 1-11-2006
Hardware-Based Speculation Example
Example on page 229
Show speculated single-issue Tomasulo status when MUL.D is ready to commit
Single-issue speculative Tomasulo Example
EECC551 - ShaabanEECC551 - Shaaban#36 lec # 6 Winter 2005 1-11-2006
speculated Tomasulo status when MUL.D is ready to commit(next cycle)
Reorder bufferentry # for MUL.D
Reorder bufferentry # for DIV.D
Single-issue speculative Tomasulo Example
No result value For DIV.D(not done executing)
(AKA Commit or Retirement buffer)
EECC551 - ShaabanEECC551 - Shaaban#37 lec # 6 Winter 2005 1-11-2006
Hardware-Based Speculation Example
Example on page 231
First iteration instructions committed
Single-issue speculative Tomasulo
Next instruction to commit
EECC551 - ShaabanEECC551 - Shaaban#38 lec # 6 Winter 2005 1-11-2006
L.D. and MUL.D of first iteration have committed, other instructions completed execution
Nexttocommit
Single-issue speculative Tomasulo
What if branch was mispredicted ?
EECC551 - ShaabanEECC551 - Shaaban#39 lec # 6 Winter 2005 1-11-2006
Multiple Issue with Speculation Example(2-way superscalar with no restriction on issue instruction type)
Example on page 235
A sufficient number of reservation stations and reorder (commit) buffer entries are available.
Branches still single issue
Integer codeEx = 1 cycle
EECC551 - ShaabanEECC551 - Shaaban#40 lec # 6 Winter 2005 1-11-2006
Answer: Without Speculation
For instructions after a branch: Execution starts after branch is resolved
Branches Still Single Issue
EECC551 - ShaabanEECC551 - Shaaban#41 lec # 6 Winter 2005 1-11-2006
Answer: 2-way Superscalar Tomasulo With Speculation
Arrows show data dependencies
Branches Still Single Issue 14 cycles vs. 19 without speculation
EECC551 - ShaabanEECC551 - Shaaban#42 lec # 6 Winter 2005 1-11-2006
• Empty or wasted issue slots can be defined as either vertical waste or horizontal waste:
– Vertical waste is introduced when the processor issues no instructions in a cycle.
– Horizontal waste occurs when not all issue slots can be filled in a cycle.
Superscalar Architecture Limitations:Superscalar Architecture Limitations:Issue Slot Waste Classification
Example:
4-IssueSuperscalar
Ideal IPC =4Ideal CPI = .25
Instructions Per Cycle = IPC = 1/CPIAlso applies to VLIW
Result of issue slot waste: Actual Performance << Peak Performance
EECC551 - ShaabanEECC551 - Shaaban#43 lec # 6 Winter 2005 1-11-2006
Sources of Unused Issue Cycles in an 8-issue Superscalar Processor.
Processor busy represents the utilized issue slots; allothers represent wasted issue slots.
61% of the wasted cycles are vertical waste, theremainder are horizontal waste.
Workload: SPEC92 benchmark suite.
Source: Simultaneous Multithreading: Maximizing On-Chip Parallelism Dean Tullsen et al., Proceedings of the 22rd Annual International Symposium on Computer Architecture, June 1995, pages 392-403.
Ideal Instructions Per Cycle, IPC = 8Here real IPC about 1.5
Real IPC << Ideal IPC
1.5 << 8
(CPI = 1/8)
~ 81% of issue slots wasted
(18.75 % of ideal IPC)
(wasted)
EECC551 - ShaabanEECC551 - Shaaban#44 lec # 6 Winter 2005 1-11-2006
Superscalar Architecture Limitations :Superscalar Architecture Limitations :All possible causes of wasted issue slots, and latency-hiding or latency reducing
techniques that can reduce the number of cycles wasted by each cause.
Source: Simultaneous Multithreading: Maximizing On-Chip Parallelism Dean Tullsen et al., Proceedings of the 22rd Annual International Symposium on Computer Architecture, June 1995, pages 392-403.
Main Issue: One Thread leads to limited ILP (cannot fill issue slots)
Possible Solution: Exploit Thread Level Parallelism (TLP) within a single physical processor: - Simultaneous Multithreaded (SMT) Processor: The processor issues and executes instructions from a number of threads creating a number of logical processors within a single physical processor e.g. Intel’s HyperThreading (HT), each physical processor executes instructions from two threads