View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Branch Prediction TechniquesBranch Prediction Techniques
Alex RamirezAlex Ramirez
(based on the work of others)(based on the work of others)
UPC-BarcelonaUPC-Barcelona
Apr 18, 2023 Branch Prediction Strategies 2
Motivation (Run-time)Motivation (Run-time) Pipelined execution
A new intruction enters the pipeline every cycle...
…but still takes several cycles to execute
Control flow changes Two possible paths after a
branch is fetched Introduces pipeline
"bubbles"• Branch delay slots
Prediction offers a chance to avoid this bubbles
Problem increases with superscalar execution
FF WWMMAADD
FF WWMMAADD
FF WWMMAADD
A branch is fetched
But takes N cycles to execute
Pipeline bubble
FF WWMMAADD
Apr 18, 2023 Branch Prediction Strategies 3
Motivation (Compile-time)Motivation (Compile-time)
Many compiler optimizations depend on accurate branch prediction Trace / Superblock scheduling
Routine inline expansion
Inter-procedural register allocation
Code transformations
Code layout optimizations
Apr 18, 2023 Branch Prediction Strategies 4
Performance MetricsPerformance Metrics
Misprediction rate Mispredicted branches per executed branch
• Unfortunately the most usually found
Instructions per mispredicted branch Gives a better idea of the program behaviour
• Branches are not evenly spaced
Apr 18, 2023 Branch Prediction Strategies 5
Talk OutlineTalk Outline Static branch prediction
Simple heuristics Complex heuristics
Semi-static branch prediction Profile-based
Dynamic branch prediction Bimodal predictors Two level predictors Hybrid predictors
Multiple branch prediction Adapted branch predictors Tree-like subgraph predictors Trace predictors
Apr 18, 2023 Branch Prediction Strategies 6
Simple Static PredictorsSimple Static Predictors[J.E.Smith ISCA´81][J.E.Smith ISCA´81]
Simple heuristics Always taken Always not taken Backwards taken / Forward not taken
• Relies on the compiler to arrange the code following this assertion
Certain opcodes taken
Programmer provided hints
Apr 18, 2023 Branch Prediction Strategies 7
Simple Static Predictors (II)Simple Static Predictors (II)
20
30
40
50
60
70
80m
88ks
im gcc li
ijpe
g
perl
vort
ex
post
gres
takennot takenBTFNT
Apr 18, 2023 Branch Prediction Strategies 8
Simple Static Predictors (III)Simple Static Predictors (III)Optimized code layoutOptimized code layout
0
20
40
60
80
100m
88ks
im gcc li
ijpe
g
perl
vort
ex
post
gres
takennot takenBTFNT
Apr 18, 2023 Branch Prediction Strategies 9
Complex Static PredictorsComplex Static Predictors[Ball & Larus PLDI´93][Ball & Larus PLDI´93]
Based on: Control flow analysis of the code to determine loops Classify branches in:
• Pointer comparison: false• Avoid executing subroutine calls
– exception handlers
• Less than zero: false, Greater than zero: true– error codes less than zero
• FP equal: false• Avoid returning from a subroutine
– recursion base case
• Avoid blocks containing a store instruction• Go towards loop headers, avoid exiting loops• Favor reuse of the branch operand
Misprediction rate about 20%
Apr 18, 2023 Branch Prediction Strategies 10
Improving Static PredictorsImproving Static Predictors[Mueller & Whalley PLDI´92/95, Krall PLDI´94, Young & Smith ASPLOS´94][Mueller & Whalley PLDI´92/95, Krall PLDI´94, Young & Smith ASPLOS´94]
The compiler can lay out the code to match the static prediction
Code replicating techniques allow: Unconditional jump elimination
• Replicate code after the if-then-else convergence in both conditional paths
• Change unconditional loop edges to conditional ones Conditional branch elimination
• In some situations the branch outcome is known– Test a variable which has not been modified or has just been set
Static prediction using branch history information• Replicate if-then-else bodies to know the previous
branch outcomes
Apr 18, 2023 Branch Prediction Strategies 11
Removing Unconditional JumpsRemoving Unconditional Jumps
Apr 18, 2023 Branch Prediction Strategies 12
Removing Unconditional Jumps (II)Removing Unconditional Jumps (II)
Apr 18, 2023 Branch Prediction Strategies 13
Removing Conditional BranchesRemoving Conditional Branches
flag = 1;while (cnd1 && flag) { A; if (cnd2) { B; flag = 0; } C;}
while (cnd1) { A; if (cnd2) { B; C; break; } C;}
Apr 18, 2023 Branch Prediction Strategies 14
Static Prediction Using Branch HistoryStatic Prediction Using Branch History
If x>10
True False
If x>20
True
False
If x>10
FalseTrue
If x>20 If x>20
True
False
True
False
Apr 18, 2023 Branch Prediction Strategies 15
Semi-static PredictorsSemi-static Predictors[Mc Farling & Hennessy ISCA´86, Fisher & Freudenberger ASPLOS´92][Mc Farling & Hennessy ISCA´86, Fisher & Freudenberger ASPLOS´92]
Use profile information from previous program runs Branches tend to behave in a fixed way Branches tend to behave in the same way across
different program executions
Performance metric: How close to a perfect static predictor
• Best direction to statically predict a branch Real static predictors
• Based on a past dataset / group of datasets
Apr 18, 2023 Branch Prediction Strategies 16
Semi-static Predictors (II)Semi-static Predictors (II)
50
60
70
80
90
100m
88ks
im gcc li
ijpe
g
perl
vort
ex
post
gres
perfectBTFNT
Apr 18, 2023 Branch Prediction Strategies 17
Bimodal Branch PredictorsBimodal Branch Predictors Dynamically store information about the
branch behaviour Branches tend to behave in a fixed way Branches tend to bevave in the same way across
program execution Index a Pattern History Table using the branch
address 1 bit: branch behaves as it did last time Saturating 2 bit counter: branch behaves as it
usually does Usually used as an extension to the BTB
Apr 18, 2023 Branch Prediction Strategies 18
Bimodal Predictors & BTBBimodal Predictors & BTB
Branch Address
Tag Taken Target Pred TypeFall Through
Apr 18, 2023 Branch Prediction Strategies 19
Bimodal Branch Predictors (II)Bimodal Branch Predictors (II)
70
75
80
85
90
95
100m
88ks
im gcc li
ijpe
g
perl
vort
ex
post
gres
bimodalsemi-staticBTFNT
Apr 18, 2023 Branch Prediction Strategies 20
Two-level Branch PredictorsTwo-level Branch Predictors[Pan, So & Rahmeh ASPLOS´92, Yeh & Patt ISCA´93][Pan, So & Rahmeh ASPLOS´92, Yeh & Patt ISCA´93]
A branch outcome depends on the outcomes of previous branches
First level: Branch History Registers (BHR) Global history / Branch correlation: past executions
of all branches Self history / Private history: past executions of the
same branch Second level: Pattern History Table (PHT)
Use first level information to index a table• Possibly XOR with the branch address [McFarling ´93]
PHT: Usually saturating 2 bit counters Also private, shared or global
Apr 18, 2023 Branch Prediction Strategies 21
Gshare Two-level PredictorGshare Two-level Predictor
Branch History
Branch Address
Index
PHT
Apr 18, 2023 Branch Prediction Strategies 22
PAp Two-level PredictorPAp Two-level Predictor
Branch Address
Branch History
PHT
Apr 18, 2023 Branch Prediction Strategies 23
Two-level Branch Predictors (II)Two-level Branch Predictors (II)
85
87
89
91
93
95
97
99m
88ks
im gcc li
ijpe
g
perl
vort
ex
post
gres
Gsharebimodalsemi-static
Apr 18, 2023 Branch Prediction Strategies 24
Improving Dynamic PredictorsImproving Dynamic Predictors Combining branch
predictors [McFarling WRL'93] Use two different branch
predictors• Access both in parallel
A third table determines which prediction to use
Using value prediction to predict branch outcomes [Gonzalez & Gonzalez PACT'99]
Combine a branch predictor with a value predictor
• Predict the branch inputs
• Compute branch outcome
Variable history length [Juan Sanjeevan & Navarro ISCA'98, Stark Evers & Patt ASPLOS'98]
Record accuracy of a given history length
• Define a time frame Use the best recorded
history length
Apr 18, 2023 Branch Prediction Strategies 25
Hybrid Branch PredictorsHybrid Branch PredictorsBranch Address
Predictor A:•static
•bimodal•PA{psg}
Predictor B:•GA{psg}•GshareMeta
Predictor:•bimodal
•two-level*
* does not replicatethe history register
Apr 18, 2023 Branch Prediction Strategies 26
Hybrid Branch Predictors (II)Hybrid Branch Predictors (II)
90
92
94
96
98
100m
88ks
im gcc li
ijpe
g
perl
vort
ex
post
gres
Gshare+Bimod (12K)
Gshare (16K)
Gshare (4K)
Apr 18, 2023 Branch Prediction Strategies 27
Multiple Branch PredictorsMultiple Branch Predictors Adapt existing branch predictors
Multi-bank BTB with bimodal predictor• Access many consecutive addresses Invalid predictions after the first taken branch
Adapted two-level predictor [Yeh,Marr & Patt SC'93]
• Fast sequential accesses to the same PHT Tree-like subgraph predictors [S.Dutta & M.Franklin
MICRO'95]
Predict which of the tree-like paths to follow Next trace predictor [Jacobson, Rottenberg & Smith MICRO'97]
Designed to work in conjuntion with the trace cache
Apr 18, 2023 Branch Prediction Strategies 28
Multi-bank Branch Target BufferMulti-bank Branch Target Buffer
Target
Pred
Tag
Type
=
=
==
Branch Address
Taken!Not taken
Apr 18, 2023 Branch Prediction Strategies 29
Adapted Gshare PredictorAdapted Gshare Predictor
Branch History
Branch Address
Index
PHT
P3P2P1
Apr 18, 2023 Branch Prediction Strategies 30
Adapted (fast) Gshare PredictorAdapted (fast) Gshare Predictor
Branch History
Branch Address
Index
PHT
P3P2P1
Apr 18, 2023 Branch Prediction Strategies 31
Tree-like Subgraph Predictor (I)Tree-like Subgraph Predictor (I)Subgraph Address A0
L0
A1
L1
A2
L2
A3
L3
A4
L4
A5
L5
A6
L6
T0 T1 T2 T3
Path 0 Attributes: {{A0,A1,A2},{L0,L1,L2},T0}
Apr 18, 2023 Branch Prediction Strategies 32
Tree-like Subgraph Predictor (II)Tree-like Subgraph Predictor (II)
Tag Path AttributesSubgraph history 2-bit counters
PathSelector
To FetchUnit
Subgraph Address
Apr 18, 2023 Branch Prediction Strategies 33
Next Trace PredictorNext Trace Predictor
Hashed ID Hashed ID Hashed ID
History register
IndexGeneration Next Trace Cnt
=
HashFunction
Next Trace Cnt Tag
Correlating table
Secondary table
Apr 18, 2023 Branch Prediction Strategies 35
BibliographyBibliography Static branch prediction
J.E.Smith. A study of branch prediction strategies. ISCA-8, 1981.
F.Mueller and D.B.Whalley. Avoiding unconditional jumps by code replication. PLDI, 1992.
T.Ball and J.R.Larus. Branch prediction for free. PLDI, 1993.
C.Young and M.D.Smith. Improving the accuracy of static branch prediction using branch correlation. ASPLOS-6, 1994.
F.Mueller and D.B.Whalley. Avoiding conditional branches by code replication. PLDI, 1995.
Apr 18, 2023 Branch Prediction Strategies 36
BibliographyBibliography Semi-static branch prediction
S.McFarling and J.Hennessy. Reducing the cost of branches. ISCA-13, 1986.
D.E.Wall. Predicting program behavior using real or estimated profiles. PLDI, 1991.
J.A.Fisher and S.M.Freudenberger. Predicting conditional branch directions from previous runs of a program. ASPLOS-5, 1992.
A.Krall. Improving semi-static branch prediction by code replication. PLDI, 1994.
B.Calder and D.Grunwald. Reducing branch costs via branch alignment. ASPLOS-6, 1994.
B.Calder, D.Grunwald, D.Lindsay, J.Martin, M.Mozer and B.Zorn. Corpus-based static branch prediction. PLDI, 1995.
Apr 18, 2023 Branch Prediction Strategies 37
BibliographyBibliography Bimodal predictors and BTBs
J.E.Smith. A study of branch prediction strategies. ISCA-8, 1981.
J.Lee and A.Smith. Branch prediction strategies and branch target buffer design. IEEE Computer 21(7). 1984.
S.McFarling and J.Hennessy. Reducing the cost of branches. ISCA-13, 1986.
T.Yeh and Y.N.Patt. A comprehensive instruction fetch mechanism for a processor supporting speculative execution. MICRO-25, 1992.
B.Calder and D.Grunwald. Fast & accurate instruction fetch and branch prediction. ISCA-21, 1994.
Apr 18, 2023 Branch Prediction Strategies 38
BibliographyBibliography Two-level branch predictors
S.Pan, K.So and J.Rahmeh. Improving the accuracy of dynamic branch prediction using branch correlation. ASPLOS-5, 1992.
T.Yeh and Y.N.Patt. Alternative implementations of two-level adaptive branch prediction. ISCA-19, 1992.
T.Yeh and Y.N.Patt. A comparison of dynamic branch predictors that use two levels of branch history. ISCA-20, 1993.
S.McFarling. Combining branch predictors. Technical note TN-36, DEC-WRL, 1993.
Apr 18, 2023 Branch Prediction Strategies 39
BibliographyBibliography Combining branch predictors
S.McFarling. Combining branch predictors. Technical note TN-36, DEC-WRL, 1993.
P.Y.Chang, E.Hao and Y.N.Patt. Alternative Implementations of hybrid branch predictors. MICRO-28, 1995.
Dynamic history length T.Juan, S.Sanjeevan and J.Navarro. Dynamic history
length fitting: a third level of adaptivity for branch prediction. ISCA-25, 1998.
J.Stark, M.Evers and Y.N.Patt. Variable length path branch prediction. ASPLOS-8, 1998.
Apr 18, 2023 Branch Prediction Strategies 40
BibliographyBibliography Novel branch predictors
C.C.Lee, I.Chen and T.Mudge. The bi-mode branch predictor. MICRO-30 1997.
E.Sprangle, R.Chappell, M.Alsup and Y.N.Patt. The agree predictor: a mechanism for reducing negative branch history interference. ISCA-24, 1997.
J.Gonzalez and A.Gonzalez. Control-flow speculation through value prediction for superscalar processors. PACT 1999.
Apr 18, 2023 Branch Prediction Strategies 41
BibliographyBibliography Multiple branch predictors
T.Yeh, D.Marr and Y.N.Patt. Increasing instruction fetch rate via multiple branch prediction and a branch address cache. SC-7, 1993.
S.Dutta and M.Franklin. Control flow prediction with tree-like subgraphs for superscalar processors. MICRO-28, 1995.
T.Conte, K.Menezes, P.Mills and B.Patel. Optimization of instruction fetch mechanisms for high issue rates. ISCA-22, 1995.
S.Wallance and N.Bagherzadeh. Multiple branch and block prediction. 1997.