Chapter 4 — The Processor — 1 Lecture 7 Pipelining

Chapter 4 — The Processor — 2 Pipelining Analogy Pipelining is an implementation technique in which multiple instructions are overlapped in execution.  nearly universal. Non-pipelined approach to laundry: 1. Place one dirty load of clothes in the washer. 2. When the washer is finished, place the wet load in the dryer. 3. When the dryer is finished, place the dry load on a table and fold. 4. When folding is finished, ask your roommate to put the clothes away. §4.5 An Overview of Pipelining

Chapter 4 — The Processor — 3 Pipelining Analogy Pipelined laundry: overlapping execution Parallelism improves performance §4.5 An Overview of Pipelining Four loads: Speedup = 8/3.5 = 2.3 Non-stop: Speedup = 2n/0.5n ≈ 4 = number of stages

Chapter 4 — The Processor — 4 Pipelining All steps in pipelining are operating concurrently. Improves throughput of our laundry system. Not decrease the time to complete one load of laundry, but the improvement in throughput decreases the total time to complete the work.

Chapter 4 — The Processor — 5 MIPS Pipeline Five stages, one step per stage 1.IF: Instruction fetch from memory 2.ID: Instruction decode & register read 3.EX: Execute operation or calculate address 4.MEM: Access memory operand 5.WB: Write result back to register

Chapter 4 — The Processor — 6 Pipeline Performance Assume time for stages is 100ps for register read or write 200ps for other stages Compare pipelined datapath with single-cycle datapath InstrInstr fetchRegister read ALU opMemory access Register write Total time lw200ps100 ps200ps 100 ps800ps sw200ps100 ps200ps 700ps R-format200ps100 ps200ps100 ps600ps beq200ps100 ps200ps500ps

Chapter 4 — The Processor — 7 Pipeline Performance (2400 vs 1400 ps) Single-cycle (T c = 800ps) Pipelined (T c = 200ps)

Chapter 4 — The Processor — 8 Pipeline Speedup If all stages are balanced i.e., all take the same time Time between instructions pipelined = Time between instructions nonpipelined Number of stages If not balanced, speedup is less Speedup due to increased throughput Latency (time for each instruction) does not decrease

Chapter 4 — The Processor — 9 Pipelining and ISA Design MIPS ISA designed for pipelining All instructions are 32-bits Easier to fetch and decode in one cycle c.f. x86: 1- to 17-byte instructions Few and regular instruction formats Can decode and read registers in one step Load/store addressing Can calculate address in 3 rd stage, access memory in 4 th stage Alignment of memory operands Memory access takes only one cycle

Chapter 4 — The Processor — 10 Hazards Situations that prevent starting the next instruction in the next cycle Structure hazards A required resource is busy Data hazard Need to wait for previous instruction to complete its data read/write Control hazard Deciding on control action depends on previous instruction

Chapter 4 — The Processor — 11 Structure Hazards A structural hazard in the laundry room would occur if we used a washer-dryer combination instead of a separate washer and dryer, or if our roommate was busy doing something else and wouldn’t put clothes away. Our carefully scheduled pipeline plans would then be foiled.

Chapter 4 — The Processor — 12 Structure Hazards Conflict for use of a resource In MIPS pipeline with a single memory Load/store requires data access Instruction fetch would have to stall for that cycle Would cause a pipeline “bubble” Hence, pipelined datapaths require separate instruction/data memories Or separate instruction/data caches

Chapter 4 — The Processor — 13 Data Hazards Suppose you found a sock at the folding station for which no match existed. One possible strategy is to run down to your room and search through your clothes bureau to see if you can find the match. Obviously, while you are doing the search, loads that have completed drying and are ready to fold and those that have finished washing and are ready to dry must wait.

Chapter 4 — The Processor — 14 Data Hazards An instruction depends on completion of data access by a previous instruction add$s0, $t0, $t1 sub$t2, $s0, $t3

Chapter 4 — The Processor — 15 Forwarding (aka Bypassing) Use result when it is computed Don’t wait for it to be stored in a register Requires extra connections in the datapath

Chapter 4 — The Processor — 16 Load-Use Data Hazard Can’t always avoid stalls by forwarding If value not computed when needed Can’t forward backward in time!

Chapter 4 — The Processor — 17 Code Scheduling to Avoid Stalls Reorder code to avoid use of load result in the next instruction C code for A = B + E; C = B + F; lw$t1, 0($t0) lw$t2, 4($t0) add$t3, $t1, $t2 sw$t3, 12($t0) lw$t4, 8($t0) add$t5, $t1, $t4 sw$t5, 16($t0) stall lw$t1, 0($t0) lw$t2, 4($t0) lw$t4, 8($t0) add$t3, $t1, $t2 sw$t3, 12($t0) add$t5, $t1, $t4 sw$t5, 16($t0) 11 cycles13 cycles

Chapter 4 — The Processor — 18 Control Hazards Branch determines flow of control Fetching next instruction depends on branch outcome Pipeline can’t always fetch correct instruction Still working on ID stage of branch In MIPS pipeline Need to compare registers and compute target early in the pipeline Add hardware to do it in ID stage

Chapter 4 — The Processor — 19 Stall on Branch Wait until branch outcome determined before fetching next instruction

Chapter 4 — The Processor — 20 Branch Prediction Longer pipelines can’t readily determine branch outcome early Stall penalty becomes unacceptable Predict outcome of branch Only stall if prediction is wrong In MIPS pipeline Can predict branches not taken Fetch instruction after branch, with no delay

Chapter 4 — The Processor — 21 MIPS with Predict Not Taken Prediction correct Prediction incorrect

Chapter 4 — The Processor — 22 More-Realistic Branch Prediction Static branch prediction Based on typical branch behavior Example: loop and if-statement branches Predict backward branches taken Predict forward branches not taken Dynamic branch prediction Hardware measures actual branch behavior e.g., record recent history of each branch Assume future behavior will continue the trend When wrong, stall while re-fetching, and update history

Chapter 4 — The Processor — 23 Pipeline Summary Pipelining improves performance by increasing instruction throughput Executes multiple instructions in parallel Each instruction has the same latency Subject to hazards Structure, data, control Instruction set design affects complexity of pipeline implementation The BIG Picture