Inserting a Pipeline Stage into Existing Logic

The previous article was about deciding where the stages go. This one is about actually cutting.

Timing closure has failed on a block that used to be fine at a lower clock. Nobody is asking you to redesign it; they want it faster without changing what it computes. In my experience this is the most common pipelining task you will get in a real job, and it is a narrower, more mechanical problem than designing a pipeline from scratch.

Step 1: find a legal cut point

Not every point in an expression is a valid place for a register. The cut has to be on a signal that, once registered, still lets the rest of the expression compute correctly, which usually means a point where a sub-result is fully formed.

// Original combinational block: too slow at the new target frequency
assign stage_result = (a + b) * (c - d) + e;

Two natural cut points: after (a + b) and (c - d) are computed, or after their product. Cutting in the middle of the multiply, with only some of the partial products summed, is not a legal cut unless you also carry the intermediate carry-save state through the register, as in Section 7.2. The safe cuts are where a complete value already exists.

Step 2: insert the register, split the expression

// Stage 1: compute both sub-expressions, register both results
reg [WIDTH-1:0] sum_ab, diff_cd;
reg [WIDTH-1:0] e_reg;   // e must also be registered, see step 3

always @(posedge clk) begin
    sum_ab  <= a + b;
    diff_cd <= c - d;
    e_reg   <= e;
end

// Stage 2: combine the registered sub-results
always @(posedge clk) begin
    stage_result <= sum_ab * diff_cd + e_reg;
end

The expression has not changed, it is still (a+b) * (c-d) + e, but it now takes two cycles instead of settling combinationally in one. Everything downstream of stage_result now has to expect it a cycle later, which is why this kind of change ripples outward and needs a valid signal alongside it in any design where a one-cycle shift is not automatically fine.

Step 3: the part everyone forgets

Look at e_reg. e is not combined with anything in stage 1; it only has to still be around, correctly aligned, when stage 2 runs. Forget to register it and stage 2 adds the current e to a product computed from operands that are a cycle old, quietly combining data from two different cycles.

This is the single most common bug introduced when retrofitting a stage: some signal that used to flow straight through the block is left unregistered because it “was not part of the computation being split”. It was never part of the computation, but it still has to cross the same boundary everything else just started crossing.

A checklist for any retrofit

1. Find the sub-expression that already produces a complete value at the intended cut.
2. Register that value. If it is more than one wire (the multiplier’s two rows in Section 7.2), register all the pieces together in the same stage.
3. Check every other signal that crosses the same boundary. Anything that used to flow straight through needs its own passthrough register, even if it had nothing to do with the arithmetic being split.
4. Update or add a valid signal through the new boundary if the output timing matters to anything downstream.
5. Re-run the existing testbench, changing only the expected timing. If the original testbench only checked values and never which cycle they arrived on, this is where that gap will hide a bug.

Point 5 deserves emphasis. A testbench that samples an output “eventually”, after an unspecified number of cycles or through a wait loop, will often report a pipelined block as passing even when a passthrough like e was left unregistered and every result is using misaligned data. Pipelining is exactly the change that turns a loosely timed testbench from good enough into actively misleading.

Practice Exercise

The block below computes a multiply followed by an optional clamp and has become the critical path. Insert one pipeline stage at a legal cut point, and make sure every signal, including the passthrough clamp_enable, survives the new boundary.

assign result = clamp_enable ? clamp(a * b, min_val, max_val) : (a * b);

reg [2*WIDTH-1:0] product_reg;
reg               clamp_enable_reg;
reg [WIDTH-1:0]   min_val_reg, max_val_reg;

always @(posedge clk) begin
    product_reg      <= a * b;         // computed twice in the original, now shared and registered
    clamp_enable_reg <= clamp_enable;  // passthrough, easy to miss
    min_val_reg      <= min_val;       // passthrough
    max_val_reg      <= max_val;       // passthrough
end

always @(posedge clk) begin
    result <= clamp_enable_reg ? clamp(product_reg, min_val_reg, max_val_reg) : product_reg;
end

Retrofitting a stage is mechanical once you know what to check: cut where a complete value exists, register it, and then audit every other signal crossing the same boundary, because it is always the one that “was not part of the real computation” that gets forgotten. In the next article we deal with what happens when the thing downstream cannot accept a new result every cycle: stalls, bubbles and backpressure.