A Systematic Debugging Method

The last four articles each covered one class of bug. This one is the method that uses them.

Module 6 gave us the tools: self-checking testbenches, waveform analysis, and the general shape of “simulation first, then waveform, then re-read the RTL”. This article is the method that converges: an ordered sequence, with the bug classes from this section serving as a diagnostic checklist rather than four separate facts to remember.

The sequence

1. Confirm the failure is real and reproducible. Before spending an hour, re-run the same test and confirm it fails the same way. A failure that only sometimes appears is itself a clue: it points at the timing-sensitive classes from this section (reset domain problems, X from an uninitialised register that depends on the simulator’s seed) rather than at a deterministic logic error.

2. Find the first moment something went wrong, not the moment you noticed. This is the single most important discipline in the method, and it is the X trace-back technique generalised: a wrong output at 500 ns very often traces to a root cause at 50 ns, with 450 ns of correctly propagating a problem in between. Open the waveform and scan backward from the symptom, not forward from the start.

3. Classify the earliest anomaly against the checklist, in order:

   • Is the signal reading X? X-propagation (first article). Trace it back to an uninitialised register, an unconnected port or an explicit 'x.
   • Is a signal holding an old value when you expected it to update, with no clock edge involved? Possibly an unintended latch (second article). Check the driving always block for missing defaults.
   • Does the failure correlate with reset release, or only appear near power-up? Reset domain problem (third article). Check cross-domain reset synchronization and inter-module reset assumptions.
   • Is the signal X specifically because two sources are driving it? Multi-driver conflict (fourth article). Find both drivers and check whether their conditions can overlap.
   • None of the above? Probably a functional logic error rather than one of these classes. Re-read the RTL against the specification, the Module 6 sequence.

4. Form a specific, falsifiable hypothesis before touching the RTL. “Signal X is X because register Y is missing a reset” is falsifiable: you can check Y’s reset logic and confirm or deny it in minutes. “Something is wrong with the reset logic somewhere” is not, and chasing a vague hypothesis is the biggest time sink in real debugging.

5. Fix the root cause, not the symptom. The warning from the X article: patching a downstream symptom (clearing an X where it surfaces, adding an arbitrary default that makes one test pass) hides the bug until the next test exercises it differently.

6. Re-run the original failing test, then the full regression. Confirm the failure is gone and that you have not introduced another, which is what every self-checking testbench from Module 6 was built to support.

Worked example: a failing arbiter

A two-master bus arbiter’s testbench reports: FAIL at time 340ns: expected grant=2'b01, got grant=2'bxx.

Steps 1 and 2: the failure reproduces identically on re-run, so it is not timing-sensitive, an early sign this is a deterministic bug rather than a reset-domain or X-race issue. Scanning backward from 340 ns, grant first reads X at 335 ns, five nanoseconds before the check, so the anomaly predates the moment it was caught by a small but real margin.

Step 3: grant reading X suggests X-propagation or a multi-driver conflict, the two classes that produce X on a signal that was a clean 0 or 1 the cycle before. Tracing grant’s drivers finds two assign statements, one in the arbiter’s priority logic and one in a debug-override block added later for testing, both targeting grant.

Step 4: the hypothesis: “the debug override’s enable condition and the normal priority logic’s condition are both true at 335 ns, creating a multi-driver conflict, the fourth article’s case”. Directly checkable by inspecting both conditions at that time.

Step 5: the waveform at 335 ns shows both req[0] in the priority logic and debug_force_grant in the override high at the same time. Confirmed.

Step 6: the fix is not clearing the X on grant. It is restructuring the override to take explicit priority through a mux, the fix from the multi-driver article’s exercise, so there is a single driver: assign grant = debug_force_grant ? debug_grant_value : normal_grant;. The debug path now takes precedence structurally, not coincidentally.

Step 7: the original test now shows grant clean throughout, and the full regression is re-run to confirm nothing else depended on the previous, broken behaviour.

This sequence, reproduce, find the first anomaly, classify, hypothesise, confirm, fix the root cause, regress, is essentially universal in hardware debugging, from one module’s testbench failure to a post-silicon bench chasing an intermittent chip-level fault. The tools change (a lab bench has an oscilloscope and a logic analyser instead of a waveform viewer) but the reasoning, and the discipline of finding the first anomaly rather than reacting to the most visible one, stays the same.

Practice Exercise

A testbench reports an FSM stuck permanently in an unexpected state, with no X anywhere in the design. Which of this section’s bug classes does the absence of X rule out, and which remain worth checking first?

Debugging converges into one sequence: reproduce, find the first anomaly rather than the most visible symptom, classify against a known checklist, form a falsifiable hypothesis, confirm it against the RTL, fix the root cause, and re-run both the test and the regression. That closes Module 8 and with it the four qualities, style, synthesis awareness, timing literacy and debugging method, that separate RTL that works from RTL a team can ship. In the next module we turn to the language itself one more time: SystemVerilog, which extends everything built in classic Verilog with better types, better verification constructs, and assertions that check the design’s behaviour directly.