On this page
A coding agent can run a test, change a line, run the test again, and still make no progress. The issue is not necessarily a lack of ideas. It may be that each attempt begins without a reliable record of what was tried, what the result actually showed, and which alternatives remain.
One useful remedy is to maintain a hypothesis tree outside the conversation. Each branch records a possible explanation and a discriminating experiment. The tree is a working map, not a promise that the agent will find the cause or that the real system is tree-shaped.
Start with an observable failure
Before listing causes, write down the expected behavior, the observed behavior, the smallest safe reproduction, and the version or environment in which it occurred. If the failure is intermittent, record that too: one passing run is not proof that a suspected cause is impossible.
Consider this invented example: a CSV export should contain three valid records, but the produced file has two when the input includes a blank line. We have a small fixture and a failing test. The initial tree might be:
Export contains 2 records; expected 3
├─ A. Parser drops a record
│ └─ Parse fixture alone: 3 records plus blank marker [tested]
├─ B. Export filtering drops a valid record
│ └─ Inspect counts immediately before and after filtering [open]
└─ C. Serializer omits a record
└─ Serialize 3 known records without the filter [open]
The result at A narrows the question; it does not prove the parser can never be responsible under other inputs. B and C are explicitly open, not silently ruled out. These results are illustrative, not an account of a measured incident.
Give each branch an experiment
A useful node needs enough information for someone else—or a later agent session—to continue without guessing:
| Field | Question it answers |
|---|---|
| Hypothesis | What specific behavior would explain the symptom? |
| Test and prediction | What will we do, and what different outcomes would support or weaken it? |
| Observation | What actually happened, under which inputs and version? |
| Status and scope | Is it open, supported, weakened, or inconclusive? What did the test not cover? |
| Next step | Which safe experiment should be attempted next? |
For branch B, “check filtering” is too vague. A better experiment is: “With the same fixture, count valid records immediately before and after the filter; if the count falls from three to two, inspect that transition.” If the input or environment changes, record it rather than attributing the difference to the hypothesis.
Keep observations separate from interpretations. A log entry might say “the filter received three and returned two” and then “this supports B for this fixture.” It should not say “the filter is always broken.” A failed command, an unavailable dependency, or a flaky test is inconclusive, not evidence that the hypothesis is false.
Breadth first, depth first, or neither?
Breadth-first search (BFS) samples alternatives near the current question before going deeper. In the example, inexpensive checks of parsing, filtering, and serialization may prevent a long investigation of the wrong component. It is useful when several plausible explanations have cheap, distinct tests.
Depth-first search (DFS) follows a promising branch into more specific questions. If the filter is where the count changes, inspect its handling of blank lines, then make a minimal change and rerun the reproduction. It is useful when a result sharply narrows the search.
Real investigations need not use strict BFS or DFS. Prefer a low-risk, high-information experiment; switch branches when evidence changes or a time budget expires. Record a link if two branches depend on the same component: the underlying dependencies can form a graph, even if the notebook displays them as a tree.
Save the frontier, not a transcript
At a pause or context reset, preserve a compact checkpoint:
Goal: export all 3 fixture records.
Observed: export produces 2 when a blank line is present.
Tested: parser alone returns 3 records and a blank marker for this fixture.
Open: compare counts around filter; test serializer independently.
Next: measure the filter transition using the same fixture.
Limits: one fixture; no conclusion about other inputs.
Add references to reproducible tests or approved artifacts when available, plus any changes made during the investigation. Recheck the current version and rerun the reproduction after resuming: code and environments can change while the notebook sits idle. Store the actual checkpoint wherever your project’s privacy rules permit, and remove personal data, credentials, and sensitive logs before sharing it.
When is the investigation done?
First reproduce the failure and make the smallest justified fix. Then rerun the original failing test, relevant regression tests, and a normal-path case. For an intermittent issue, use a repeated or targeted test and report the remaining uncertainty. If the experiment cannot distinguish two causes, leave both open rather than declaring victory.
This is a practical application of established ideas, not a new search algorithm. Google’s SRE troubleshooting guidance recommends hypotheses, controlled tests, and clear notes; git bisect searches a different space—revision history—when a reproducible regression can be classified. The Tree of Thoughts paper explores branching language-model reasoning, but its reported experiments do not establish that this particular debugging notebook improves agent performance. The notebook’s modest purpose is to make the evidence and remaining work inspectable.