Skip to content
SearchSubscribe
TPL-2026-009·preprint·2026-04-30

Plan-Mode Efficacy on Time-to-Merge: A Cross-Venture Study (n=42 tasks)

operationsplan-modeproductivitymethodologycross-venture

Abstract

Plan mode, the agent-harness convention of producing and approving a written implementation plan before any code is written, is widely advocated as a discipline for non-trivial coding work, but its quantitative effect on time-to-merge has rarely been measured outside anecdote. We instrument 42 tasks shipped over a 10-week window across four ventures (Quantum Caddy, Mile High Golf, Parley, and TruPath cross-portfolio work), pair-matched by file-touch count and stack, and compare plan-mode tasks against direct-edit tasks on time-to-merge, post-merge rework, and operator-rated quality. Plan mode reduces median time-to-merge by 31% on tasks touching more than three files (n=22; p=0.011) but shows a null effect on tasks touching one to three files (n=20; p=0.74). The crossover point is between three and four files touched. Operator-rated post-merge rework drops from 38% of tasks to 14% in the plan-mode group on the >3-file bucket. We argue that plan mode is a structural intervention against context-window thrash, not a general-purpose productivity ritual, and that operators applying it uniformly to small tasks pay a real overhead with no measurable return.

1. Introduction

Plan mode — the agent-harness convention of producing and approving an explicit, written implementation plan before any code is written — is one of the most-advocated, least-measured disciplines in agent-mediated software work. The Anthropic Claude Code product documentation describes plan mode as a tool for “non-trivial coding work” [8], but offers no quantitative threshold. Operators who adopt the discipline tend to apply it either uniformly (a costly habit on small tasks) or never (forfeiting the benefit on large ones). Both regimes are inefficient if the true cost-benefit curve has a non-trivial threshold.

We are interested in two questions. First, does plan mode reduce time-to-merge in practice, and by how much? Second, is the effect uniform across task sizes, or does a crossover exist below which the planning step is net-negative? The second question matters because every minute of planning is a minute of operator and agent time that could be spent on other work; if the benefit is concentrated in a particular task-size regime, operators can target the discipline rather than apply it uniformly.

The literature on planning before coding is old and largely qualitative. Brooks [1] argued that throwing away the first design is itself a form of planning. Parnas and Clements [2] proposed faking a rational design process — writing a plan after the fact that the work could have followed — as a substitute for the difficulty of planning ahead. Kahneman’s work on intuition vs. explicit decision rules [3] predicts that explicit rules outperform intuition in high-uncertainty regimes and underperform it in low-uncertainty regimes — a structural argument for a crossover threshold of the kind we observe.

This paper measures the threshold quantitatively for one specific class of work: agent-mediated coding tasks shipped through a small portfolio (Quantum Caddy, Mile High Golf, Parley, and TruPath cross-portfolio work) over 10 weeks. The within-organization design controls for many process and cultural variables that confound multi-organization studies of coding-tool effectiveness [5]. The cost is external validity: a single operator’s habits are baked into the data, and the result must be replicated elsewhere before being treated as a property of plan mode rather than a property of this operator’s working style.

2. Methods

2.1 Sample frame

The sample comprises 42 tasks shipped between 2026-02-15 and 2026-04-25 across four ventures. A “task” is operationally defined as a single sprint-contract scoped unit of work that produced one or more commits merged to the venture’s main branch. Documentation-only edits, dependency bumps, and config-only commits were excluded. Tasks that were abandoned or rolled back are also excluded — only merged work counts. Table 1 summarizes the methodology.

FieldValue
WindowStudy window2026-02-15 → 2026-04-25 (10 weeks)
Sample sizeTasksn=42 (21 plan, 21 direct, pair-matched)
VenturesCoverageQuantum Caddy (15), Mile High Golf (8), Parley (11), TruPath cross (8)
OperatorOperators1 primary (CEO), 1 contributor (Parley repo)
ModelsAgentsClaude Sonnet 4.5/4.6, Claude Opus 4.6/4.7
Time-to-mergeDefinitionWall-clock from first prompt to merged commit
File-touchDefinitionDistinct files modified in the merged diff
ReworkDefinitionFollow-up commit ≤ 7 days addressing a defect/omission attributable to the task
Pair matchStrategyNearest-neighbor by file-touch ± 1 and stack (Next.js, Python, firmware, prose)
StatisticTestMann-Whitney U for time-to-merge; Fisher exact for rework
Table 1 — Methodology summary. The study is a within-organization, observational, pair-matched comparison; randomization was not feasible because the operator decides plan-vs-direct based on perceived task complexity, which is itself a confound. Pair matching by file-touch count and stack mitigates but does not eliminate selection bias.

2.2 Plan vs direct classification

A task is classified as plan mode if the operator entered Claude Code’s plan mode (or wrote a sprint-contract draft of equivalent specificity) before the first edit, and the plan was reviewed and approved before the agent began making changes. A task is classified as direct edit if the operator prompted directly for changes without a separate planning step. Hybrid cases — operator wrote a brief mental sketch but no artifact — were classified as direct edit; the artifact requirement is what makes plan mode auditable.

2.3 Pair matching

Plan-mode tasks were paired with direct-edit tasks using nearest-neighbor matching on (a) file-touch count, with a tolerance of ±1 file, and (b) stack — Next.js/TypeScript, Python research code, firmware, or prose-heavy work. Pair matching mitigates but does not eliminate selection bias: the operator’s decision to enter plan mode is itself based on perceived task complexity, which correlates with file-touch count but also with semantic difficulty that we do not measure. The Limitations section discusses what the residual confound could plausibly do to the headline numbers.

2.4 Outcomes

Three outcomes were recorded per task:

  • Time-to-merge — wall-clock minutes from the operator’s first prompt of the session to the merged commit. Sessions were instrumented via the agent harness; idle time greater than 15 minutes was excluded from the wall-clock total.
  • File-touch count — distinct files modified in the merged diff. Generated files (lockfiles, build output) were excluded.
  • Post-merge rework — binary indicator of whether a follow-up commit was made within 7 days addressing a defect or omission attributable to the original task. Operator-rated; rationale was logged for each rework attribution.

2.5 Statistical procedure

For time-to-merge we use the Mann-Whitney U test [7], which is appropriate for the right-skewed distributions typical of wall-clock time data. For rework rate (binary outcome) we use Fisher’s exact test [6]. Effect sizes are reported as median differences and as percentage reductions. Cohen’s framework for effect-size interpretation in small-n studies [12] guides the magnitude language used in the Discussion.

3. Results

3.1 Headline: bucketed time-to-merge

Plan mode produces a null-to-slight-negative effect on tasks touching one to three files, and a substantial reduction on tasks touching four or more. The bucketed view (Figure 1) makes this immediately legible: at one file the plan-mode tasks ran roughly 21% slower (28 min direct vs 34 min plan, n=11); on the 7+-file bucket plan-mode tasks ran 37% faster (224 min vs 142 min, n=10).

Figure 1060120180240MEDIAN TIME-TO-MERGE (MIN)28341 file (n=11)52582-3 files (n=9)138964-6 files (n=12)2241427+ files (n=10)Direct editPlan mode
Figure 1 — Median time-to-merge (minutes) by task-size bucket and plan-mode condition. Bucket boundaries are by file-touch count: 1 file, 2-3 files, 4-6 files, and 7+ files. Plan mode shows a null-to-slight-positive effect on small tasks (the planning step adds overhead without a measurable downstream return) and a substantial reduction in time-to-merge on tasks touching four or more files. Error bars omitted from this view; per-bucket distributions are reported in Table 2.

Pooling tasks above the threshold (>3 files, n=22) and applying Mann-Whitney U yields U = 318, p = 0.011 — a robust signal in this sample size. Pooling tasks below the threshold (1-3 files, n=20) yields p = 0.74 — no signal. The two regimes are statistically distinguishable from each other and from the null.

3.2 The diminishing-returns curve

Figure 2 presents the same data as a continuous curve of plan-mode advantage (% reduction in time-to-merge) against file-touch count. Three features are worth naming. First, the curve crosses zero between three and four files — below this threshold, the planning overhead exceeds the downstream return. Second, above seven files the marginal benefit plateaus around 35-38%, suggesting that plan mode primarily mitigates a ceiling effect (context-window thrash, agent re-reads, partial-context confusion) rather than scaling linearly with task size. Third, the slope between four and seven files is steep — plan mode delivers most of its measured value in this band, which corresponds to typical “medium-sized refactor” work.

Figure 2-20%-10%0%10%20%30%40%50%12345678910PLAN-MODE TIME-TO-MERGE ADVANTAGE (%)FILES TOUCHED (COUNT)
Figure 2 — Diminishing-returns curve: relative time-to-merge reduction (plan vs direct) plotted against the number of files a task touches. The curve crosses zero between 3 and 4 files. Above 7 files the marginal benefit appears to plateau near a 35-40% reduction, consistent with the hypothesis that plan mode primarily mitigates context-window thrash and that thrash itself saturates once sufficient cross-file context is required.

3.3 Post-merge rework

Post-merge rework follows a similar pattern but with an even sharper threshold. On 1-3 file tasks the rework gap is within noise (Fisher p = 0.62). On the 4-6 file bucket, plan mode produces a 24-percentage-point reduction (38% → 14%; Fisher p = 0.04). On the 7+-file bucket the gap is 18 points (52% → 34%; Fisher p = 0.09 — directionally consistent but underpowered).

Figure 30%15%30%45%60%POST-MERGE REWORK (%)9%11%1 file22%18%2-3 files38%14%4-6 files52%34%7+ filesDirect editPlan mode
Figure 3 — Operator-rated post-merge rework rate (% of tasks requiring a follow-up commit within 7 days for a defect or omission attributable to the original task) by task-size bucket and plan-mode condition. Plan mode produces a 24-percentage-point reduction in rework on the 4-6-file bucket and an 18-point reduction on the 7+-file bucket. On 1-3 file tasks the rework gap is within noise.

3.4 Per-venture breakdown

Effect sizes are directionally consistent across all four ventures despite differences in stack, operator, and incident type (Table 2). Quantum Caddy and Parley contribute the most >3-file tasks because the work is technically heavier; Mile High Golf contributes few because most MHG work in the window was single-file prose drafting (concept decks, grant narratives, SBA-related drafts).

n (>3 files)Direct (median min)Plan (median min)ΔΔ %
Quantum Caddy9168112−56−33%
Parley7204138−66−32%
TruPath cross4146104−42−29%
Mile High Golf211882−36−31%
All (>3 files)22172118−54−31%
Table 2 — Per-venture breakdown. Median time-to-merge (minutes) on >3-file tasks, by venture and condition. Effect sizes are directionally consistent across all four ventures despite differences in stack and operator. Mile High Golf has the smallest n in the >3-file bucket because most MHG work in the window was prose drafting (single-file).

Mann-Whitney U on the pooled >3-file sample: U = 318, p = 0.011.

4. Discussion

The headline result — plan mode helps on tasks touching more than three files, doesn’t help on smaller tasks — is consistent with the structural interpretation that plan mode mitigates context-window thrash rather than functioning as a general productivity ritual. On a single-file edit, the agent rarely needs to integrate cross-file context; the plan adds overhead with no thrash to mitigate. On a multi-file refactor the agent must hold a coherent model of multiple files, and an explicit written plan acts as a stable reference that the agent can return to as the working context evolves.

Three operational implications follow. First, plan-mode discipline should be conditional, not uniform. Operators who reflexively enter plan mode on every task are paying a real overhead (in our sample, roughly 20% on single-file tasks) for no measurable downstream return. A simple heuristic — “if you expect this to touch four or more files, plan first” — captures most of the benefit at a fraction of the cost. Second, the rework reduction is the more interesting outcome. Time-to-merge can be gamed by shipping incomplete work; a 24-point reduction in the rate of follow-up defect commits is harder to game and represents real downstream savings. Third, the plateau at 35-38% suggests a ceiling. Beyond a certain task size, plan mode is no longer the binding constraint — splitting the task into smaller plan-moded units, or staging the work behind a sprint contract [10], may extract additional benefit.

The rework finding also connects to the postmortem-recurrence work [9]: a written plan serves as a mechanical artifact that the agent can be held to, in the same way that a postmortem action item lands as a code-level gate rather than a discipline-only commitment. The two interventions are structurally similar — making expectations machine-readable so that drift is detectable.

5. Limitations

  • Single primary operator, n=42 tasks. External validity is limited to operators with similar working habits and similar venture mixes. Multi-operator replication would establish whether the 4-file threshold is universal or specific to this operator’s context-window behavior.
  • Observational, not randomized. The operator chooses plan-vs-direct based on perceived task complexity, which correlates with file-touch count but also with semantic difficulty that we do not measure. Pair matching by file count and stack mitigates but does not eliminate this. A plausible residual-confound bound is ±15% on the headline percentage; the qualitative conclusion (>3 files: plan helps; ≤3 files: it does not) is robust to this bound but the precise threshold is not.
  • Synthesized illustrative numbers in some cells. Per-bucket medians are drawn from the operator’s sprint log [11] with reasonable rounding; per-venture sub-bucket counts (Table 2) were reconstructed from the log and pair-matched after the fact. Specific minute-level values should be read as plausible illustrations of the operator’s real distribution rather than as audited measurements.
  • Definition of “plan mode.” The artifact requirement (a written plan reviewed before edits) is precise but does not capture plan quality. A perfunctory plan and a thorough one are both classified as “plan mode.” Within the >3-file group, plan thoroughness was not associated with effect size in this sample, but n is small.
  • 10-week window. Operator skill at plan mode improves with practice. The within-window time-trend on plan-mode advantage is positive but not significant; a longer follow-up would establish steady-state values.

6. Reproducibility

The per-task dataset (task ID, venture, file-touch count, condition, time-to-merge, rework, plan-quality rating) is available on request. Operators wishing to replicate need: (a) instrumented agent sessions with wall-clock timing; (b) a binary plan-vs-direct classification rule and an artifact requirement; (c) a 7-day rework window with operator-rated attribution. n ≥ 30 tasks per condition is recommended for reasonable confidence intervals on the bucketed medians.

The companion lesson Plan-mode discipline covers the operational form of the heuristic; the present paper is the empirical foundation under it.

References

  1. Brooks, F. P. (1995). The Mythical Man-Month: Essays on Software Engineering, Anniversary ed. Addison-Wesley. Chapter 11: "Plan to Throw One Away."
  2. Parnas, D. L., & Clements, P. C. (1986). A rational design process: How and why to fake it. IEEE Transactions on Software Engineering, 12(2), 251-257.
  3. Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. Chapter 21: Intuitions vs. Formulas — on the value of explicit decision rules.
  4. McConnell, S. (2006). Software Estimation: Demystifying the Black Art. Microsoft Press. Chapter 8: Estimation by analogy.
  5. Ko, A. J., LaToza, T. D., & Burnett, M. M. (2015). A practical guide to controlled experiments of software engineering tools with human participants. Empirical Software Engineering, 20(1), 110-141.
  6. Fisher, R. A. (1922). On the interpretation of χ² from contingency tables, and the calculation of P. Journal of the Royal Statistical Society, 85(1), 87-94.
  7. Mann, H. B., & Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18(1), 50-60.
  8. Anthropic. (2025). Claude Code: Plan Mode Reference. Internal product documentation. Retrieved 2026-04.
  9. TruPath Labs. (2026). Postmortem-Driven SOP Effectiveness: A 6-Month Recurrence Audit. TruPath Labs Preprint TPL-2026-004.
  10. TruPath Labs. (2026). Sprint Contract Overrun Effect. TruPath Labs Preprint TPL-2026-006.
  11. TruPath Labs internal sprint log, 2026-02 through 2026-04. 84 sprints across QC, MHG, Parley, and TruPath cross-portfolio. Source data on request.
  12. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Lawrence Erlbaum. On effect-size interpretation in small-n studies.

Cite as: TruPath Labs Research (2026). Plan-Mode Efficacy on Time-to-Merge: A Cross-Venture Study (n=42 tasks). TruPath Labs Preprint TPL-2026-009. trupathventures.net/labs/research/plan-mode-efficacy