Recursion of Thought
Some problems cannot be solved at one level of depth. Recursion of Thought applies divide-and-conquer recursively — decomposing sub-problems into their own sub-problems until reaching base cases simple enough to solve directly.
Introduced: Recursion of Thought was published in 2023 by Lee et al. The key insight is that context windows are limited, so when a problem is too complex for a single pass, RoT uses a special “going-up” token to pause the current problem, recurse into a sub-problem in a fresh context, solve it, and return the result to the parent level. This mirrors how recursive functions in programming call themselves on smaller inputs until reaching a base case, then unwind the call stack to assemble the final answer.
Modern LLM Status: RoT’s recursive decomposition principle has directly influenced agentic AI architectures where parent agents spawn child agents for sub-tasks. Modern frameworks like AutoGen and CrewAI implement recursive delegation patterns that echo RoT’s design. However, RoT as a pure prompting technique remains research-stage — it requires programmatic orchestration to manage the recursion stack across multiple context windows. Understanding RoT is valuable for designing hierarchical agent systems and recognizing the limits of single-context reasoning.
When Sub-Problems Have Sub-Problems
Standard decomposition frameworks like Least-to-Most break a problem into parts and solve each part. But what happens when one of those parts is itself too complex to solve directly? Flat decomposition hits a ceiling — it assumes one level of breakdown is enough. For genuinely complex problems, it is not.
Recursion of Thought eliminates this ceiling. Instead of decomposing once, RoT applies the same decomposition strategy at every level. If a sub-problem is still too hard, decompose it again. And again. Each level follows the identical pattern: check if the problem is a base case, and if not, break it down further. This creates a tree of reasoning where the leaves are simple, solvable problems and the internal nodes are combination points where results flow back upward.
The critical innovation is the “going-up” token — a mechanism that allows the model to pause its current reasoning, open a new context for a sub-problem, solve it there, and return the result to continue the parent computation. This overcomes the fundamental limitation of fixed context windows by distributing reasoning across multiple contexts organized in a recursive call stack.
In computer science, recursion is one of the most powerful problem-solving paradigms. A recursive function solves a problem by: (1) checking the base case — if the input is small enough, return the answer directly; (2) otherwise, breaking the input into smaller pieces, calling itself on each piece, and combining the results.
RoT applies this exact pattern to language model reasoning. The “function” is the decomposition prompt itself, the “input” is the problem, the “base case” is a question simple enough for the model to answer in one step, and the “recursive call” is spawning a new context to handle a sub-problem. The result is a reasoning tree of arbitrary depth, limited only by the complexity of the original problem.
Why It Works
Adaptive Depth
Recursion depth automatically matches problem complexity. Simple branches resolve at level one while complex branches recurse deeper — no need to predict the required depth in advance. The reasoning tree grows exactly as deep as the problem demands.
Context Isolation
Each sub-problem gets a clean, dedicated context free from the noise of the parent problem. By spawning a new reasoning window for each recursive call, the model avoids context pollution and can focus entirely on the sub-task at hand.
Compositional Reasoning
Solutions compose naturally from the bottom up. Base-case answers flow upward through the recursion tree, combining at each level into progressively more complete answers. The final result is assembled from verified building blocks rather than generated in a single uncertain pass.
The Recursive Reasoning Process
Five steps from complex problem to assembled solution
Check if Base Case
Before doing any decomposition, first assess whether the current problem is simple enough to solve directly. A base case is a question that can be answered in a single reasoning step without further breakdown. If it is a base case, solve it immediately and return the result.
"What is 7 times 8?" — This is a base case. Answer directly: 56. No decomposition needed.
Decompose into Sub-Problems
If the problem is not a base case, identify the logical sub-problems it can be broken into. Each sub-problem should be a self-contained question whose answer contributes to solving the parent problem. The decomposition should produce sub-problems that are strictly simpler than the original.
"Summarize this 50-page report" decomposes into: "Summarize Chapter 1," "Summarize Chapter 2," ... "Summarize Chapter 10." Each chapter summary is a sub-problem.
Recurse into Each Sub-Problem
For each sub-problem, open a new context and apply the same process from Step 1. If the sub-problem is a base case, solve it. If it is still too complex, decompose it again. This is the recursive step — the same strategy applied at a deeper level. The “going-up” token signals when to pause the parent and descend into a child context.
"Summarize Chapter 3" is still too long for one pass. Decompose further: "Summarize Section 3.1," "Summarize Section 3.2," "Summarize Section 3.3." Each section is now a base case — short enough to summarize directly.
Solve Base Cases Directly
At the deepest level of recursion, every sub-problem is a base case. Solve each one directly using straightforward reasoning. These are the “leaf nodes” of the reasoning tree — concrete answers that require no further decomposition.
"Summarize Section 3.1" — a 2-page section on methodology. Base case reached. Output: "Section 3.1 describes the survey methodology, including a sample of 500 participants across three age groups."
Combine Results Bottom-Up
Once all base cases are solved, unwind the recursion. Combine section summaries into chapter summaries, combine chapter summaries into the final report summary. Each level merges results from its children into a coherent answer for its parent, flowing upward until the original question is answered.
Section summaries merge into "Chapter 3 Summary." All chapter summaries merge into "Full Report Summary." The original problem is solved by composing the answers bottom-up through the recursion tree.
See the Difference
Why adaptive depth matters for complex problems
Flat Decomposition
One level of breakdown. The original problem is split into parts, and each part is solved directly. Fixed depth is decided in advance, regardless of actual problem complexity.
If a sub-problem is too large for a single pass, flat decomposition has no recourse. Example: “Summarize this book” splits into chapters — but an 80-page chapter still cannot be summarized in one pass.
Recursive Decomposition
Multi-level, adaptive breakdown. Sub-problems are decomposed further whenever they remain too complex. Simple branches resolve at level one while complex branches recurse deeper.
By spawning new contexts for sub-problems, each reasoning step operates within manageable scope. Example: “Summarize this book” becomes chapters, then sections, then sub-sections — recursing until every piece fits a single context.
Recursion of Thought in Action
See how recursive decomposition solves problems that flat approaches cannot handle
"Summarize this 200-page technical report on climate adaptation strategies across 12 regions, including policy recommendations and implementation timelines for each."
Level 0 (Root): "Summarize the full 200-page report." Not a base case — far too long for one context. Decompose.
Level 1 — Decompose: Split into 12 regional sections plus an introduction and conclusion = 14 sub-problems.
Level 2 — Recurse into Region 5: "Summarize the Southeast Asia section (22 pages)." Still too long for one pass. Decompose further: "Policy Overview (4 pages)," "Implementation Timeline (6 pages)," "Case Studies (8 pages)," "Recommendations (4 pages)."
Level 3 — Base Cases: "Summarize Case Studies (8 pages)" — manageable in one context. Solve directly: "Three case studies in Vietnam, Thailand, and Indonesia demonstrate that early-warning flood systems reduced displacement by 40% when paired with community training programs."
Combine Level 2: Merge the four sub-section summaries into: "The Southeast Asia section recommends scaled early-warning systems based on successful pilots, with a 2025-2030 implementation timeline and $2.1B estimated investment."
Combine Level 1: Merge all 14 section summaries into a coherent executive summary of the full report.
Result: A structured, accurate summary that could not have been produced in a single pass due to the volume and complexity of the source material.
"A medical test has 95% sensitivity and 99% specificity. The disease prevalence is 1 in 10,000. A patient tests positive. What is the probability they actually have the disease? Explain the reasoning using Bayes’ theorem."
Level 0 (Root): "Calculate posterior probability using Bayes’ theorem with the given parameters." Not a base case — requires multiple intermediate calculations. Decompose.
Level 1 — Decompose: Bayes’ theorem: P(Disease|Positive) = P(Positive|Disease) * P(Disease) / P(Positive). Three sub-problems: (A) Calculate P(Positive|Disease) * P(Disease), (B) Calculate P(Positive), (C) Divide A by B.
Level 2 — Sub-problem A: P(Positive|Disease) = 0.95 (sensitivity). P(Disease) = 1/10000 = 0.0001. Multiply: 0.95 * 0.0001 = 0.000095. Base case — solved.
Level 2 — Sub-problem B: P(Positive) = P(Pos|Disease)*P(Disease) + P(Pos|No Disease)*P(No Disease). Not a base case — decompose further.
Level 3 — Sub-problem B1: P(Pos|No Disease) = 1 - specificity = 1 - 0.99 = 0.01. P(No Disease) = 1 - 0.0001 = 0.9999. Multiply: 0.01 * 0.9999 = 0.009999. Base case — solved.
Level 3 — Sub-problem B2: P(Positive) = 0.000095 + 0.009999 = 0.010094. Base case — solved.
Combine Level 1: P(Disease|Positive) = 0.000095 / 0.010094 = 0.00941 = approximately 0.94%.
Result: Despite the positive test, there is only about a 0.94% chance the patient actually has the disease. The low prevalence means false positives vastly outnumber true positives.
"Analyze the operational efficiency of a multinational corporation with 4 divisions, each containing 3-5 departments. Identify bottlenecks, redundancies, and optimization opportunities at every organizational level."
Level 0 (Root): "Analyze full organizational efficiency." Not a base case — too many interacting components. Decompose by division.
Level 1 — Decompose: Four sub-problems: Analyze Manufacturing Division, Analyze Sales Division, Analyze R&D Division, Analyze Operations Division.
Level 2 — Recurse into Manufacturing: "Analyze Manufacturing Division efficiency." Still too broad with 5 departments. Decompose: Production, Quality Control, Logistics, Procurement, Maintenance.
Level 3 — Base Cases: "Analyze Procurement Department" — single department with defined processes. Base case. Solve directly: "Procurement shows 23-day average vendor approval cycle vs. 10-day industry standard. Dual-approval bottleneck in contracts under $50K adds unnecessary overhead. Recommendation: raise single-approval threshold to $50K."
Combine Level 2: Merge the 5 department analyses into a Manufacturing Division report: "Manufacturing shows strong production throughput but procurement and maintenance are bottlenecks. Cross-department redundancy: both QC and Production maintain separate testing equipment inventories."
Combine Level 1: Merge all 4 division reports into a corporate-level analysis. Identify cross-division patterns: "Three of four divisions maintain independent IT support teams — consolidation could save 15% on IT staffing costs."
Result: A three-level analysis identifying bottlenecks at the department, division, and corporate levels, with actionable recommendations at each level. A flat approach would miss the department-level details or the cross-division patterns.
When to Use Recursion of Thought
Best for problems with hierarchical or variable-depth complexity
Perfect For
When the problem has a natural tree structure with multiple levels of detail that cannot be captured in a single decomposition pass.
When different branches of the problem vary significantly in complexity, requiring deeper decomposition in some areas but not others.
When the total input or reasoning chain exceeds what can fit in a single context window, requiring distributed reasoning across multiple contexts.
When designing multi-agent systems where parent agents delegate to child agents, mirroring the recursive call stack pattern.
Skip It When
When the question can be answered directly without any decomposition, the overhead of recursive structure adds no value.
When you cannot control context management programmatically. RoT requires an orchestration layer to manage the recursion stack across multiple LLM calls.
When all sub-problems are at the same level of complexity and a single decomposition pass handles them adequately.
Use Cases
Where Recursion of Thought delivers the most value
Long Document Analysis
Recursively summarize or analyze documents that exceed context limits by decomposing into chapters, sections, and paragraphs, then combining results bottom-up.
Codebase Understanding
Analyze a large codebase by recursing through modules, classes, and functions, building understanding from individual components up to the full architecture.
Multi-Agent Orchestration
Design agent hierarchies where parent agents spawn child agents for sub-tasks, with each level managing its own scope and returning results upward.
Research Synthesis
Synthesize findings across dozens of papers by recursively analyzing individual studies, then grouping by theme, and finally combining into a comprehensive literature review.
Organizational Audits
Audit multilayer organizations by recursing into divisions, departments, and teams, identifying issues at each level and surfacing cross-cutting patterns.
Complex Project Planning
Break down large projects into phases, phases into workstreams, workstreams into tasks, and tasks into subtasks — each level planned with appropriate detail.
Where Recursion of Thought Fits
A recursive evolution in decomposition framework design
Recursion of Thought maps directly to modern agentic AI architectures. When a parent agent encounters a task too complex for its context, it spawns a child agent — exactly like a recursive function call. Techniques like AutoGen and CrewAI implement this pattern. Use RoT as your mental model when designing hierarchical agent systems that need to handle arbitrary problem depth.
Related Techniques
Explore complementary decomposition approaches
Think Recursively
Design recursive reasoning workflows or explore more decomposition frameworks in the Praxis Library.