Skip to content

refactor(ast-analysis): decompose engine and visitors, break visitor-utils cycle#1231

Merged
carlos-alm merged 5 commits into
mainfrom
refactor/titan-decomposition-ast-analysis
May 27, 2026
Merged

refactor(ast-analysis): decompose engine and visitors, break visitor-utils cycle#1231
carlos-alm merged 5 commits into
mainfrom
refactor/titan-decomposition-ast-analysis

Conversation

@carlos-alm

Copy link
Copy Markdown
Contributor

Summary

  • Breaks the destructuring cycle between visitor-utils and visitors
  • Splits the engine + visitors into smaller modules

Commits

  • 24c8cf5: refactor(ast-analysis): break visitor-utils destructuring cycle
  • 4f34034: refactor(ast-analysis): decompose engine and visitors

Context

Part of the Titan Paradigm cleanup pass (see .codegraph/titan/TITAN_REPORT.md). Merge order: this PR is #2 of 10 (mergeOrder position: 2).

Caveats

  • WASM grammars not available in dev worktree — CI will run full test matrix

Test plan

  • CI passes (lint, build, full test matrix)
  • Verify no new cycles introduced (codegraph stats)

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Claude finished @carlos-alm's task —— View job


PR Review: AST Analysis Refactoring

Completed comprehensive review following the 5-phase process:

Todo List

  • Phase 1: Root cause analysis - understand the problem being solved
  • Phase 2: Configuration change detection
  • Phase 3: Test integrity verification
  • Phase 4: Code quality review
  • Phase 5: Backlog compliance assessment
  • Final: Provide comprehensive assessment with rating

Root Cause Analysis

Problem Being Solved: Breaking a destructuring cycle in visitor-utils.ts where mutual recursion between extractParamNames, extractObjectDestructNames, and extractArrayDestructNames created a 3-node cycle. The refactoring also decomposes the engine and visitors into smaller, more focused modules.

Why This Approach: The solution uses an iterative worklist pattern instead of mutual recursion to handle nested destructuring patterns. This is architecturally sound - replacing recursion with iteration is a standard approach for breaking call graph cycles while preserving functionality.

Risk Assessment: Low risk. The refactoring maintains existing interfaces and behavior while improving code structure. No configuration changes that mask underlying issues.

Configuration Change Detection

No problematic configuration changes found

  • No linting rules disabled or relaxed
  • No test timeouts increased or coverage thresholds lowered
  • No TypeScript strict mode flags disabled
  • No build configuration changes that hide errors

Test Integrity

No test integrity issues identified

  • No removed or weakened assertions
  • No tests made less strict
  • No suspicious test modifications alongside implementation changes
  • No eslint-disable comments without justification

Code Quality Review

Positive Aspects:

  1. Clean cycle breaking: The visitor-utils.ts refactoring replaces mutual recursion with a worklist-based approach, which is the correct architectural pattern for breaking cycles
  2. Good separation of concerns: Engine decomposition splits visitor setup, result processing, and delegation into focused functions
  3. Maintains API compatibility: Public interfaces remain unchanged
  4. Proper error handling: Maintains existing error handling patterns
  5. Performance conscious: Optimizes repeated calculations (e.g., newTypesFor cache)

Areas of Concern:

  1. Large diff scope: 425 additions, 223 deletions across complex AST analysis code increases review difficulty
  2. Complex refactoring: The visitor pattern changes and engine decomposition touch critical parsing logic
  3. Limited test context: PR mentions "WASM grammars not available in dev worktree" which limits verification

Code-Specific Observations:

visitor-utils.ts:174-203 - The worklist implementation correctly breaks the cycle:

const stack: TreeSitterNode[] = [node];
while (stack.length > 0) {
  const current = stack.pop();
  // ... process without recursion
}

engine.ts:896-946 - Good architectural decomposition with clear separation between native fast path, WASM fallback, and visitor orchestration.

ast-store-visitor.ts:282-334 - Clean visitor factory pattern with proper context encapsulation.

Backlog Compliance

  • Zero-dep: ✓ - No new runtime dependencies added
  • Foundation-aligned: ✓ - Maintains core principle of local-only analysis without cloud dependencies
  • Problem-fit: 3/5 - Improves codebase maintainability but doesn't directly address the core "lost AI" problem of token waste and hallucinated dependencies
  • Breaking: No - Maintains existing API contracts and behavior
  • Tier: 1 (zero-dep ✓ + foundation-aligned ✓)

Critical Concerns

None identified. This is a well-executed refactoring that improves code structure without introducing risk.

Final Recommendation

  • Rating: ⭐⭐⭐⭐☆ (4/5)
  • Action: APPROVE WITH CONDITIONS
  • Reasoning: Solid architectural improvement that breaks a real cycle problem. The refactoring follows good patterns and maintains API stability. Deducting one star due to the large diff scope in critical parsing code and limited test verification context.

Conditions for approval:

  1. Ensure CI passes with full test matrix (mentioned in test plan)
  2. Verify no new cycles introduced with codegraph stats (mentioned in test plan)
  3. Consider adding integration test coverage for the worklist destructuring logic to prevent regression

This refactoring represents good technical debt cleanup as part of the broader Titan Paradigm work. The cycle-breaking approach is architecturally sound and the code quality is high.


@greptile-apps

greptile-apps Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR is a pure structural refactor of src/ast-analysis: it breaks a mutual-recursion cycle in visitor-utils.ts by converting extractParamNames to an iterative worklist, and decomposes large inline closures in engine.ts, ast-store-visitor.ts, and dataflow-visitor.ts into named, module-level helpers with explicit context structs (CollectCtx, DataflowDispatchCtx).

  • visitor-utils.ts: All destructuring cases (shorthand, rest, wrapper, default, array, object) verified equivalent to original.
  • engine.ts: Walk loop decomposed into named helpers; call-site threading and return semantics unchanged.
  • ast-store-visitor.ts / dataflow-visitor.ts: Closure-scoped helpers lifted to standalone functions; varDeclaratorNode/varDeclaratorNodes branches correctly merged.

Confidence Score: 5/5

Purely structural decomposition — no algorithmic changes, all extracted helpers thread the same state they previously closed over.

Every extracted function is a direct mechanical lift of the original inline code. Argument ordering and reference sharing (e.g. scopeStack between enterFunction/exitFunction and dispatchCtx) are correct. The iterative worklist in extractParamNames handles all the same cases as the previous mutual recursion with identical traversal order. No behavioral delta was found across all four files.

No files require special attention.

Important Files Changed

Filename Overview
src/ast-analysis/engine.ts Inline walk loop decomposed into four named helpers; fast/native paths extracted — no behavioral changes, argument threading matches original closures.
src/ast-analysis/visitor-utils.ts Mutual-recursion cycle replaced with an iterative worklist; resolveParamNode returns {names} or {next}; all edge cases verified equivalent to original.
src/ast-analysis/visitors/ast-store-visitor.ts Closure-scoped helpers lifted to module-level functions with CollectCtx struct; hasOwn guard moved into resolveAstKind; behavior preserved.
src/ast-analysis/visitors/dataflow-visitor.ts enterFunctionScope and dispatchDataflowNode extracted; varDeclaratorNode/varDeclaratorNodes branches merged; scopeStack reference sharing correct.

Reviews (5): Last reviewed commit: "Merge branch 'main' into refactor/titan-..." | Re-trigger Greptile

Comment on lines +154 to +157
if (rules.shorthandPropPattern && child.type === rules.shorthandPropPattern) {
// Shorthand prop is a direct identifier — handled in the worklist
// by `resolveParamNode` once requeued.
next.push(child);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Inaccurate inline comment for shorthand prop handling

The comment says shorthand props are "handled in the worklist by resolveParamNode once requeued", but that's not what happens. resolveParamNode has no case for shorthandPropPattern and returns null if it ever receives one. The node is handled by the shorthandPropPattern guard at the top of the while loop in extractParamNames — before resolveParamNode is ever called. The comment should say "handled by the shorthand guard in the extractParamNames worklist loop" to avoid confusion for anyone who traces the path.

Fix in Claude Code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — the comment now says "handled by the shorthand guard in the extractParamNames worklist loop (before resolveParamNode)", which accurately describes the actual execution path.

Comment on lines +443 to 487
/**
* Route a node to the appropriate dataflow handler based on its type, or return
* `false` if no handler matched. Function-definition nodes are signalled by
* a `true` return so the caller can short-circuit.
*/
function dispatchDataflowNode(ctx: DataflowDispatchCtx, node: TreeSitterNode): boolean {
const { rules } = ctx;
const t = node.type;

if (rules.functionNodes.has(t)) return true;

if (rules.returnNode && t === rules.returnNode) {
handleReturn(node, rules, ctx.scopeStack, ctx.returns);
return true;
}

if (
(rules.varDeclaratorNode && t === rules.varDeclaratorNode) ||
rules.varDeclaratorNodes?.has(t)
) {
handleVarDeclarator(node, rules, ctx.scopeStack, ctx.assignments, ctx.isCallNode);
return true;
}

if (ctx.isCallNode(t)) {
handleCallExpr(node, rules, ctx.scopeStack, ctx.argFlows);
return true;
}

if (rules.assignmentNode && t === rules.assignmentNode) {
handleAssignment(node, rules, ctx.scopeStack, ctx.assignments, ctx.mutations, ctx.isCallNode);
return true;
}

if (rules.expressionStmtNode && t === rules.expressionStmtNode) {
handleExprStmtMutation(node, rules, ctx.scopeStack, ctx.mutations, ctx.isCallNode);
return true;
}

return false;
}

export function createDataflowVisitor(rules: AnyRules): Visitor {
const isCallNode: (t: string) => boolean = rules.callNodes
? (t: string) => rules.callNodes.has(t)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 dispatchDataflowNode return value is never used

The function returns boolean but every call site discards it — enterNode always returns undefined regardless. The true/false values serve only as early-exit signals inside the function itself. Changing the signature to void (or using return; for early exits) would remove the false implication that the caller cares about the result and prevent a future reader from mistaking the boolean for a meaningful EnterNodeResult-style signal.

Fix in Claude Code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — changed dispatchDataflowNode return type from boolean to void and replaced all return true/return false statements with plain return. The JSDoc was updated to drop the misleading mention of a boolean signal.

@github-actions

github-actions Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Codegraph Impact Analysis

37 functions changed30 callers affected across 9 files

  • accumulateWalkTime in src/ast-analysis/engine.ts:757 (3 transitive callers)
  • applyVisitorResults in src/ast-analysis/engine.ts:778 (3 transitive callers)
  • processFileWalk in src/ast-analysis/engine.ts:800 (3 transitive callers)
  • runUnifiedWalkPass in src/ast-analysis/engine.ts:835 (3 transitive callers)
  • tryNativeStandaloneAnalysis in src/ast-analysis/engine.ts:858 (3 transitive callers)
  • runFastPathIfApplicable in src/ast-analysis/engine.ts:878 (3 transitive callers)
  • runAnalyses in src/ast-analysis/engine.ts:896 (3 transitive callers)
  • resolveParamNode in src/ast-analysis/visitor-utils.ts:100 (5 transitive callers)
  • collectObjectDestructChildren in src/ast-analysis/visitor-utils.ts:148 (4 transitive callers)
  • extractParamNames in src/ast-analysis/visitor-utils.ts:174 (6 transitive callers)
  • buildKindHandlers in src/ast-analysis/visitors/ast-store-visitor.ts:190 (7 transitive callers)
  • findParentDef in src/ast-analysis/visitors/ast-store-visitor.ts:211 (3 transitive callers)
  • resolveParentNodeId in src/ast-analysis/visitors/ast-store-visitor.ts:224 (2 transitive callers)
  • CollectCtx.rows in src/ast-analysis/visitors/ast-store-visitor.ts:237 (0 transitive callers)
  • CollectCtx.matched in src/ast-analysis/visitors/ast-store-visitor.ts:238 (0 transitive callers)
  • CollectCtx.relPath in src/ast-analysis/visitors/ast-store-visitor.ts:239 (0 transitive callers)
  • CollectCtx.defs in src/ast-analysis/visitors/ast-store-visitor.ts:240 (0 transitive callers)
  • CollectCtx.nodeIdMap in src/ast-analysis/visitors/ast-store-visitor.ts:241 (0 transitive callers)
  • CollectCtx.skipParentLookup in src/ast-analysis/visitors/ast-store-visitor.ts:242 (0 transitive callers)
  • CollectCtx.kindHandlers in src/ast-analysis/visitors/ast-store-visitor.ts:243 (0 transitive callers)

@carlos-alm

Copy link
Copy Markdown
Contributor Author

@greptileai

@carlos-alm carlos-alm merged commit 387bcb8 into main May 27, 2026
21 checks passed
@carlos-alm carlos-alm deleted the refactor/titan-decomposition-ast-analysis branch May 27, 2026 22:22
@github-actions github-actions Bot locked and limited conversation to collaborators May 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant