Skip to content

engine: decouple module-aware parse context from per-variable parse cache #372

@bpowers

Description

@bpowers

Problem

parse_source_variable currently has a tension between correctness and incrementality:

  • Making it globally module-aware fixes module-expansion-sensitive parsing cases (for example PREVIOUS(x) where x is a user-written stdlib-call auxiliary that should be treated as module-backed).
  • But global module-aware parsing also broadens invalidation and broke per-variable cache stability guarantees in incremental compilation tests.

In this branch we reverted the global behavior and limited module-aware parsing to codegen-specific paths to restore cache stability while preserving correctness where needed.

Why this matters

This is an architectural boundary issue in the incremental pipeline. Without a dedicated refactor, we risk repeated regressions between:

  1. Correct module-aware expansion behavior, and
  2. Stable, minimal per-variable invalidation/reparse behavior.

Current workaround

  • Keep parse_source_variable on the narrow per-variable path (no global module context).
  • Thread module-aware parsing context only through specific codegen/implicit-info paths that require it.

This reduces risk now, but it is not a clean long-term design.

Proposed direction

Introduce an explicit model-level input for module parsing context (for example a tracked module_idents set) and thread it into parsing/codegen in a way that preserves salsa cache granularity.

Design goal: correctness-sensitive module context should be explicit and local to the computations that require it, not an implicit global dependency that destabilizes unrelated variable caches.

Acceptance criteria

  • Add coverage for module-aware correctness cases (including PREVIOUS over module-backed identifiers) through incremental paths.
  • Preserve existing per-variable cache stability tests (unchanged variable should not be reparsed due to unrelated edits).
  • No regression in incremental compile behavior for implicit/module-expanded variables.
  • Document the final parse-context dependency model in docs/design/incremental-compilation.md (or equivalent).

Related

  • docs/tech-debt.md item: "parse_source_variable Missing module_idents Context"
  • Recent regression observed while making parse_source_variable globally module-aware and subsequently scoping behavior back to codegen-specific paths.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions