Phase 3: BASIC compiler frontend (zxbc) — 980/1036 parse-only parity

[Xalior]

Summary

Complete C port of the BASIC compiler frontend (zxbc), achieving 94.6% exit-code parity with Python's --parse-only mode across all 1036 functional test files.

Parse-only parity

Metric	Count	Notes
Total test files	1036	tests/functional/arch/zx48k/*.bas
Exit code matches Python	980 (94.6%)	C and Python agree
False positives (C=1, Python=0)	0	No regressions — C never wrongly rejects valid code
False negatives (C=0, Python=1)	56	C missing errors that Python catches

A test "matches" when C and Python produce the same exit code (0 or non-zero). Python's --parse-only still runs the full pipeline (semantic analysis, optimizer, translator) before the parse_only check — so parity requires more than just syntax parsing.

What the C port catches

Beyond syntax parsing, the C port now performs post-parse validation matching several Python semantic checks:

• GOTO/GOSUB label targets — undefined label detection
• Forward-declared functions — DECLARE without matching implementation
• DECLARE/definition mismatches — return type, parameter count, type, and byref mode
• SUB-as-FUNCTION — calling a SUB in expression context
• Duplicate function/sub definitions
• READ/DATA consistency — READ without DATA, READ into non-lvalue/array targets
• Mandatory-after-optional params — parameter ordering validation
• Explicit mode — #pragma explicit undeclared variable checks
• GOSUB inside functions — not permitted
• Duplicate labels — within the same scope
• Constant assignment — reject LET on CONST variables

Remaining 56 false negatives

Suspected upstream bugs (3 tests) — @Xalior please confirm

chr, chr1, and const6 all crash in Python's optimizer with an AttributeError:

File "src/api/optimize.py", line 331, in visit_LET
    if x.token == "FUNCCALL" or getattr(x, "fname") in {"IN", "RND", "USR"}
                                ^^^^^^^^^^^^^^^^^^^
AttributeError: 'SymbolSTRING' object has no attribute 'fname'

These are valid programs (e.g. LET a$ = CHR$ 33). The parser and semantic checker accept them. The crash happens because the optimizer's constant-folding resolves CHR$(65) into a SymbolSTRING node, but then visit_LET calls getattr(x, "fname") without a default — which throws on string nodes. The fix in Python would be getattr(x, "fname", None).

Our C port correctly exits 0 for these. Are these known bugs upstream? Should we match Python's crash behaviour, or is our exit-0 the correct answer here?

POKE with array subscript (4 tests)

poke3, poke4, poke5, poke6 — Python's PLY lexer emits a distinct ARRAY_ID token for identifiers previously declared as arrays. The grammar rule POKE ARRAY_ID COMMA expr COMMA expr catches multi-arg POKE with array targets as a syntax error. Our hand-written lexer doesn't distinguish ARRAY_ID from ID, so we parse the POKE differently and don't hit the error path.

Semantic analysis needed (42 tests)

These require type checking, constant evaluation, or argument validation that goes beyond our current parser-level checks:

• Type checking (11): 01, 50, 51, bad_sigil, do_crash, sn_crash, optional_param3, lcd2, typecast2, substr_expr_err, rman
• Argument validation (12): 62, bad_fname_err0–6, keyword_arg1, keyword_arg3, refconstparam3, pararray2–4
• Array semantics (7): 53, error_array, array_err, let_array_substr4/6/8, let_array_wrong_dims
• Constant evaluation (8): array11, arrlabels11/11b, const4/5/9, dim_const_crash, dim_dyn_err
• Declaration/scope (4): explicit7, include_error, llb, label_decl2

Syntax edge cases (7 tests)

PLY's LALR(1) grammar rejects certain constructs that our recursive-descent parser is more permissive about:

• bin02 — BIN followed by identifier (PLY expects binary digit literal)
• def_func_inline — FUNCTION f() END FUNCTION on one line (PLY requires newline before body)
• due_crash — CODE as parameter name (always a keyword token in PLY)
• ifthencoendif2 — ELSE: with trailing colon
• doloopuntilsplitted — LOOP UNTIL on same line as DO body via colon separator
• whileempty — WHILE ... END WHILE on one line
• let_expr_type_crash — identifier used as type name in AS

Architecture

• 132 C unit tests across 7 programs matching Python's tests/api/, tests/symbols/, tests/cmdline/ suites
• 4 shell cmdline tests matching tests/cmdline/test_zxb.py
• Hand-written recursive-descent lexer + parser (~3,500 lines)
• Full AST with 30 tagged-union node kinds
• Symbol table with lexical scoping, type registry, all basic types
• CLI with all zxbc flags, config file loading, --parse-only
• Preprocessor integration via zxbpp static library

Component	Files	What
Lexer	lexer.c/h	All BASIC tokens, string escapes, ASM mode
Parser	parser.c/h	Recursive descent with Pratt expression parsing
AST	ast.c, zxbc.h	30 node kinds, type system, arena-allocated
Symbol table	compiler.c	Scoping, declare/lookup, suffix handling
CLI	args.c/h, options.c/h	ya_getopt, config files, cmdline_set bitmask
Error system	errmsg.c/h	Format-matching Python error output
Unit tests	7 test programs	132 tests matching Python test suites

Test plan

• 980/1036 parse-only tests match Python exit codes (0 false positives)
• All 132 C unit tests pass
• All 4 shell cmdline tests pass
• All 96 zxbpp tests still pass
• All 61 zxbasm tests still pass
• CI green on Linux x86_64, Linux ARM64, macOS ARM64, Windows x86_64

🤖 Generated with Claude Code