Reference

Architecture

Overview

Eta’s compiler is structured as a six-stage pipeline that transforms UTF-8 source text into stack-based bytecode and executes it on a virtual machine. All phases are orchestrated by the Driver class, which owns the full runtime state and supports incremental execution (each REPL input shares the same VM globals and linker state).

Pipeline:  Source → Lex → Parse → Expand → Link → Analyze → Emit → Execute

System Diagram

Stage-by-Stage Walkthrough

1. Lexer

File: lexer.h / lexer.cpp

The lexer converts raw UTF-8 source into a stream of Token values. It handles:

• Parentheses, brackets, and dot notation
• Scheme number literals in binary, octal, decimal, and hexadecimal radixes
• Character literals (#\a, #\newline, #\x03BB)
• Strings with escape sequences
• Booleans (#t / #f)
• Block comments (#| … |#), line comments (;), and datum comments (#;)
• #( vector and #u8( byte-vector start tokens

Each token carries a Span with file ID, line, column, and byte offset for precise diagnostic messages.

2. Parser

File: parser.h / parser.cpp

The parser consumes tokens and produces an S-expression tree. The AST node types form a variant (SExpr):

3. Expander (Macro Expansion)

File: expander.h / expander.cpp

The expander desugars high-level Scheme forms into a minimal core language. After expansion, only these forms remain:

if  begin  set!  lambda  quote
define  defun  module
dynamic-wind  values  call-with-values  call/cc  apply

Derived forms that are expanded away:

The expander also rewrites internal defines in lambda bodies into letrec per R7RS semantics (controlled by enable_internal_defines_to_letrec).

For syntax-rules, literal keywords are matched by lexical binding identity (not raw symbol text), and free template identifiers are rebound to their definition-site context so use-site locals do not capture them.

4. Module Linker

File: module_linker.h / module_linker.cpp

The linker resolves inter-module dependencies in two phases:

1. Index — scan all (module …) forms, collecting define and export names into a ModuleTable.
2. Link — resolve import clauses, applying only, except, and rename filters, populating each module’s visible set.

Errors: UnknownModule, CircularDependency, ConflictingImport, ExportOfUnknownName, DuplicateModule.

5. Semantic Analyzer

File: semantic_analyzer.h / semantic_analyzer.cpp

The semantic analyzer walks the desugared S-expressions and produces a Core IR graph (core_ir.h). Key responsibilities:

• Name resolution — binds identifiers to BindingIds in nested Scopes
• Closure conversion — detects variables captured across lambda boundaries and assigns them Address::Upval slots
• Address assignment — every variable gets a concrete Address (Local / Upval / Global) with a numeric slot
• Tail-position marking — sets Node::tail = true on expressions in tail position so the emitter can generate TailCall
• Builtin seeding — pre-allocates global slots 0..N−1 for primitives from the BuiltinEnvironment

Core IR Node Types

All nodes are arena-allocated (core::Arena, 16 KB blocks) for pointer stability and cache locality.

Node  =  Var | Const | Quote | If | Begin | Set
       | Lambda | Call | Values | CallWithValues
       | DynamicWind | CallCC | Apply

6. Emitter

File: emitter.h / emitter.cpp

The emitter walks the Core IR tree and produces BytecodeFunction objects containing sequences of Instructions. Each module produces a top-level _init function. Nested lambdas are emitted recursively and stored in a thread-safe BytecodeFunctionRegistry (backed by std::deque for pointer stability). Named functions use deterministic <module>:<binding> labels, and anonymous lambdas use source-stable labels <lambda@module:line:column>.

The emitter’s key decisions:

• Constants are deduplicated per function (string literal caching)
• Closures use MakeClosure with upvalue sources pushed on the stack
• Letrec self-references are patched post-hoc with PatchClosureUpval
• Quoted data is recursively materialized into heap objects at emit time
• Tail calls emit TailCall / TailApply instead of Call / Apply

7. VM Execution

File: vm.h / vm.cpp

The VM is a stack-based interpreter with a frame stack. See Bytecode & VM for full details.

Diagnostic System

File: diagnostic.h

All compiler phases report errors through a single DiagnosticEngine. Each error carries:

• A DiagnosticCode (lexer 0–99, parser 100–199, expander 200–299, linker 300–399, semantic 400–499, runtime 500–599)
• A Severity (Error, Warning, Note, Hint)
• A Span for source location
• Optional related spans with labels for multi-span errors

Stage-specific error types (LexError, ParseError, ExpandError, LinkError, SemanticError, VMError) are converted via to_diagnostic<T>() template specializations.

Frontends

All frontends create a Driver instance, which constructs the Heap, InternTable, VM, BuiltinEnvironment, and BytecodeFunctionRegistry and wires them together.

Builtin Catalog Contract

Files: builtin_catalog.h / builtin_catalog.cpp

The builtin catalog is the single source of truth for builtin slot metadata (name, arity, rest-arg flag, owner, and optional docs metadata).

• register_builtin_specs(...) seeds analysis/LSP environments from builtin_catalog() in stable registration order.
• Driver bootstrap then patches those slots via begin_patching() -> register_all_primitives(...) -> verify_all_patched(), preserving analyzer/runtime slot parity across eta, etai, etac, and eta_lsp.