Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
40d5124
Debugger plan and next steps
lagergren Feb 11, 2026
983dcd2
Temporarily enabling lang resolution for this branch.
lagergren Feb 12, 2026
8648137
Add shared run configurations, build profiling benchmarks, and plugin…
lagergren Feb 12, 2026
4cb5e09
Upgrade to IntelliJ 2025.3.2 and fix LSP server restart loop
lagergren Feb 12, 2026
ec8df78
Document LSP4IJ design choice and simplify race condition comment
lagergren Feb 12, 2026
01d20ec
Rename debug-adapter to dap-server for consistency with lsp-server
lagergren Feb 12, 2026
1b2036c
Wire up LSP4IJ DAP extension point for debug adapter integration
lagergren Feb 12, 2026
3cb1ddf
Extract shared PluginPaths utility and update lsp-processes.md
lagergren Feb 12, 2026
3ad3aa7
Document DAP process lifecycle and update plans
lagergren Feb 12, 2026
24c76ca
Expand LSP next-steps plan with full feature audit and IDE integration
lagergren Feb 12, 2026
f6b7013
Revise sprint ordering and add compiler adapter milestone
lagergren Feb 12, 2026
6a2d84c
Expand multi-IDE strategy with market data and priority rankings
lagergren Feb 12, 2026
60306eb
Stub out all planned LSP features with input-logging defaults
lagergren Feb 12, 2026
ab2e545
Refactor adapter defaults from interface into AbstractXtcCompilerAdapter
lagergren Feb 12, 2026
cc17380
Add semantic tokens (tree-sitter Tier 1) with opt-in build flag
lagergren Feb 12, 2026
280879f
Update plan with implementation status and add semantic tokens benefi…
lagergren Feb 12, 2026
cf1ef3c
Add workspace symbol index with cross-file definition and fuzzy search
lagergren Feb 13, 2026
75785d7
Clean up tree-sitter internals, fix highlights template, remove dead …
lagergren Feb 13, 2026
907e2b6
Fix LSP server issues, add configurable logging, expand JAR resolutio…
lagergren Feb 19, 2026
c9b58d8
Fix review follow-ups, enable semantic tokens, update plans
lagergren Feb 19, 2026
c3625d5
Fix composite build property isolation for lang included build
lagergren Feb 19, 2026
ada9ac5
Fix semantic tokens crash on rename, EDT violation, add match logging
lagergren Feb 19, 2026
49f9503
Document known IntelliJ/LSP4IJ issues, add rename regression tests
lagergren Feb 19, 2026
1f79704
Improve LSP pipeline logging, replace Unicode with ASCII in all output
lagergren Feb 19, 2026
4bda9a8
Disable includeBuildLang for PR (default off for CI and other develop…
lagergren Feb 19, 2026
3ca88bd
Fix misleading incremental parsing comment in TreeSitterAdapter.compi…
lagergren Feb 19, 2026
c15d429
Clean up LSP logging, fix stale docs, skip cached downloads, fix EDT …
lagergren Feb 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .run/Build Plugin ZIP.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Build Plugin ZIP" type="GradleRunConfiguration" factoryName="Gradle">
<ExternalSystemSettings>
<option name="executionName" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="externalSystemIdString" value="GRADLE" />
<option name="scriptParameters" value="" />
<option name="taskDescriptions">
<list />
</option>
<option name="taskNames">
<list>
<option value="lang:intellij-plugin:buildPlugin" />
</list>
</option>
<option name="vmOptions" />
</ExternalSystemSettings>
<GradleScriptDebugEnabled>true</GradleScriptDebugEnabled>
<method v="2" />
</configuration>
</component>
50 changes: 50 additions & 0 deletions .run/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Shared Run Configurations

IntelliJ automatically discovers `*.run.xml` files in this directory and adds them
to the run configuration dropdown in the toolbar. These are shared via version control
so all contributors get them out of the box.

## IntelliJ Plugin Development

| Configuration | What It Does |
|---|---|
| **Run Plugin in IDE** | Launches a development IntelliJ IDEA instance with the XTC plugin installed in its sandbox. Uses Gradle task `lang:runIntellijPlugin`. Supports debugging (Shift+F9). |
| **Run Plugin Tests** | Runs the IntelliJ plugin test suite (unit tests + platform tests if configured). Uses Gradle task `lang:intellij-plugin:test`. |
| **Build Plugin ZIP** | Builds a distributable plugin ZIP archive at `lang/intellij-plugin/build/distributions/`. Uses Gradle task `lang:intellij-plugin:buildPlugin`. The ZIP can be installed in any IntelliJ IDE via Settings > Plugins > Install from Disk. |

## XTC Compiler / Runtime

| Configuration | What It Does |
|---|---|
| **xtc build** | Compiles an XTC module using the XTC compiler (`org.xvm.tool.Launcher build`). Defaults to `manualTests/src/main/x/FizzBuzz.x`. Runs as a Java application in the `javatools` module. |
| **xtc run** | Compiles and runs an XTC module (`org.xvm.tool.Launcher run`). Defaults to FizzBuzz.x — outputs FizzBuzz to the console. Useful for quick end-to-end validation of compiler + runtime. |

### Prerequisites for `xtc` Configurations

The `xtc build/run` configurations require the XDK standard library modules to be
installed. Run once before first use:

```bash
./gradlew xdk:installDist
```

This populates `xdk/build/install/xdk/lib/` with the compiled standard library modules
(ecstasy.xtc, json.xtc, collections.xtc, etc.) and `xdk/build/install/xdk/javatools/`
with the runtime bridge modules. These rarely change — you only need to re-run
`installDist` after modifying the standard library or javatools.

### Customizing the Target Module

To compile/run a different `.x` file, duplicate the run configuration and change the
program arguments. The format is:

```
build|run <path-to-file.x> -L xdk/build/install/xdk/lib -L xdk/build/install/xdk/javatools/javatools_turtle.xtc -L xdk/build/install/xdk/javatools/javatools_bridge.xtc
```

Good candidates in `manualTests/src/main/x/`:
- `TestSimple.x` — minimal module with Console injection and arithmetic
- `FizzBuzz.x` — pattern matching with switch expressions
- `collections.x` — exercises the collections library
- `numbers.x` — numeric type operations
- `lambda.x` — lambda and closure tests
21 changes: 21 additions & 0 deletions .run/Run Plugin Tests.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Run Plugin Tests" type="GradleRunConfiguration" factoryName="Gradle">
<ExternalSystemSettings>
<option name="executionName" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="externalSystemIdString" value="GRADLE" />
<option name="scriptParameters" value="" />
<option name="taskDescriptions">
<list />
</option>
<option name="taskNames">
<list>
<option value="lang:intellij-plugin:test" />
</list>
</option>
<option name="vmOptions" />
</ExternalSystemSettings>
<GradleScriptDebugEnabled>true</GradleScriptDebugEnabled>
<method v="2" />
</configuration>
</component>
21 changes: 21 additions & 0 deletions .run/Run Plugin in IDE.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Run Plugin in IDE" type="GradleRunConfiguration" factoryName="Gradle">
<ExternalSystemSettings>
<option name="executionName" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="externalSystemIdString" value="GRADLE" />
<option name="scriptParameters" value="" />
<option name="taskDescriptions">
<list />
</option>
<option name="taskNames">
<list>
<option value="lang:runIntellijPlugin" />
</list>
</option>
<option name="vmOptions" />
</ExternalSystemSettings>
<GradleScriptDebugEnabled>true</GradleScriptDebugEnabled>
<method v="2" />
</configuration>
</component>
9 changes: 9 additions & 0 deletions .run/xtc build.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="xtc build" type="Application" factoryName="Application">
<option name="MAIN_CLASS_NAME" value="org.xvm.tool.Launcher" />
<module name="javatools.main" />
<option name="PROGRAM_PARAMETERS" value="build manualTests/src/main/x/FizzBuzz.x -L xdk/build/install/xdk/lib -L xdk/build/install/xdk/javatools/javatools_turtle.xtc -L xdk/build/install/xdk/javatools/javatools_bridge.xtc" />
<option name="VM_PARAMETERS" value="-ea --enable-native-access=ALL-UNNAMED" />
<method v="2" />
</configuration>
</component>
9 changes: 9 additions & 0 deletions .run/xtc run.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="xtc run" type="Application" factoryName="Application">
<option name="MAIN_CLASS_NAME" value="org.xvm.tool.Launcher" />
<module name="javatools.main" />
<option name="PROGRAM_PARAMETERS" value="run manualTests/src/main/x/FizzBuzz.x -L xdk/build/install/xdk/lib -L xdk/build/install/xdk/javatools/javatools_turtle.xtc -L xdk/build/install/xdk/javatools/javatools_bridge.xtc" />
<option name="VM_PARAMETERS" value="-ea --enable-native-access=ALL-UNNAMED" />
<method v="2" />
</configuration>
</component>
30 changes: 30 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,36 @@ val taskName by tasks.registering {
- NEVER run without the configuration cache enabled. Everything MUST work with the configuration cache.


## Lang Composite Build Properties

The `lang/` directory is a **Gradle composite build** that is disabled by default. Two `-P` properties are required to enable it when running any `./gradlew :lang:*` task from the project root:

```bash
./gradlew :lang:<task> -PincludeBuildLang=true -PincludeBuildAttachLang=true
```

### What the properties do
- **`-PincludeBuildLang=true`**: Includes the `lang/` directory as a composite build, making `:lang:*` tasks visible to the root project
- **`-PincludeBuildAttachLang=true`**: Wires `lang/` lifecycle tasks (build, test, etc.) to the root build's lifecycle, so `./gradlew build` from the root also builds lang

### Why they exist
Both properties default to `false` in the root `gradle.properties` so that:
1. CI and other developers don't need to build `lang/` unless they're working on it
2. The main XDK build stays fast for contributors who aren't touching language tooling
3. The composite build inclusion is opt-in to avoid unexpected build interactions

### When to use them
- **Always** when running any `./gradlew :lang:*` task from the project root
- Both properties are always needed together -- using only one will fail
- If you forget them, the build will fail with "project ':lang' not found" or similar

### Alternative: local gradle.properties override
Instead of passing `-P` flags every time, developers can set them in their local `gradle.properties`:
```properties
includeBuildLang=true
includeBuildAttachLang=true
```

# important-instruction-reminders
- Do what has been asked; nothing more, nothing less.
- NEVER proactively create documentation files (*.md) or README files. Only create documentation files if explicitly requested by the User.
Expand Down
14 changes: 11 additions & 3 deletions gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -101,14 +101,22 @@ includeBuildAttachManualTests=false
#
# Should lang (LSP server + IntelliJ plugin) be included and attached to the root build?
#
# includeBuildLang: Whether to include lang as a composite build (IDE visibility, task
# 1. includeBuildLang: Whether to include lang as a composite build (IDE visibility, task
# addressability via ./gradlew lang:..., configuration cache inheritance). Does NOT
# affect root lifecycle tasks (build, check, clean, assemble) — see attach flag below.
# includeBuildAttachLang: Whether lang lifecycle tasks are wired to root lifecycle tasks,
# affect root lifecycle tasks (build, check, clean, assemble).
# 2. includeBuildAttachLang: Whether lang lifecycle tasks are wired to root lifecycle tasks,
# so that ./gradlew build also builds lang. Requires includeBuildLang=true.
#
# Use ./gradlew lang:lsp-server:build to build LSP server separately.
# CI overrides includeBuildLang to false for push/PR builds (see commit.yml).
#
# TODO: We want to have lang part of the default build, ideally, provided that it can be shown to bring no significant overhead with warm caches.
#
#includeBuildLang=true
#includeBuildAttachLang=true
includeBuildLang=false
includeBuildAttachLang=false
lsp.buildSearchableOptions=false
lsp.semanticTokens=true
lsp.adapter=treesitter

4 changes: 2 additions & 2 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ lang-slf4j = "2.0.17"
# =============================================================================
# Lang tooling - IntelliJ plugin
# =============================================================================
lang-intellij-ide = "2025.1"
lang-intellij-ide = "2025.3.2"
lang-intellij-jdk = "21"
lang-intellij-platform-gradle-plugin = "2.10.5"
lang-intellij-platform-gradle-plugin = "2.11.0"
# JUnit 4.x is required at RUNTIME by the IntelliJ Platform test harness
# (com.intellij.testFramework). IntelliJ's test infrastructure internally depends on JUnit 4
# runners and rules, even when tests themselves are written with JUnit 5/Jupiter. Without this
Expand Down
38 changes: 20 additions & 18 deletions lang/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ and editor support.
>
> We are actively working to improve stability and move toward a supported beta release.

> **Note:** All `./gradlew :lang:*` commands below assume `-PincludeBuildLang=true -PincludeBuildAttachLang=true` are passed when running from the project root. See [Composite Build Properties](../CLAUDE.md) in the project CLAUDE.md for details.

## Enabling the Lang Build

By default, the lang build is **disabled** in `gradle.properties`. To include it in the XVM composite build,
Expand All @@ -28,7 +30,7 @@ you need to set two properties:
### Option 1: Command Line (Temporary)

```bash
./gradlew build -PincludeBuildLang=true -PincludeBuildAttachLang=true
./gradlew build
```

### Option 2: Environment Variables (Session/Shell)
Expand All @@ -53,7 +55,7 @@ includeBuildAttachLang=true

## Testing the Tree-sitter Parser

The tree-sitter adapter is tested against the entire XDK corpus (675+ `.x` files from `lib_*` directories).
The tree-sitter adapter is tested against the entire XDK corpus (692 `.x` files from `lib_*` directories).
This ensures the grammar can parse real-world Ecstasy code.

```bash
Expand Down Expand Up @@ -143,7 +145,7 @@ The sandbox IDE data is stored in `lang/intellij-plugin/build/idea-sandbox/`.

### Building a Distributable Plugin ZIP

To create a plugin ZIP that can be installed in any IntelliJ IDEA 2025.1+ instance:
To create a plugin ZIP that can be installed in any IntelliJ IDEA 2025.3+ instance:

```bash
./gradlew :lang:intellij-plugin:buildPlugin
Expand Down Expand Up @@ -250,7 +252,7 @@ The lang tooling uses several interdependent libraries. This section documents v

| Library | Version | Purpose | Notes |
|---------|---------|---------|-------|
| **lsp4j** | 0.21.1 | LSP protocol types & JSON-RPC | Eclipse's Java LSP implementation |
| **lsp4j** | 0.24.0 | LSP protocol types & JSON-RPC | Eclipse's Java LSP implementation |
| **lsp4ij** | 0.19.1 | IntelliJ LSP client plugin | Red Hat's plugin, uses lsp4j internally |

**How they work together:**
Expand All @@ -263,38 +265,38 @@ The lang tooling uses several interdependent libraries. This section documents v

| Library | Version | Purpose | Notes |
|---------|---------|---------|-------|
| **jtreesitter** | 0.24.1 | Java bindings for tree-sitter | JVM FFI to native tree-sitter |
| **tree-sitter-cli** | 0.24.3 | Parser generator CLI | Must match jtreesitter major.minor |
| **jtreesitter** | 0.26.0 | Java bindings for tree-sitter | JVM FFI to native tree-sitter |
| **tree-sitter-cli** | 0.26.5 | Parser generator CLI | Must match jtreesitter major.minor |

**Version Constraint (Java 21):**
**Version Constraint (Java 25):**
```
⚠️ jtreesitter 0.25+ requires Java 22
⚠️ jtreesitter 0.26+ requires Java 23
jtreesitter 0.24.x works with Java 21
⚠️ jtreesitter 0.26.x requires Java 23+ (FFM API)
✅ The LSP server runs OUT-OF-PROCESS with its own provisioned JRE (Java 25)
IntelliJ uses JBR 21, but the LSP server is a separate process
```

IntelliJ 2025.1 ships with JBR 21 (JetBrains Runtime = Java 21), so we must use jtreesitter 0.24.x
until IntelliJ ships with JBR 22+ (expected: IntelliJ 2026.x).
The LSP server runs as a separate out-of-process Java 25 process (provisioned via Foojay Disco API),
so the IntelliJ JBR 21 constraint does not limit the jtreesitter version.

Track JBR releases: https://github.com/JetBrains/JetBrainsRuntime/releases

### IntelliJ Platform

| Library | Version | Purpose |
|---------|---------|---------|
| **intellij-ide** | 2025.1 | Target IDE version |
| **intellij-ide** | 2025.3.2 | Target IDE version |
| **intellij-jdk** | 21 | Plugin JDK requirement |
| **intellij-platform-gradle-plugin** | 2.10.5 | Build plugin for IntelliJ plugins |
| **intellij-platform-gradle-plugin** | 2.11.0 | Build plugin for IntelliJ plugins |

### Compatibility Matrix

| Component | Requires | Provides |
|-----------|----------|----------|
| IntelliJ 2025.1 | JBR 21 | Plugin runtime |
| IntelliJ 2025.3.2 | JBR 21 | Plugin runtime |
| lsp4ij 0.19.1 | IntelliJ 2023.2+ | LSP client |
| lsp4j 0.21.1 | Java 11+ | LSP protocol |
| jtreesitter 0.24.1 | Java 21 | Native parsing |
| tree-sitter-cli 0.24.3 | - | Parser generation |
| lsp4j 0.24.0 | Java 11+ | LSP protocol |
| jtreesitter 0.26.0 | Java 23+ | Native parsing (runs in out-of-process LSP server) |
| tree-sitter-cli 0.26.5 | - | Parser generation |

All versions are defined in `/gradle/libs.versions.toml`.

Expand Down
2 changes: 1 addition & 1 deletion lang/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ val updateGeneratedExamples by tasks.registering(Copy::class) {
// =============================================================================

// Projects to aggregate standard lifecycle tasks from
val coreProjects = listOf(":dsl", ":tree-sitter", ":lsp-server", ":debug-adapter", ":intellij-plugin")
val coreProjects = listOf(":dsl", ":tree-sitter", ":lsp-server", ":dap-server", ":intellij-plugin")
val allProjects = coreProjects + ":vscode-extension"

// Map of aggregate task -> subproject task (null means same name)
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import java.lang.invoke.MethodHandles
/**
* Launcher for the XTC Debug Adapter Protocol (DAP) server.
*
* Usage: `java -jar xtc-debug-adapter.jar`
* Usage: `java -jar xtc-dap-server.jar`
*
* The DAP server uses stdio for communication. All logging goes to stderr
* to keep stdout clean for the JSON-RPC protocol.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
is reserved for JSON-RPC protocol messages. Any output to stdout
will corrupt the protocol stream.

Log file location: ~/.xtc/logs/debug-adapter.log
Tail with: tail -f ~/.xtc/logs/debug-adapter.log
Log file location: ~/.xtc/logs/dap-server.log
Tail with: tail -f ~/.xtc/logs/dap-server.log
-->
<configuration>
<!-- Suppress logback's own status messages -->
Expand All @@ -21,11 +21,11 @@
</encoder>
</appender>

<!-- Log to file for terminal tailing: tail -f ~/.xtc/logs/debug-adapter.log -->
<!-- Log to file for terminal tailing: tail -f ~/.xtc/logs/dap-server.log -->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${user.home}/.xtc/logs/debug-adapter.log</file>
<file>${user.home}/.xtc/logs/dap-server.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${user.home}/.xtc/logs/debug-adapter.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<fileNamePattern>${user.home}/.xtc/logs/dap-server.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<maxFileSize>10MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>50MB</totalSizeCap>
Expand All @@ -36,7 +36,10 @@
</appender>

<!-- XTC DAP server logs - INFO level by default -->
<logger name="org.xvm.debug" level="${xtc.logLevel:-INFO}"/>
<!-- Override via: -Plog=DEBUG, -Dxtc.logLevel=DEBUG, or XTC_LOG_LEVEL=DEBUG -->
<!-- Valid levels: TRACE, DEBUG, INFO, WARN, ERROR -->
<variable name="LOG_LEVEL" value="${xtc.logLevel:-${XTC_LOG_LEVEL:-INFO}}"/>
<logger name="org.xvm.debug" level="${LOG_LEVEL}"/>

<!-- Eclipse LSP4J - only errors -->
<logger name="org.eclipse.lsp4j" level="ERROR"/>
Expand Down
2 changes: 1 addition & 1 deletion lang/doc/LANGUAGE_SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -1949,7 +1949,7 @@ IDE-integrated debugging via DAP, leveraging existing console debugger infrastru
#### Deliverables

**5.1 DAP Adapter Implementation** ✨ *Primary Work*
- New module: `debug-adapter/`
- New module: `dap-server/`
- Implement DAP protocol (JSON-RPC)
- **Reuse existing**: `Debugger` interface and `DebugConsole` implementation
- Translate DAP requests to `DebugConsole` commands
Expand Down
Loading
Loading