action-server v2.16.1 fails during Docker build - RCC micromamba removal error

[joshyorko]

Environment

• action-server version: 2.16.1 (works in 2.14.0)
• Platform: Docker (debian:bookworm-slim base image)
• RCC version: 20.3.2 (auto-downloaded by action-server)
• Build context: Fresh Docker container build

Description

When building a Docker image with action-server v2.16.1, the action-server import --datadir=/action-server/datadir command fails during RCC environment bootstrapping with a micromamba removal error. The exact same Dockerfile works perfectly with version 2.14.0.

Error Output

RCC not available at: /home/as-user/.sema4ai/bin/action-server/internal/2.16.1/_internal/sema4ai/action_server/bin/rcc-20.3.2. Downloading.
Downloading 'https://cdn.sema4.ai/rcc/releases/v20.3.2/linux64/rcc' to '/home/as-user/.sema4ai/bin/action-server/internal/2.16.1/_internal/sema4ai/action_server/bin/rcc-20.3.2'
Download attempt #1
File successfully downloaded

  ⚡️ Starting Action Server... v2.16.1

Using user-specified datadir: /action-server/datadir
Logs may be found at: /action-server/datadir/server_log.txt.
Database file does not exist. Creating it at: /action-server/datadir/server.db
Action package seems ok. Bootstrapping RCC environment (please wait, this can take a long time).
Error running: /home/as-user/.sema4ai/bin/action-server/internal/2.16.1/_internal/sema4ai/action_server/bin/rcc-20.3.2 ht hash /action-server/actions/package.yaml --silent --no-temp-management --warranty-voided --controller action-server --bundled --sema4ai.
SEMA4AI_HOME: <unset>

Stdout: 
Stderr: Fatal [Remove of micromamba failed, reason:]: remove /home/as-user/.sema4ai/micromamba/v1.5.11_/micromamba: no such file or directory
Fatal [Remove of micromamba failed, reason:]: remove /home/as-user/.sema4ai/micromamba/v1.5.11_/micromamba: no such file or directory
Fatal [Remove of micromamba failed, reason:]: remove /home/as-user/.sema4ai/micromamba/v1.5.11_/micromamba: no such file or directory

The command exits with code 113, causing the Docker build to fail.

Root Cause Analysis

RCC v20.3.2 (bundled with action-server 2.16.1) is attempting to remove a micromamba binary at /home/as-user/.sema4ai/micromamba/v1.5.11_/micromamba before initializing the environment. In a fresh Docker container, this directory structure doesn't exist yet, causing the removal to fail.

This appears to be a cleanup/initialization step that:

1. Works locally where .sema4ai persists between runs
2. Fails in Docker where the container starts with a clean filesystem
3. Worked in v2.14.0 (which likely had different cleanup logic or proper error handling)

Why This Is A Bug

• Regression: Functionality that worked in v2.14.0 is broken in v2.16.1
• Missing error handling: RCC should either:
  • Check if the file exists before trying to remove it (rm -f behavior)
  • Handle "file not found" gracefully
  • Create the directory structure before attempting cleanup
• Critical use case: Docker deployments are a standard deployment pattern

Reproduction

Dockerfile snippet:

FROM debian:bookworm-slim AS builder
RUN apt-get update && apt-get install -y wget curl ca-certificates && rm -rf /var/lib/apt/lists/*
ADD https://cdn.sema4.ai/action-server/releases/2.16.1/linux64/action-server /usr/local/bin/action-server
RUN chmod +x /usr/local/bin/action-server

FROM debian:bookworm-slim
# ... install dependencies ...
COPY --from=builder /usr/local/bin/action-server /usr/local/bin/action-server
RUN useradd -m as-user
RUN mkdir -p /action-server/datadir /action-server/actions && chown -R as-user:as-user /action-server
WORKDIR /action-server/actions
COPY . .
USER as-user
RUN action-server import --datadir=/action-server/datadir  # FAILS HERE

Build with: docker build --no-cache .

Expected Behavior

action-server v2.16.1 should initialize successfully in a fresh Docker container, as it does in v2.14.0.

Current Workaround

Pin to version 2.14.0:

ADD https://cdn.sema4.ai/action-server/releases/2.14.0/linux64/action-server /usr/local/bin/action-server

Impact

• Breaks Docker-based CI/CD pipelines
• Prevents users from upgrading to the latest version
• Affects production deployments using containers

Full reproduction repository available if needed.