Skip to content

LCORE-497, LCORE-498, LCORE-499: Bundle most of ansible, assist-installer, default run.yaml dependencies, with pytorch and libs that depends on it. #292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tisnik merged 7 commits into from
Aug 18, 2025
Merged

LCORE-497, LCORE-498, LCORE-499: Bundle most of ansible, assist-installer, default run.yaml dependencies, with pytorch and libs that depends on it. #292

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d36cc67
LCORE-497: Bundle most of ansible dependencies, no sentence-transform…
matysek Jul 25, 2025
ea978df
LCORE-498: Add more dependencies required by the Assist-Installer
matysek Aug 13, 2025
fd0e2c2
LCORE-499: Add more dependencies required by the example llama-stack …
matysek Aug 14, 2025
5aa5643
LCORE-499: Fix pyproject.toml to install pytorch cpu variant
matysek Aug 18, 2025
f73fe71
LCORE-431: Add pytroch-cpu as dependency for running in library mode
matysek Aug 18, 2025
0c457fc
LCORE-499: Fix aarch64 build failure - autoevals dependency
matysek Aug 18, 2025
ae83068
LCORE-499: Move library mode dependency to a llslibdev group.
matysek Aug 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pylint.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ jobs:
with:
python-version: '3.12'
- name: Install dependencies
run: uv sync
run: uv sync --group llslibdev
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix: install dev tools (pylint) — use dev group, not llslibdev

llslibdev doesn’t include pylint; this step will fail to find pylint for uv run pylint. Install the dev group (which contains pylint). Also avoids pulling heavy ML deps unnecessarily into the linter job.

-        run: uv sync --group llslibdev
+        run: uv sync --group dev
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
run: uv sync --group llslibdev
run: uv sync --group dev
🤖 Prompt for AI Agents 
In .github/workflows/pylint.yaml around line 21, the workflow syncs the wrong
poetry group (llslibdev) which doesn't include pylint; change the command to
install the dev group instead by replacing the uv sync --group llslibdev
invocation with uv sync --group dev so pylint (and other dev tools) are
installed for the linter job and heavy ML deps from the llslibdev group are not
pulled into this job.

- name: Python linter
run: uv run pylint src tests
2 changes: 1 addition & 1 deletion .github/workflows/pyright.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ jobs:
with:
python-version: '3.12'
- name: Install dependencies
run: uv sync
run: uv sync --group llslibdev
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix: install dev tools (pyright) — use dev group, not llslibdev

pyright is in the dev group; installing only llslibdev will not provide the pyright CLI and pulls in heavy ML deps unnecessarily.

-        run: uv sync --group llslibdev
+        run: uv sync --group dev
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
run: uv sync --group llslibdev
run: uv sync --group dev
🤖 Prompt for AI Agents 
.github/workflows/pyright.yaml around line 21: the workflow installs the wrong
dependency group ("llslibdev") which does not provide the pyright CLI and pulls
in heavy ML deps; change the group to "dev" so the pyright CLI is installed
(i.e., run the uv sync command with the dev group) and remove reliance on the
llslibdev group to avoid unnecessary dependencies.

- name: Run Pyright tests
run: uv run pyright src
2 changes: 1 addition & 1 deletion .github/workflows/unit_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- name: uv version
run: uv --version
- name: Install dependencies
run: uv pip install --python ${{ matrix.python-version }} -e .
run: uv pip install --python ${{ matrix.python-version }} --group llslibdev .
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Broken install command: uv pip install --group isn’t a valid flag — use uv sync with groups

uv pip install doesn’t support dependency groups; this will error. Use uv sync to install base + selected groups and the project (default behavior).

-        run: uv pip install --python ${{ matrix.python-version }} --group llslibdev .
+        run: uv sync --group dev --group llslibdev

Note: If your unit tests don’t need the llslibdev stack, prefer --group dev only to keep CI lean.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
run: uv pip install --python ${{ matrix.python-version }} --group llslibdev .
run: uv sync --group dev --group llslibdev
🤖 Prompt for AI Agents 
.github/workflows/unit_tests.yaml around line 27: the workflow uses an invalid
command flag "uv pip install --group" which will fail because uv's "pip install"
subcommand doesn't accept groups; replace this with "uv sync" to install the
project plus dependency groups (e.g., uv sync --python ${{ matrix.python-version
}} --group llslibdev .), or if tests only need dev deps use uv sync --group dev
to keep CI lean.

- name: Install pdm # Required for dynamic version test
run: uv pip install pdm
- name: Run unit tests
Expand Down
18 changes: 14 additions & 4 deletions Containerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# vim: set filetype=dockerfile
FROM registry.access.redhat.com/ubi9/python-312-minimal AS builder
FROM registry.access.redhat.com/ubi9/python-312 AS builder

ARG APP_ROOT=/app-root
ARG LSC_SOURCE_DIR=.
Expand All @@ -11,16 +11,21 @@ ENV UV_COMPILE_BYTECODE=0 \

WORKDIR /app-root

# Install gcc - required by polyleven python package on aarch64
# (dependency of autoevals, no pre-built binary wheels for linux on aarch64)
USER root
RUN dnf install -y --nodocs --setopt=keepcache=0 --setopt=tsflags=nodocs gcc

# Install uv package manager
RUN pip3.12 install uv
RUN pip3.12 install "uv==0.8.11"

# Add explicit files and directories
# (avoid accidental inclusion of local directories or env files or credentials)
COPY ${LSC_SOURCE_DIR}/src ./src
COPY ${LSC_SOURCE_DIR}/pyproject.toml ${LSC_SOURCE_DIR}/LICENSE ${LSC_SOURCE_DIR}/README.md ${LSC_SOURCE_DIR}/uv.lock ./

RUN uv sync --locked --no-dev

# Bundle additional dependencies for library mode.
RUN uv sync --locked --no-dev --group llslibdev
Comment on lines +27 to +28
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Contradiction with PR objective: this installs PyTorch into the image

uv sync --locked --no-dev --group llslibdev pulls the entire llslibdev group, which currently includes:

  • torch==2.7.1
  • peft>=0.15.2
  • trl>=0.18.2

This contradicts the stated PR goal: “no pytorch and libs that depends on it”.

Please either:

  • Remove torch, peft, trl from llslibdev and move them to a new optional group (e.g., ml-train) that is not installed here, or
  • Stop installing the llslibdev group in this base image and create a separate image or build target for the ML training stack.

Option A (keep llslibdev, exclude ML training libs from base):

-RUN uv sync --locked --no-dev --group llslibdev
+RUN uv sync --locked --no-dev

Then in pyproject.toml (see separate comment), split the group: keep non-PyTorch deps in llslibdev, move torch, peft, trl into a new group (e.g., ml-train) installed only in derived images.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Bundle additional dependencies for library mode.
RUN uv sync --locked --no-dev --group llslibdev
# Bundle additional dependencies for library mode.
RUN uv sync --locked --no-dev
🤖 Prompt for AI Agents 
In Containerfile around lines 27-28, running "RUN uv sync --locked --no-dev
--group llslibdev" pulls the llslibdev group which currently includes torch,
peft and trl and therefore installs PyTorch contrary to the PR goal; either (A)
modify pyproject.toml to remove torch, peft and trl from the llslibdev group and
add them to a new optional group (e.g., ml-train) so the base image keeps only
non-ML deps and continue to sync llslibdev here, or (B) stop syncing the
llslibdev group in this base Containerfile and instead create a separate build
stage or derived image that runs uv sync for an ML training image which installs
an ml-train group; update the Containerfile accordingly (remove or change the uv
sync line in the base image and add a separate target/stage or image that
installs the ML group) and update pyproject.toml to reflect the new ml-train
group if you choose option A.


# Final image without uv package manager
FROM registry.access.redhat.com/ubi9/python-312-minimal
Expand All @@ -42,6 +47,11 @@ COPY --from=builder --chown=1001:1001 /app-root /app-root
# this directory is checked by ecosystem-cert-preflight-checks task in Konflux
COPY --from=builder /app-root/LICENSE /licenses/

# Add uv to final image for derived images to add additional dependencies
# with command:
# $ uv pip install <dependency>
RUN pip3.12 install "uv==0.8.11"

# Add executables from .venv to system PATH
ENV PATH="/app-root/.venv/bin:$PATH"

Expand Down
64 changes: 50 additions & 14 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ requires-python = ">=3.12,<3.14"
readme = "README.md"
license = {file = "LICENSE"}
dependencies = [
"fastapi>=0.115.6",
"fastapi>=0.115.12",
"uvicorn>=0.34.3",
"kubernetes>=30.1.0",
"llama-stack==0.2.17",
Expand All @@ -33,11 +33,12 @@ dependencies = [
"starlette>=0.47.1",
"aiohttp>=3.12.14",
"authlib>=1.6.0",
"openai==1.99.1",
"sqlalchemy>=2.0.42",
"email-validator>=2.2.0",
"openai==1.99.9",
"sqlalchemy>=2.0.42",
]


[tool.pyright]
exclude = [
# TODO(lucasagomes): This module was copied from road-core
Expand All @@ -58,6 +59,15 @@ path = "src/version.py"
Homepage = "https://github.com/lightspeed-core/lightspeed-stack"
Issues = "https://github.com/lightspeed-core/lightspeed-stack/issues"

# PyTorch has multiple wheel variants for different backends - cpu, gpu, etc.
# By default on pypi.org is the gpu variant. Forces uv to use the cpu variant.
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = [{ index = "pytorch-cpu" }]

[dependency-groups]
dev = [
"black>=25.1.0",
Expand All @@ -80,23 +90,49 @@ dev = [
"openapi-to-md>=0.1.0b2",
]
llslibdev = [
"fastapi>=0.115.12",
"opentelemetry-sdk>=1.34.0",
"opentelemetry-exporter-otlp>=1.34.0",
"opentelemetry-instrumentation>=0.55b0",
# To check llama-stack API provider dependecies:
#
# $ uv run llama stack list-providers
#
# API agents: inline::meta-reference
"matplotlib>=3.10.0",
"pillow>=11.1.0",
"pandas>=2.2.3",
"scikit-learn>=1.5.2",
"psycopg2-binary>=2.9.10",
# API eval: inline::meta-reference
"tree_sitter>=0.24.0",
"pythainlp>=3.0.10",
"langdetect>=1.0.9",
"emoji>=2.1.0",
"nltk>=3.8.1",
# API inference: remote::gemini
"litellm>=1.75.5.post1",
# API vector_io: inline::faiss
"faiss-cpu>=1.11.0",
# API scoring: inline::basic
"requests>=2.32.4",
# API datasetio: inline::localfs
"aiosqlite>=0.21.0",
"litellm>=1.72.1",
"uvicorn>=0.34.3",
"blobfile>=3.0.0",
# API datasetio: remote::huggingface
"datasets>=3.6.0",
"sqlalchemy>=2.0.41",
"faiss-cpu>=1.11.0",
# API telemetry: inline::meta-reference
"opentelemetry-sdk>=1.34.1",
"opentelemetry-exporter-otlp>=1.34.1",
# API tool_runtime: inline::rag-runtime
"transformers>=4.34.0",
"numpy==2.2.6",
# API tool_runtime: remote::model-context-protocol
"mcp>=1.9.4",
# Other
"autoevals>=0.0.129",
"psutil>=7.0.0",
"torch>=2.7.1",
"torch==2.7.1",
"peft>=0.15.2",
"trl>=0.18.2",
"fire>=0.7.0",
"opentelemetry-instrumentation>=0.55b0",
"blobfile>=3.0.0",
"psutil>=7.0.0",
]

build = [
Expand Down
18 changes: 1 addition & 17 deletions test.containerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,22 +23,6 @@ RUN uv -h
# Install from pyproject.toml using uv sync for version consistency
# Note: using --no-install-project since this is llama-stack container, not lightspeed-stack
# Include dev deps for testing (pytest, behave, etc.)
RUN uv sync --locked --no-install-project && \
uv pip install \
opentelemetry-sdk \
opentelemetry-exporter-otlp \
opentelemetry-instrumentation \
aiosqlite \
litellm \
blobfile \
datasets \
sqlalchemy \
faiss-cpu \
mcp \
autoevals \
psutil \
torch \
peft \
trl
RUN uv sync --locked --no-install-project --group dev --group llslibdev

CMD ["uv", "run", "llama", "stack", "run", "run.yaml"]
Loading