Skip to content

TEST: Add ec2 kfp runner and enable docling-vlm local_run.py(#74)#75

Closed
shruthis4 wants to merge 15 commits intomainfrom
KFPEC2
Closed

TEST: Add ec2 kfp runner and enable docling-vlm local_run.py(#74)#75
shruthis4 wants to merge 15 commits intomainfrom
KFPEC2

Conversation

@shruthis4
Copy link
Contributor

  • Add CI for KFP local runner

  • Set ocr to False

  • filter vlm runner

  • Change OCR option to use 'rapidocr'

  • Update OCR parameters in local_run.py

  • Add option to run from branch or pr

  • Remove OCR options from local_run.py

Removed OCR options from docling_convert_standard call.

  • Change permissions of local_outputs directory

  • trigger on workflow edits too

  • Update base container images for Docling Pipelines

  • Add OCR option to docling_convert_standard call

  • Update base image references in constants.py

  • Pin kfp version to 2.14.6 in workflow

  • Update base container images for Kubeflow Pipelines

  • Configure Quay registry login in workflow

Added environment variable for Quay registry and login step.

  • Configure Quay registry login in workflow

Added environment variable for Quay registry and login step.

  • Remove empty new line

  • Refactor GitHub Actions workflow to launch EC2 runner and update container images

  • Introduced a new job to launch an EC2 runner for data processing.
  • Updated the container images for both standard and VLM pipelines to use quay.io/aipcc/docling/cuda-ubi9.
  • Enhanced the test-local-pipelines job to depend on the EC2 runner and adjusted Python setup steps.
  • Added a job to stop the EC2 runner after the pipeline execution.

Description

How Has This Been Tested?

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

* Add CI for KFP local runner

* Set ocr to False

* filter vlm runner

* Change OCR option to use 'rapidocr'

* Update OCR parameters in local_run.py

* Add option to run from branch or pr

* Remove OCR options from local_run.py

Removed OCR options from docling_convert_standard call.

* Change permissions of local_outputs directory

* trigger on workflow edits too

* Update base container images for Docling Pipelines

* Add OCR option to docling_convert_standard call

* Update base image references in constants.py

* Pin kfp version to 2.14.6 in workflow

* Update base container images for Kubeflow Pipelines

* Configure Quay registry login in workflow

Added environment variable for Quay registry and login step.

* Configure Quay registry login in workflow

Added environment variable for Quay registry and login step.

* Remove empty new line

* Refactor GitHub Actions workflow to launch EC2 runner and update container images

- Introduced a new job to launch an EC2 runner for data processing.
- Updated the container images for both standard and VLM pipelines to use `quay.io/aipcc/docling/cuda-ubi9`.
- Enhanced the test-local-pipelines job to depend on the EC2 runner and adjusted Python setup steps.
- Added a job to stop the EC2 runner after the pipeline execution.
@coderabbitai
Copy link

coderabbitai bot commented Dec 20, 2025

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch KFPEC2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

shruthis4 and others added 10 commits December 20, 2025 18:45
Change pip installation method to ensurepip before upgrading.
Added Docker setup steps to the workflow.
Updated the setup workflow to include Podman installation and configuration alongside Python.
Updated Podman socket management for improved compatibility.
Updated Podman socket handling and permissions for compatibility.
Updated setup for Docker instead of Podman in CI workflow.
Updated user group management for Docker access.
@shruthis4 shruthis4 changed the title TEST: Add ec2 kfp runner (#74) TEST: Add ec2 kfp runner and enable docling-vlm local_run.py(#74) Dec 21, 2025
shruthis4 and others added 4 commits December 22, 2025 13:19
Enhanced the execute-kfp-localrunners workflow to allow overriding of PYTHON_BASE_IMAGE and DOCLING_BASE_IMAGE through input parameters. Updated constants to use environment variables for base images.
…g pipelines to use quay.io/amaredia/aipcc-docling-image
@mergify
Copy link

mergify bot commented Dec 22, 2025

🔀 Merge Conflict Detected

This pull request has merge conflicts that must be resolved before it can be merged.
@shruthis4 please rebase your branch.

Syncing a Fork Guide

@shruthis4 shruthis4 closed this Dec 23, 2025
@shruthis4 shruthis4 deleted the KFPEC2 branch December 23, 2025 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments