Skip to content

Comments

updated docker overrides#1381

Open
kheiss-uwzoo wants to merge 7 commits intoNVIDIA:mainfrom
kheiss-uwzoo:kheiss/docker-overwrite
Open

updated docker overrides#1381
kheiss-uwzoo wants to merge 7 commits intoNVIDIA:mainfrom
kheiss-uwzoo:kheiss/docker-overwrite

Conversation

@kheiss-uwzoo
Copy link
Collaborator

Summary

  1. Tip in Step 1f (Start core services)

Tip that explains overrides in one sentence and links to the new section:
"Use a docker compose override file to reduce VRAM usage. Override files typically lower per-service memory allocation, batch sizes, or concurrency..."

  1. New section: "Docker Compose override files"
  • Added a dedicated section (after Profile Information, before Specify MIG slices) that:
  • Explains overrides: When default compose exceeds VRAM, override files reduce memory/batch size/concurrency so the pipeline fits on one GPU.
  • Explains usage: Use a second -f with the override file; Docker Compose merges and the override wins.
  • Lists all current override files in a table:
Override file GPU target
docker-compose.a10g.yaml NVIDIA A10G
docker-compose.a100-40gb.yaml NVIDIA A100-SXM4-40GB
docker-compose.l40s.yaml NVIDIA L40S
  • Covers RTX Pro 6000 Server Edition: Notes that for this GPU and others with limited VRAM, users should pick the override that best matches GPU memory (e.g. docker-compose.l40s.yaml or docker-compose.a10g.yaml). (There is no docker-compose.rtx-pro-6000.yaml in the repo.)

Adds examples for A100 40GB, A10G, and L40S.

@kheiss-uwzoo kheiss-uwzoo requested a review from a team as a code owner February 5, 2026 22:21
@kheiss-uwzoo kheiss-uwzoo added the doc Improvements or additions to documentation label Feb 5, 2026
Copy link
Collaborator

@nkmcalli nkmcalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions.

--profile retrieval up
```

**Example — A10G:**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here - please adjust and add an intro sentence

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated per comment

--profile retrieval up
```

**Example — L40S:**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here - please adjust and add an intro sentence

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated per review comment

kheiss-uwzoo and others added 3 commits February 11, 2026 17:41
Co-authored-by: nkmcalli <nkmcalli@yahoo.com>
Co-authored-by: nkmcalli <nkmcalli@yahoo.com>
Updated per review comments
Copy link
Collaborator Author

@kheiss-uwzoo kheiss-uwzoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated file per review comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants