Add --max-parallel-count flag to control MSBuild and Docker build parallelism during deployment

[Copilot]

Ticket

#4405

Brief Description

Deploying many services causes excessive RAM consumption. Root cause: MSBuild runs unbounded parallel project builds during solution compilation, and Docker builds run in parallel without limits. Added --max-parallel-count flag (default: 8) that controls parallelism at two levels: passes -m:{count} to MSBuild to limit concurrent build nodes, and throttles concurrent Docker image builds.

Changes

• BuildSolutionCommand: Accepts maxParallelCount parameter, passes -m:{count} to dotnet publish with documentation explaining the flag
• DeploymentService:
  • Threads maxParallelCount from command args to BuildSolutionCommand.Build()
  • Throttles Docker builds to respect maxParallelCount regardless of sequential build flag
  • Emits progress reports at zero for all services before throttling
• CLI Options: Added --max-parallel-count flag to beam deploy plan and beam deploy release
• Tests: Added MaxParallelCountTests.cs with 3 NUnit tests following existing test standards
• .gitignore: Fixed to exclude unityenginestubs.addressables/bin and /obj (lowercase variant)

Usage

beam deploy plan --max-parallel-count=5

Limits both MSBuild to 5 parallel compilation nodes and Docker builds to 5 concurrent operations, reducing peak memory usage proportionally.

Implementation Details

The -m flag is the short form of -maxCpuCount in MSBuild (see dotnet msbuild --help). The throttling mechanism uses Task.WhenAny to wait for completion when concurrent limits are reached, ensuring smooth progress reporting while preventing memory exhaustion.

Checklist

• Have you added appropriate text to the CHANGELOG.md files?
• Have you documented all your public methods and interfaces?

Notes

The implementation now controls parallelism at two critical points:

1. MSBuild compilation: Prevents excessive parallel C# project builds
2. Docker image creation: Prevents excessive parallel Docker build processes

Both contribute significantly to memory usage when deploying many services.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[cdhanna]

1. add tests that use the existing standard
2. I was wrong before, we also need to limit the docker build as well. We can re-use the same flag, but regardless of sequential builds, it should make sure only so many docker builds are allowed to trigger at once. Ideally, it would emit the progress report for all them as zero, but then limit them to only run given the max parallel option.

[cdhanna]

where did this -m flag come from? I don't see that in the docs. Check env variables as well.

[Copilot]

The -m flag is the short form of -maxCpuCount in MSBuild. From dotnet msbuild --help:

-maxCpuCount[:n]   Specifies the maximum number of concurrent processes to
                   build with. (Short form: -m[:n])

I've added documentation comments in the code (commit ac492f4) explaining this.

[Copilot]

1. add tests that use the existing standard
2. I was wrong before, we also need to limit the docker build as well. We can re-use the same flag, but regardless of sequential builds, it should make sure...

Done in commit ac492f4:

1. Added tests following existing NUnit standard in MaxParallelCountTests.cs (3 tests, all pass)
2. Added Docker build throttling in CreateReleaseManifestFromLocal - progress reports are emitted at zero for all services, then Docker builds are throttled to respect maxParallelCount regardless of the useSequentialBuild flag

The throttling now works at both levels: MSBuild compilation parallelism and Docker build parallelism.