Skip to content

[Spark] Support spark-operator on multi-namespace deployments#258

Draft
alxtkr77 wants to merge 1 commit intomlrun:developmentfrom
alxtkr77:CEML-540
Draft

[Spark] Support spark-operator on multi-namespace deployments#258
alxtkr77 wants to merge 1 commit intomlrun:developmentfrom
alxtkr77:CEML-540

Conversation

@alxtkr77
Copy link
Member

Summary

  • Admin namespace holds CRDs + ClusterRole with no running pods (replicas=0), each user namespace runs its own spark-operator controller with a namespace-scoped RoleBinding to the shared ClusterRole
  • Adds CE-level spark config section to decouple ConfigMap rendering from subchart enablement
  • Adds spark-controller-rbac.yaml CE template that creates RoleBinding → ClusterRole + leader election RBAC in user namespaces
  • Updates admin/non-admin/cluster-ip values files with spark-operator split configuration
  • Adds multi-ns Kind test command validating admin + 2 user namespaces with RBAC and functional SparkApplication checks
  • Bumps chart version to 0.11.0-rc9

Test plan

  • helm lint passes
  • tests/kind-test.sh multi-ns — admin + 2 user namespaces, all RBAC checks pass, SparkApplication processed by controller in both user namespaces
  • tests/kind-test.sh full — single-NS backward compatibility (no behavior change)
  • Standard installation with admin_installation_values.yaml + non_admin_installation_values.yaml verified on Kind

Admin namespace holds CRDs + ClusterRole (no running pods), each user
namespace runs its own spark-operator controller with a namespace-scoped
RoleBinding to the shared ClusterRole.

- Add CE-level `spark` config section to decouple ConfigMap from subchart
- Add spark-controller-rbac.yaml template for user NS RBAC gap
- Update admin/non-admin values files with spark-operator split config
- Add `multi-ns` command to Kind test with 2 user namespace validation
- Bump chart version to 0.11.0-rc9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments