Pulsar Operator: PulsarProxy is not able configure correct Liveness check for WebSocket

[mehmetsalgar]

Hi,

I am using the following CRD configuration for the PulsarProxy...

apiVersion: pulsar.streamnative.io/v1alpha1
kind: PulsarProxy
metadata:
  name: proxys
  namespace: pulsar
spec:
  pod:
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
  brokerAddress: brokers-broker
  replicas: 2
  config:
    custom:
      authorizationEnabled: "false"
    **tls:
      enabled: false**
  webSocketServiceEnabled: true
  dnsNames:
    []
  issuerRef:
    name: ""

This configuration uses 'http' port for liveness check for the pulsar-proxy but it uses the 'https' check on 'pulsar-proxy -websocket' I can't find any extra setting CRD to convince 'pulsar-operator-controller' to create the liveness check on http port.

This is what is created at StatefulSet

- name: pulsar-proxy-websocket
      image: >-
        apachepulsar/pulsar-all@sha256:6f6f4e0563ced8d9546b6dd009e345bc9959a6f0a60683b15e801db2b367704f
      command:
        - sh
        - '-c'
      args:
        - >-
          bin/apply-config-from-env.py conf/websocket.conf && echo 'OK' > status
          &&exec bin/pulsar websocket
      ports:
        - name: websocket
          containerPort: 9090
          protocol: TCP
        **- name: websocket-tls
          containerPort: 9443
          protocol: TCP**
      envFrom:
        - configMapRef:
            name: proxys-proxy-websocket-config
      resources:
        requests:
          cpu: 200m
          memory: 512Mi
      volumeMounts:
        - name: kube-api-access-fpt7z
          readOnly: true
          mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      livenessProbe:
        httpGet:
          path: /status.html
          **port: websocket-tls**
          scheme: HTTPS
        initialDelaySeconds: 30
        timeoutSeconds: 100
        periodSeconds: 10
        successThreshold: 1
        failureThreshold: 3
      readinessProbe:
        httpGet:
          path: /status.html
          **port: websocket-tls**
          scheme: HTTPS
        initialDelaySeconds: 30
        timeoutSeconds: 100
        periodSeconds: 10
        successThreshold: 1
        failureThreshold: 3

I could not find the code for the 'pulsar-operator-controller' may be there is a configuration parameter for it but I can' figure out.

I am in my test setup and I don't want to fight with the complexity TLS until proof of concept works, so is there a way to configure for Websocket container to use http port?

[ericsyh]

@mehmetsalgar Hi, after checking this issue, I think the implementation for the Proxy WebSocket is a little weird and different with the Broker WebSocket CR API.
You can try with this config to enable the WebSocket on Proxy and don't try Proxy WebSocket CR API.

apiVersion: pulsar.streamnative.io/v1alpha1
kind: PulsarProxy
metadata:
  name: proxys
  namespace: pulsar
spec:
  image: "streamnative/sn-platform-slim:2.10.3.4"
  pod:
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
  brokerAddress: brokers-broker
  replicas: 1
  config:
    custom:
      PULSAR_PREFIX_webSocketServiceEnabled: "true"

We may refactor and change the Proxy WebSocket CR API to keep the consistent with the Broker WebSocket implementation.

[yuweisung]

The workaround did activate the websocket service, but it did not add the 9090 port to proxy service.

[yuweisung]

The following config works.
Pre-requisite: cert-manager issuer. in my case, I have clusterissuer ca-issuer which can sign "s1.home.lab" certificate.

---
apiVersion: pulsar.streamnative.io/v1alpha1
kind: PulsarProxy
metadata:
  name: proxys
  namespace: pulsar
spec:
  image: streamnative/private-cloud:2.11.1.1
  replicas: 2
  brokerAddress: brokers-broker
  pod:
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
  configurationStoreServers: zookeepers-zk:2181
  dnsNames:
    - s1.home.lab
  webSocketServiceEnabled: true
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: ca-issuer
  config:
    tls:
      enabled: true
---