Intermittent SSH connection failures to Gitea via Traefik on K3s

[MarcRibasG]

I'm a little desperate and I don't know where I can find help.

Problem Description:

I am experiencing intermittent connection issues when trying to perform Git operations (e.g., git clone) over SSH to a Gitea instance running in my K3s cluster. The Gitea instance is exposed via Traefik using an SSH entrypoint on port 2222.

Sometimes, commands like git clone git@git.example.com:marc.ribas/test.git fail with:

Cloning into 'test'...
kex_exchange_identification: Connection closed by remote host
Connection closed by x.x.x.x port 2222
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

When this happens, I see corresponding errors in my Traefik logs:

2025-05-21T11:20:04Z ERR Error while handling TCP connection error="writeto tcp 10.42.0.228:44464->10.42.2.244:22: read tcp 10.42.0.228:44464->10.42.2.244:22: read: connection reset by peer"

(Note: The Gitea pod IP like 10.42.2.244 can change if the pod restarts; the issue persists across different Gitea pod IPs.)

And some gitea log to higlight: Timeout before authentication for connection from 10.42.0.228 to 10.42.2.244, pid = 1272

The issue is intermittent. Sometimes connections work fine, other times they fail with the reset by peer from Traefik. HTTP/S access to Gitea (also via Traefik) seems stable.

Environment Details:

Kubernetes: K3s (I'm using Traefik v3 deployed via Helm)
Master Node (core-1): K3s v1.32.4+k3s1
Worker Node (code-1): K3s v1.31.4+k3s1
Cloud Provider: Hetzner Cloud
Ingress Controller: Traefik (deployed via HelmChartConfig, v3 )
Traefik pod runs on node core-1.
Application: Gitea gitea/gitea:1.23.8
Gitea pod runs on node code-1.

My configuration:

Traefik HelmChartConfig:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    installCRDs: true
    logs:
      level: DEBUG # General log level for Traefik

    entryPoints:
      web:
        address: ":80"
      websecure:
        address: ":443"
        # ... (TLS config) ...
      ssh: # SSH Entrypoint for Gitea
        address: ":2222/tcp"
        transport:
          respondingTimeouts:
            readTimeout: 600s
            writeTimeout: 600s
            idleTimeout: 7200s

    ports:
      # ... (web, websecure ports) ...
      ssh: # Exposing port 2222 on the Traefik host
        port: 2222
        protocol: TCP
        targetPort: 2222 # Internal port for the ssh entrypoint in Traefik container
        expose:
          default: true
    # ... (persistence, certresolvers, probes) ...
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                  - core-1

Gitea Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: gitea
  name: gitea
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitea
  template:
    metadata:
      labels:
        app: gitea
    spec:
      containers:
        - name: gitea
          image: gitea/gitea:1.23.8
          ports:
            - containerPort: 3000 # HTTP
            - containerPort: 22   # SSH
          env:
            - name: GITEA__log__LEVEL
              value: 'Debug'
            # ... (other USER_UID, APP_NAME env vars) ...
            - name: GITEA__server__DOMAIN
              value: 'git.example.com'
            - name: GITEA__server__ROOT_URL
              value: 'https://git.example.com/'
            - name: GITEA__server__SSH_PORT # Port Gitea's internal SSH server listens on
              value: '22'
            - name: GITEA__server__START_SSH_SERVER
              value: 'true'
            - name: GITEA__server__SSH_LISTEN_PORT # Port Gitea advertises in SSH URLs
              value: '2222'
            - name: GITEA__server__SSH_LOG_LEVEL
              value: 'DEBUG'
            # ... (other Gitea env vars) ...
          # ... (volumeMounts, probes) ...
      # ... (volumes) ...
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/hostname
                operator: In
                values:
                  - code-1

apiVersion: v1
kind: Service
metadata:
  name: gitea-ssh
  namespace: gitea
spec:
  selector:
    app: gitea
  ports:
    - name: ssh
      port: 22    
      targetPort: 22 
      protocol: TCP

apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: gitea-ssh
  namespace: gitea
spec:
  entryPoints:
    - ssh
  routes:
    - match: HostSNI(`*`) 
      services:
        - name: gitea-ssh 
          port: 22

I really don't know how to start solving this intermittent problem with the SSH connection when doing git commands. Thank you very much in advance.

[vitobotta]

Traefik can be a bit buggy with raw TCP. I remember facing issues with it in the past, and I've seen similar discussions in various places. I no longer use Traefik myself. It might be better to use a load balancer for Gitea's SSH access.