HARVESTER CLAIM

[patrick-hermann-sva]

cat /tmp/harvester-dev7.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dev7-disk-0
  namespace: default
  annotations:
    harvesterhci.io/imageId: default/image-t9w92
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: '20Gi'
  storageClassName: longhorn-image-t9w92
  volumeMode: Block
---
apiVersion: v1
kind: Secret
metadata:
  name: dev7-cloud-init
  namespace: default
type: Secret
stringData:
  networkdata: ''
  userdata: |
    #cloud-config
    hostname: dev7
    ssh_pwauth: true
    users:
      - name: sthings
        sudo: ALL=(ALL) NOPASSWD:ALL
        lock_passwd: false
        shell: /bin/bash
        ssh_authorized_keys:
          - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...
    # Set password for user
    chpasswd:
      list: |
        sthings:Atlan7is
      expire: false
    package_update: true
    packages:
      - qemu-guest-agent
    runcmd:
      - - systemctl
        - enable
        - --now
        - qemu-guest-agent.service
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: dev7
  namespace: default
  labels:
    os: linux
  annotations:
    description: dev7-complete-vm-setup
spec:
  runStrategy: RerunOnFailure
  template:
    metadata:
      labels:
        vmName: dev7
    spec:
      hostname: dev7
      domain:
        machine:
          type: q35
        cpu:
          cores: 4
          sockets: 1
          threads: 1
        resources:
          limits:
            memory: '8Gi'
            cpu: '4'
        devices:
          disks:
          - name: disk-0
            disk:
              bus: virtio
            bootOrder: 1
          - name: cloudinitdisk
            disk:
              bus: virtio
          interfaces:
          - name: default
            bridge: {}
            model: virtio
          inputs:
          - name: tablet
            type: tablet
            bus: usb
        features:
          acpi:
            enabled: true
      evictionStrategy: LiveMigrateIfPossible
      networks:
      - name: default
        multus:
          networkName: default/vms
      volumes:
      - name: disk-0
        persistentVolumeClaim:
          claimName: dev7-disk-0
      - name: cloudinitdisk
        cloudInitNoCloud:
          secretRef:
            name: dev7-cloud-init
          networkDataSecretRef:
            name: dev7-cloud-init
      terminationGracePeriodSeconds: 120

[patrick-hermann-sva]

apiVersion: resources.stuttgart-things.com/v1alpha1
kind: HarvesterVM
metadata:
  name: dev9
  namespace: default
spec:
  providerConfigRef: "default"  # Update this to match your ProviderConfig name
  
  volume:
    pvcName: dev9-disk-0
    namespace: default
    storageClassName: longhorn-image-t9w92
    storage: "20Gi"
    accessModes:
      - ReadWriteMany
    volumeMode: Block
    annotations:
      harvesterhci.io/imageId: "default/image-t9w92"
  
  cloudInit:
    vmName: dev9
    namespace: default
    hostname: dev9
    domain: local
    sshPasswordAuth: true
    disableRoot: false
    packageUpdate: true
    users:
      - name: sthings
        sudo: "ALL=(ALL) NOPASSWD:ALL"
        lockPasswd: false
        shell: /bin/bash
        password: "Atlan7is"
        sshAuthorizedKeys:
          - "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..."
    packages:
      - qemu-guest-agent
    runcmd:
      - "systemctl enable --now qemu-guest-agent.service"
  
  vm:
    description: "dev9-complete-vm-setup"
    runStrategy: RerunOnFailure
    evictionStrategy: LiveMigrateIfPossible
    terminationGracePeriodSeconds: 120
    os: linux
    machineType: q35
    cpu:
      cores: 4
      sockets: 1
      threads: 1
    resources:
      memory: "8Gi"
      cpu: "4"
    disks:
      - name: disk-0
        bus: virtio
        bootOrder: 1
    networks:
      - name: default
        networkName: default/vms

[patrick-hermann-sva]

crossplane beta trace harvestervm.resources.stuttgart-things.com/dev9
NAME SYNCED READY STATUS
HarvesterVM/dev9 (default) False - ReconcileError: ...Kind=Object): .spec.deletionPolicy: field not declared in schema
├─ Object/dev9-d8d6f42b6b29 (default) - - Error: objects.kubernetes.m.crossplane.io "dev9-d8d6f42b6b29" not found
├─ CloudInit/dev9 (default) True True Available
│ └─ Object/dev9-c2028c20e0a4 (default) True True Available
└─ VolumeClaim/dev9 (default) True True Available
└─ Object/dev9-1046255d6944 (default) True True Available

[patrick-hermann-sva]

kubectl describe objects.kubernetes.m.crossplane.io dev9-5909024719b4
Name: dev9-5909024719b4
Namespace: default
Labels: app.kubernetes.io/instance=dev9
app.kubernetes.io/managed-by=crossplane
app.kubernetes.io/name=harvester-vm
crossplane.io/composite=dev9
Annotations: crossplane.io/composition-resource-name: dev9-vm
crossplane.io/external-create-failed: 2026-02-03T21:21:07Z
crossplane.io/external-create-pending: 2026-02-03T21:21:07Z
crossplane.io/external-name: dev9-5909024719b4
API Version: kubernetes.m.crossplane.io/v1alpha1
Kind: Object
Metadata:
Creation Timestamp: 2026-02-03T21:19:06Z
Finalizers:
finalizer.managedresource.crossplane.io
Generate Name: dev9-
Generation: 2
Owner References:
API Version: resources.stuttgart-things.com/v1alpha1
Block Owner Deletion: true
Controller: true
Kind: HarvesterVM
Name: dev9
UID: a5b3d2ca-5e21-43f0-9144-e443e6c2de03
Resource Version: 234176
UID: a62866f1-d3cc-466c-a2d5-32db87ee2a66
Spec:
For Provider:
Manifest:
API Version: kubevirt.io/v1
Kind: VirtualMachine
Metadata:
Annotations:
Description: dev9-complete-vm-setup
Labels:
Os: linux
Name: dev9
Namespace: default
Spec:
Run Strategy: RerunOnFailure
Template:
Metadata:
Labels:
Vm Name: dev9
Spec:
Domain:
Cpu:
Cores: 4
Sockets: 1
Threads: 1
Devices:
Disks:
Boot Order: 1
Disk:
Bus: virtio
Name: rootdisk
Disk:
Bus: virtio
Name: cloudinitdisk
Interfaces:
Bridge:
Model: virtio
Name: default
Features:
Acpi:
Enabled: true
Machine:
Type: q35
Resources:
Limits:
Cpu: 4
Memory: 8Gi
Eviction Strategy: LiveMigrateIfPossible
Hostname: dev9
Networks:
Multus:
Network Name: default/default
Name: default
Termination Grace Period Seconds: 120
Volumes:
Name: rootdisk
Persistent Volume Claim:
Claim Name: dev9-disk-0
Cloud Init No Cloud:
Network Data Secret Ref:
Name: dev9-cloud-init
Secret Ref:
Name: dev9-cloud-init
Name: cloudinitdisk
Management Policies:
*
Provider Config Ref:
Kind: ClusterProviderConfig
Name: harvester
Readiness:
Policy: SuccessfulCreate
Watch: false
Status:
At Provider:
Conditions:
Last Transition Time: 2026-02-03T21:19:06Z
Observed Generation: 2
Reason: Creating
Status: False
Type: Ready
Last Transition Time: 2026-02-03T21:19:06Z
Message: create failed: cannot create object: cannot create object: admission webhook "mutator.harvesterhci.io" denied the request: network-attachment-definitions.k8s.cni.cncf.io "default" not found
Observed Generation: 2
Reason: ReconcileError
Status: False
Type: Synced
Events:
Type Reason Age From Message

---

Warning CannotCreateExternalResource 1s (x7 over 2m2s) managed/object.kubernetes.m.crossplane.io cannot create object: cannot create object: admission webhook "mutator.harvesterhci.io" denied the request: network-attachment-definitions.k8s.cni.cncf.io "default" not found
10:21:08 sthings@sthings-air12 claims →

[patrick-hermann-sva]

Good catch putting the two Secrets side by side — the reason dev7 works and dev9 doesn’t is visible in the cloud-init content.

Let’s break it down.

---

1️⃣ The critical difference: passwd vs chpasswd

dev7 (working)

You set the password like this:

chpasswd:
  list: |
    sthings:Atlan7is
  expire: false

✅ This is the correct way to set a plain-text password in cloud-init.

---

dev9 (broken)

You have this instead:

users:
  - name: sthings
    ...
    passwd: Atlan7is

❌ This is wrong for cloud-init.

passwd: must be a hashed password, not plain text.
Cloud-init will silently ignore it or fail user setup.

Correct usage would be one of these, not both:

Option A — hashed password

passwd: $6$rounds=4096$....   # SHA-512 hash

Option B — plain text (recommended)

chpasswd:
  list: |
    sthings:Atlan7is
  expire: false

---

2️⃣ Why this breaks login / SSH

Because:

• The user is created

• But no valid password is set

• ssh_pwauth: true doesn’t help without a password

• Result: login fails even though the VM boots

That’s why dev9 looks “dead” compared to dev7.

---

3️⃣ Secondary differences (not the root cause)

These are fine and not the issue:

• fqdn, manage_etc_hosts

• disable_root: true

• package_update, package_upgrade

• runcmd vs list-style runcmd (both valid)

• networkdata: ""

---

4️⃣ Minimal fix for dev9

Replace this:

passwd: Atlan7is

with this:

chpasswd:
  list: |
    sthings:Atlan7is
  expire: false

Full corrected snippet:

#cloud-config
hostname: dev9
fqdn: dev9.local
manage_etc_hosts: true
ssh_pwauth: true

disable_root: true
users:

- name: sthings

sudo: ALL=(ALL) NOPASSWD:ALL

shell: /bin/bash

ssh_authorized_keys:

- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...
chpasswd:

list: |

sthings:Atlan7is

  expire: false
package_update: true

package_upgrade: false
packages:

- qemu-guest-agent
runcmd:

- systemctl enable --now qemu-guest-agent.service

---

5️⃣ TL;DR

VM | Password method | Result -- | -- | -- dev7 | chpasswd | ✅ works dev9 | passwd: Atlan7is | ❌ ignored

Plain passwords → chpasswd only.

If you want, I can:

• generate a proper SHA-512 hash for passwd

• diff the decoded base64 cleanly

• sanity-check this for your Crossplane composition

Just say the word.