- Version: v1.30.5+k3s1
- Architecture: 3 HA masters (etcd) + workers
- Ingress: Traefik
- Storage: Longhorn
- VIP: 192.168.9.92 (HAProxy + Keepalived on haproxy-k3s VM 508)
| VM |
VMID |
IP |
Node |
Status |
| haproxy-k3s (LB + VIP) |
508 |
192.168.9.95 |
bm06 |
Running |
| k3s-master01 |
509 |
192.168.9.210 |
bm06 |
control-plane,etcd,master |
| k3s-master02 |
510 |
192.168.9.211 |
bm05 |
control-plane,etcd,master |
| k3s-master03 |
511 |
192.168.9.212 |
bm01 |
control-plane,etcd,master |
SSH to master: manojm / manojM@1289
HAProxy stats: http://192.168.9.95:8404/stats
| Hostname |
VMID |
IP |
Proxmox Node |
Status |
| k3s-d01 |
500 |
192.168.9.200 |
bm05 |
virt-customize done, k3s not installed |
| k3s-d02 |
501 |
192.168.9.201 |
bm03 |
not configured |
| k3s-d03 |
502 |
192.168.9.202 |
bm03 |
not configured |
| k3s-d04 |
503 |
192.168.9.203 |
bm05 |
virt-customize done, k3s not installed |
| k3s-e01 |
504 |
192.168.9.204 |
bm01 |
not configured |
| k3s-rc05 |
505 |
192.168.9.205 |
bm04 |
not configured |
Old ESXi workers (still active): Kubernates-D01..D04 on esxi-05 (192.168.9.90)
# SSH to master
ssh manojm@192.168.9.210 # via bm02 jump or direct via Tailscale
# kubectl
kubectl get nodes
kubectl get pods -A
kubectl get ingress -A
# Cluster API endpoint
https://192.168.9.92:6443
K3S_TOKEN=K1059a6ac26b0c4df2e9f63881f8a3116650b251689e6eb973735f241361a1620e7::server:30634b753dc21b5582029419f294a586
K3S_SERVER=https://192.168.9.92:6443
#!/bin/bash
# Run on each new worker VM (as root)
K3S_SERVER="https://192.168.9.92:6443"
K3S_TOKEN="K1059a6ac26b0c4df2e9f63881f8a3116650b251689e6eb973735f241361a1620e7::server:30634b753dc21b5582029419f294a586"
K3S_VERSION="v1.30.5+k3s1"
apt-get install -y open-iscsi nfs-common qemu-guest-agent
systemctl enable --now iscsid qemu-guest-agent
mkdir -p /etc/rancher/k3s
cat > /etc/rancher/k3s/registries.yaml << 'EOF'
mirrors:
"192.168.9.144:5000":
endpoint:
- "http://192.168.9.144:5000"
configs:
"192.168.9.144:5000":
tls:
insecure_skip_verify: true
EOF
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="$K3S_VERSION" K3S_URL="$K3S_SERVER" K3S_TOKEN="$K3S_TOKEN" sh -
- Traefik handles HTTP routing inside the cluster
- SSL is terminated at OpenResty — ingresses use HTTP only
- All ingresses:
ingressClassName: traefik
# Template ingress (HTTP only — SSL at OpenResty)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp
namespace: myapp
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web,websecure
spec:
ingressClassName: traefik
rules:
- host: myapp.smartb.com.au
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80
| Namespace |
Purpose |
| wikijs |
Wiki.js documentation (this wiki) |
| cattle-system |
Rancher |
| kube-system |
Traefik, CoreDNS, metrics-server |
| longhorn-system |
Longhorn distributed storage |
| smartb-master |
Main app workloads |
- Internal registry:
192.168.9.144:5000 (on Sonar/CI VM)
- Configured in
/etc/rancher/k3s/registries.yaml on all nodes
- Wave 1: Drain old ESXi D01/D02, move workloads to new Proxmox VMs 500-503
- Wave 2: Drain D03/D04
- Wave 3: RedisCluster, E01
- Wave 4: Control plane HA refinement, migrate E02 (old single master)
- Wave 5: Free ESXi hosts → install Proxmox → join cluster