Kubernetes Homelab Cluster with K3s

Build a production-ready Kubernetes cluster using K3s - a lightweight, certified Kubernetes distribution perfect for homelabs, edge computing, and resource-constrained environments. This project includes persistent storage, ingress, TLS automation, and GitOps deployment.

Project Overview

What We're Building

┌─────────────────────────────────────────────────────────────────────┐
│                    K3s Homelab Architecture                          │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │                    Homelab Network                           │   │
│  │                     192.168.1.0/24                           │   │
│  └─────────────────────────────────────────────────────────────┘   │
│              │              │              │                        │
│              ▼              ▼              ▼                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │
│  │   k3s-cp-1   │  │  k3s-wkr-1   │  │  k3s-wkr-2   │              │
│  │   (master)   │  │   (worker)   │  │   (worker)   │              │
│  │  4GB RAM     │  │  8GB RAM     │  │  8GB RAM     │              │
│  │  50GB SSD    │  │  100GB SSD   │  │  100GB SSD   │              │
│  │  .10         │  │  .11         │  │  .12         │              │
│  └──────────────┘  └──────────────┘  └──────────────┘              │
│         │                  │                  │                     │
│         └──────────────────┴──────────────────┘                     │
│                            │                                        │
│              ┌─────────────▼─────────────┐                         │
│              │     K3s Components        │                         │
│              │  ┌────────┐ ┌──────────┐  │                         │
│              │  │Traefik │ │ CoreDNS  │  │                         │
│              │  │Ingress │ │          │  │                         │
│              │  └────────┘ └──────────┘  │                         │
│              │  ┌────────┐ ┌──────────┐  │                         │
│              │  │Longhorn│ │cert-mgr  │  │                         │
│              │  │Storage │ │          │  │                         │
│              │  └────────┘ └──────────┘  │                         │
│              │  ┌────────────────────┐   │                         │
│              │  │      ArgoCD        │   │                         │
│              │  │     (GitOps)       │   │                         │
│              │  └────────────────────┘   │                         │
│              └───────────────────────────┘                         │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Prerequisites

3+ VMs or bare metal nodes (minimum 2GB RAM each)
Ubuntu 22.04 LTS or similar Linux distribution
SSH access to all nodes
Domain name for ingress (optional but recommended)
Basic Kubernetes knowledge

Part 1: Node Preparation

Step 1: Prepare Nodes

On ALL nodes:

# Update system
sudo apt update && sudo apt upgrade -y
 
# Install required packages
sudo apt install -y curl wget open-iscsi nfs-common
 
# Disable swap (required for Kubernetes)
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
 
# Set hostnames
# On master: sudo hostnamectl set-hostname k3s-cp-1
# On workers: sudo hostnamectl set-hostname k3s-wkr-1, etc.
 
# Add hosts entries
cat << EOF | sudo tee -a /etc/hosts
192.168.1.10 k3s-cp-1
192.168.1.11 k3s-wkr-1
192.168.1.12 k3s-wkr-2
EOF
 
# Enable required kernel modules
cat << EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
 
sudo modprobe overlay
sudo modprobe br_netfilter
 
# Set sysctl params
cat << EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
 
sudo sysctl --system

Step 2: Configure Firewall

# If using UFW
sudo ufw allow 6443/tcp  # Kubernetes API
sudo ufw allow 10250/tcp # Kubelet metrics
sudo ufw allow 8472/udp  # Flannel VXLAN
sudo ufw allow 51820/udp # Wireguard (optional)
sudo ufw allow 80/tcp    # HTTP
sudo ufw allow 443/tcp   # HTTPS
 
# Or disable UFW for internal cluster network
sudo ufw disable

Part 2: K3s Installation

Step 3: Install K3s Master Node

On the control plane node (k3s-cp-1):

# Install K3s with specific configuration
curl -sfL https://get.k3s.io | sh -s - server \
  --disable servicelb \
  --disable traefik \
  --write-kubeconfig-mode 644 \
  --tls-san 192.168.1.10 \
  --tls-san k3s.homelab.local \
  --cluster-init
 
# Wait for node to be ready
sudo kubectl get nodes --watch
 
# Get the node token for workers
sudo cat /var/lib/rancher/k3s/server/node-token

Configuration Options Explained:

Option	Purpose
`--disable servicelb`	We'll use MetalLB instead
`--disable traefik`	We'll install Traefik v2 manually
`--write-kubeconfig-mode 644`	Allow non-root kubeconfig access
`--tls-san`	Add SANs for external access
`--cluster-init`	Use embedded etcd

Step 4: Install K3s Worker Nodes

On each worker node:

# Get the token from the master node
K3S_TOKEN="<token-from-master>"
K3S_URL="https://192.168.1.10:6443"
 
# Install K3s agent
curl -sfL https://get.k3s.io | K3S_URL=$K3S_URL K3S_TOKEN=$K3S_TOKEN sh -
 
# Verify on master
sudo kubectl get nodes

Expected Output:

NAME        STATUS   ROLES                       AGE   VERSION
k3s-cp-1    Ready    control-plane,etcd,master   5m    v1.29.0+k3s1
k3s-wkr-1   Ready    <none>                      2m    v1.29.0+k3s1
k3s-wkr-2   Ready    <none>                      1m    v1.29.0+k3s1

Step 5: Configure kubectl Locally

On your local machine:

# Copy kubeconfig from master
scp user@192.168.1.10:/etc/rancher/k3s/k3s.yaml ~/.kube/k3s-config
 
# Update server address in config
sed -i 's/127.0.0.1/192.168.1.10/' ~/.kube/k3s-config
 
# Set as current context
export KUBECONFIG=~/.kube/k3s-config
 
# Verify connection
kubectl get nodes
kubectl cluster-info

Part 3: Core Components

Step 6: Install Helm

# Install Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
 
# Verify
helm version
 
# Add common repos
helm repo add stable https://charts.helm.sh/stable
helm repo add jetstack https://charts.jetstack.io
helm repo add longhorn https://charts.longhorn.io
helm repo add traefik https://traefik.github.io/charts
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update

Step 7: Install MetalLB

# Create namespace
kubectl create namespace metallb-system
 
# Install MetalLB
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.3/config/manifests/metallb-native.yaml
 
# Wait for pods
kubectl wait --namespace metallb-system \
  --for=condition=ready pod \
  --selector=app=metallb \
  --timeout=90s
 
# Configure IP address pool
cat << EOF | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.200-192.168.1.220
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
spec:
  ipAddressPools:
  - default-pool
EOF

Step 8: Install Traefik

# traefik-values.yaml
deployment:
  replicas: 2
 
service:
  type: LoadBalancer
  annotations:
    metallb.universe.tf/loadBalancerIPs: "192.168.1.200"
 
ports:
  web:
    port: 8000
    exposedPort: 80
    protocol: TCP
  websecure:
    port: 8443
    exposedPort: 443
    protocol: TCP
    tls:
      enabled: true
 
ingressRoute:
  dashboard:
    enabled: true
    matchRule: Host(`traefik.homelab.local`)
    entryPoints: ["websecure"]
 
logs:
  general:
    level: INFO
  access:
    enabled: true
 
providers:
  kubernetesCRD:
    enabled: true
    allowCrossNamespace: true
  kubernetesIngress:
    enabled: true

# Install Traefik
helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace \
  --values traefik-values.yaml
 
# Verify
kubectl get pods -n traefik
kubectl get svc -n traefik

Step 9: Install cert-manager

# Install cert-manager CRDs
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.0/cert-manager.crds.yaml
 
# Install cert-manager
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.14.0
 
# Wait for deployment
kubectl wait --namespace cert-manager \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/instance=cert-manager \
  --timeout=60s
 
# Create ClusterIssuer for Let's Encrypt
cat << EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email: admin@yourdomain.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
    - http01:
        ingress:
          class: traefik
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: admin@yourdomain.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging-account-key
    solvers:
    - http01:
        ingress:
          class: traefik
EOF

Part 4: Storage with Longhorn

Step 10: Install Longhorn

# Verify iscsi is available on all nodes
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.6.0/deploy/prerequisite/longhorn-iscsi-installation.yaml
 
# Install Longhorn
helm install longhorn longhorn/longhorn \
  --namespace longhorn-system \
  --create-namespace \
  --version 1.6.0 \
  --set defaultSettings.defaultDataPath="/var/lib/longhorn" \
  --set defaultSettings.defaultReplicaCount=2
 
# Wait for deployment
kubectl wait --namespace longhorn-system \
  --for=condition=ready pod \
  --selector=app=longhorn-manager \
  --timeout=300s
 
# Set as default storage class
kubectl patch storageclass longhorn -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

Step 11: Configure Longhorn Access

# longhorn-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-staging
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - longhorn.homelab.local
    secretName: longhorn-tls
  rules:
  - host: longhorn.homelab.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: longhorn-frontend
            port:
              number: 80

Part 5: GitOps with ArgoCD

Step 12: Install ArgoCD

# Create namespace
kubectl create namespace argocd
 
# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
 
# Wait for pods
kubectl wait --namespace argocd \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/name=argocd-server \
  --timeout=180s
 
# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d; echo

Step 13: Configure ArgoCD Access

# argocd-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd-server-ingress
  namespace: argocd
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-staging
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - argocd.homelab.local
    secretName: argocd-server-tls
  rules:
  - host: argocd.homelab.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: argocd-server
            port:
              name: https

Step 14: Create GitOps Repository Structure

homelab-k8s/
├── apps/
│   ├── base/
│   │   ├── namespace.yaml
│   │   └── kustomization.yaml
│   ├── whoami/
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   ├── ingress.yaml
│   │   └── kustomization.yaml
│   └── monitoring/
│       └── kustomization.yaml
├── infrastructure/
│   ├── metallb/
│   ├── traefik/
│   ├── cert-manager/
│   └── longhorn/
└── argocd/
    └── applications.yaml

Step 15: Deploy Sample Application via ArgoCD

# argocd/whoami-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: whoami
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/yourusername/homelab-k8s.git
    targetRevision: HEAD
    path: apps/whoami
  destination:
    server: https://kubernetes.default.svc
    namespace: whoami
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

# apps/whoami/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami:latest
        ports:
        - containerPort: 80
---
# apps/whoami/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  ports:
  - port: 80
  selector:
    app: whoami
---
# apps/whoami/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: whoami
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-staging
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - whoami.homelab.local
    secretName: whoami-tls
  rules:
  - host: whoami.homelab.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: whoami
            port:
              number: 80

Part 6: Monitoring Stack

Step 16: Install Prometheus and Grafana

# Add Prometheus community Helm repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
 
# Install kube-prometheus-stack
helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set grafana.adminPassword="<YOUR_STRONG_PASSWORD>" \
  --set prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.storageClassName=longhorn \
  --set prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.accessModes[0]=ReadWriteOnce \
  --set prometheus.prometheusSpec.storageSpec.volumeClaimTemplate.spec.resources.requests.storage=10Gi
 
# Create Grafana ingress
cat << EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana
  namespace: monitoring
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-staging
spec:
  ingressClassName: traefik
  tls:
  - hosts:
    - grafana.homelab.local
    secretName: grafana-tls
  rules:
  - host: grafana.homelab.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: prometheus-grafana
            port:
              number: 80
EOF

Verification Checklist

Cluster Health:

All nodes in Ready state
Control plane components running
CoreDNS resolving internal DNS
Pod-to-pod networking working

Core Components:

MetalLB assigning IPs
Traefik routing traffic
cert-manager issuing certificates
Longhorn providing storage

GitOps:

ArgoCD accessible
Sample app deployed via GitOps
Automatic sync working

Monitoring:

Prometheus collecting metrics
Grafana dashboards accessible
Alerts configured

Troubleshooting

Issue	Cause	Solution
Node NotReady	Network/kubelet issue	Check `kubectl describe node`
Pods pending	No available nodes	Check node resources
PVC pending	Storage class issue	Verify Longhorn is healthy
No external IP	MetalLB not configured	Check IP pool config
Certificate not issued	DNS/HTTP challenge failed	Check cert-manager logs

Next Steps

After building your cluster:

Add Backup Solution - Velero for cluster backups
Implement Secrets Management - External Secrets Operator
Add Service Mesh - Linkerd or Istio
Configure Logging - Loki stack

Resources

Questions? Reach out in our community Discord!

Kubernetes Homelab Cluster with K3s

Tools & Technologies