How to Set Up a K3s Cluster with MetalLB and Multiple Agent Nodes

How to Set Up a K3s Cluster with MetalLB and Multiple Agent Nodes

Setting up a lightweight Kubernetes cluster with K3s and integrating it with MetalLB for load balancing is a powerful way to manage your containerized applications. In this guide, we’ll walk through setting up a K3s cluster with MetalLB and multiple agent nodes. We’ll configure the master node to listen on 0.0.0.0 and disable the default Service Load Balancer.

Prerequisites

  1. Servers:

    • kali Linux(192.168.1.210)(mater-node) MASTER_NODE_PUBLIC_IP is name for local setup

    • Ubuntu2((192.168.1.207)(k8s-vm-worker-1)

    • MetalLB IP Pool range (192.168.1.10-192.168.1.50)

  2. Access:

    • SSH access to all nodes.
  3. Network:

    • All nodes should be able to communicate with each other over a private network.
  4. Tools Installed:

curl, iptables, and kubectl (optional for management).

Step 1: Install K3s on the Master Node

  1. Update the Master Node:

     sudo apt update
    
  2. Install K3s:

     curl -sfL https://get.k3s.io | sh -s - server \ 
         --disable servicelb \
         --node-external-ip 192.168.1.210 \ #<MASTER_NODE_PUBLIC_IP>
         --node-ip 192.168.1.210 \ #<MASTER_NODE_PRIVATE_IP>
         --bind-address 0.0.0.0
    

    Replace:

    • <MASTER_NODE_PUBLIC_IP> with the public IP of the master node (if applicable).

    • <MASTER_NODE_PRIVATE_IP> with the private IP of the master node.

  3. Confirm K3s Installation:

     sudo kubectl get nodes
    

Step 2: Retrieve the Join Token

On the master node, retrieve the join token that agent nodes will use to join the cluster:

sudo cat /var/lib/rancher/k3s/server/node-token

Copy the token for later use.


Step 3: Install K3s on Agent Nodes

  1. Update Each Agent Node:

     sudo apt update
    
  2. Install K3s Agents:

    Run the following command on each agent node:

     curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_NODE_IP>:6443 \ 
         K3S_TOKEN=<TOKEN> sh -s - agent \
         --node-external-ip <AGENT_NODE_PUBLIC_IP> \
         --node-ip <AGENT_NODE_PRIVATE_IP>
    

    Replace:

    • <MASTER_NODE_IP> with the master node’s private IP.

    • <TOKEN> with the token copied earlier.

    • <AGENT_NODE_PUBLIC_IP> and <AGENT_NODE_PRIVATE_IP> with the respective IPs of the agent node.

  3. Verify Nodes Are Added:

    On the master node, check the status of the cluster:

     sudo kubectl get nodes
    

    All nodes (master and agents) should be listed.


Step 4: Install MetalLB

  1. Deploy the MetalLB Manifest:

    Apply the official MetalLB manifest:

     kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/main/config/manifests/metallb-native.yaml
    
  2. Create a ConfigMap for MetalLB:

    Determine a range of IPs on your network to use for load balancing. For example, 192.168.1.10-192.168.1.50.

    Create a ConfigMap for MetalLB:

     cat <<EOF | kubectl apply -f -
     apiVersion: metallb.io/v1beta1
     kind: IPAddressPool
     metadata:
       name: pool
       namespace: metallb-system
     spec:
       addresses:
       - 192.168.1.10-192.168.1.50 # example: 192.168.1.10-192.168.1.50
     ---
     apiVersion: metallb.io/v1beta1
     kind: L2Advertisement
     metadata:
       name: l2-advertisement
       namespace: metallb-system
     spec:
       ipAddressPools:
       - pool
    
     EOF
    
  3. Verify MetalLB Installation:

     kubectl get pods -n metallb-system
    

    Ensure all MetalLB pods are running.


Step 5: Test Load Balancing

  1. Deploy a Sample Application:

    —load-balancer-ip can define from matallb ip pool range

     kubectl create deployment nginx --image=nginx
     kubectl expose pod my-nginx --port=80 --type=LoadBalancer --load-balancer-ip=192.168.1.11
    
  2. Check the Load Balancer IP:

     kubectl get services
    

    You should see an external IP assigned from the MetalLB range.

  3. Access the Application:

    Open a browser or use curl to access the external IP:

     curl http://192.168.1.11
    

notes: 192.168.1.1 is not able to ping.

Best Practices

  • Regularly update K3s and MetalLB to the latest versions.

  • Monitor node health using tools like kubectl top or a monitoring solution.

  • Use a firewall to secure access to cluster nodes.