Build Kubernetes cluster Multi-Master with Raspberry PI and Containerd

Hello everyone, in this article I will show you how easy and quick it is to install a Kubernetes cluster with multi masters.

I will generate an environment as follows:

• 1-HaProxy
• 3-Masters
• 4-Workers
• 1-Node just to GitLab
• 1-Node just to database infrastructure

Step 1 — Build Image Ubuntu Server SSD

I’m using Ubuntu Service 21.04, I tried to use the image provided by Raspberry Pi Imager 1.6. But I was having trouble logging in via ssh

After installing your image on the micro SSD. Remove the SSD card and put it again and enter the system-boot folder. Next, you need to open the network-config file to set the configuration of wifi as in my case.

You need to remove the “#” comments and put your Wifi network name and password.

To help me to identify all Raspberry Pi I specify the name for all dispositive by Router interface.

Step 2— Settings Ubuntu Service

sudo apt update
sudo apt install net-tools sudo apt install vim

Change the machine names with the command below

hostname k8s-M-01
echo "k8s-M-01" > /etc/hostname
bash
sudo reboot

Step 3— Install and configure the HaProxy

sudo apt install haproxy 

Open the file for the configuration.

sudo vim /etc/haproxy/haproxy.cfg

Then go to the end of the configuration file, skip two lines and paste the settings given below. Then save the file.

frontend kubernetes
        mode tcp
        bind HAPROXY IP:6443
        option tcplog
        default_backend k8s-masters
backend k8s-masters
        mode tcp
        balance roundrobin
        option tcp-check
        server k8s-master-0 MASTER IP:6443 check fall 3 rise 2
        server k8s-master-1 MASTER IP:6443 check fall 3 rise 2
        server k8s-master-2 MASTER IP:6443 check fall 3 rise 2

Then just go to check the status of haproxy is working.

Step 4— Initial setup and installations for installing Kubernetes

first, we have to add the haproxy IP to the hosts of all machines.

then install and configure the containerd.

Then just run the existing settings below.

sudo apt install containerd
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo systemctl restart containerd
sudo systemctl enable containerd

Step 5— Configuration and installation of kubelet kubeadm and kubectl

just apply the settings and install the existing items below.

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter 
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system

Update the apt package index and install packages needed to use the Kubernetes apt repository

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

Download the Google Cloud public signing key

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

Add the Kubernetes apt repository

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

Update apt package index, install kubelet, kubeadm and kubectl, and pin their version:

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Step 6— Master startup

Then just run the existing command below to starting the cluster.

sudo kubeadm init --control-plane-endpoint "k8s-haproxy" --upload-certs

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Apply the CNI plugin of your choice: Follow these instructions to install the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the kubeadm configuration file if applicable.

In this example we are using Weave Net:

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

You can now join any number of the control-plane node running the following command on each as root:

kubeadm join k8s-haproxy:6443 --token mixsv6.113g2190t3dka6z6 \
--discovery-token-ca-cert-hash sha256:511700780d2db6fd491d4feab52ebb92b9b5efe506d02be920d36763dfc8437c \
--control-plane --certificate-key 9584ad34bd6a9d719c673fdf3e2829aa4526824716164a436e30ee3ed79f26bc

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Step 6 — Add workers nodes

Then just run the existing command below, on all workers.

kubeadm join k8s-haproxy:6443 --token mixsv6.113g2190t3dka6z6 \
--discovery-token-ca-cert-hash sha256:511700780d2db6fd491d4feab52ebb92b9b5efe506d02be920d36763dfc8437c

Step 7— Enable kubectl auto-completion

sudo apt-get install bash-completion
source /usr/share/bash-completion/bash_completion
kubectl completion bash >/etc/bash_completion.d/kubectl

All configurations made are removed from the Kubernetes here and I took a course on Uncomplicating Kubernetes on LinuxTips that added a lot of knowledge to me.