Bringing Back Kubernetes The problem Shortly after having had my last day of work some issues came up and the kubernetes cluster went down. I began looking into it (a bit half heartily) and I began noticing some strange random issues. Monitoring the kube-system namespace I saw that flannel issues seemed to crop up quickly. There are several issues open regarding flannel failing #963, 1076,… and initially the real culprit wasn’t clear.

Reinstalling the Kubernetes Production Cluster The main reason for this is a critical Kubernetes bug. Here is a bit a log of what I did (to hopefully help me next time). This is mostly taken from kubernetes doc (1, 2) and tweaks I added. Master node: Basically the same as the Visualization Setup minus the tainting and coredns editing: # disable swap: swapoff -a # comment out swap lines (the command should do it, but I prefer manual edit) # sudo sed -i '/ swap / s/^\(.

Dev Kubernetes reinstall The main reason for this is a critical Kubernetes bug. Here is a bit a log of what I did (to hopefully help me next time). Development cluster extension I installed a new cluster on labdev3, mostly following what I did last time. But labdev4 was having issues, so I asked a clean image. It took a bit to have it up. Here is the process to add it to the Kubernetes cluster.

Kubernetes Kubernetes is central to NOMAD archive and analytics (see /nomad/Kubernetes). Every installation of kubernetes has been quite different (kubernetes evolves quickly). Here is the description of the latest (more manual) installation I did on CentOS for the remote visualization, mainly as a reference for me. This uses kubeadm something that has simplified the installation of kubernetes. update kubeadm (from https://kubernetes.io/docs/setup/independent/install-kubeadm/) cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.