Sign in

Kubernetes with Windows Nodes

Oh, the agony and celebration of getting Windows nodes joined to a Kubernetes cluster. This article dives into some of the challenges I faced and their resolution in the quest to get scalable, self-healing windows containers running within a Kubernetes cluster.

Preparing the Master node

There are many how-to articles for prepping a Kubernetes cluster to accept Windows nodes. They seem to be all over the map in terms of preparation steps one performs prior to calling the final “kubectl join” command. Here was the magic combination that worked for our environment.

On the Master Node:

  1. Download the latest flannel CNI from: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
  2. Modify the net-conf.json section of the flannel manifest in order to set the backend type to “host-gw”. It should look as follows:

net-conf.json: { “Network”: “10.244.0.0/16”, “Backend”: { “Type”: “host-gw”}}

3. Apply the kube-flannel.yml:

kubectl apply -f kube-flannel.yml

4. Add Windows Flannel and Kube-Proxy DaemonSet for Windows Nodes

curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed ‘s/VERSION/v1.19.3/g’ | kubectl apply -f -

kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-host-gw.yml

Preparing the Windows Node

Next, we’ll want to get the required components installed on the windows nodes. The following assumes that the windows node already has an up-to-date version of docker running on the system and able to host containers.

  1. Create the directory “c:\k” and open up a command line window in that directory. Make sure you run as Administrator to get access to all of the docker and networking components.
  2. Grab the latest Preparation script from the Kubernetes Sig group.

curl.exe -LO https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/PrepareNode.ps1

3. Run the powershell script

powershell .\PrepareNode.ps1 -KubernetesVersion v1.19.3

NOTE: Plug in the version of Kubernetes that your master is currently running on.

4. Verify that two services called “kubelet” and “Rancher Wins” are installed.

Join the Windows Node

The windows node is ready to join the cluster. Run the following command on the Kubernetes master node to retrieve the JOIN command that needs to be run on the windows node:

kubeadm token create — print-join-command

Run the resulting JOIN command on the windows node.

This should result in the starting of the Kubelet and Rancher Wins services.

While the container images are getting downloaded and installed onto the Windows nodes, its wise to have a look at the logs under c:\var\logs\kubelet to see if there might be anything out of wack. The error message about the network not responding is normal while the container images are still downloading. This should eventually clear up.

I’ve had to manually create the folder “c:\var\lib\kubelet\etc\kubernetes\manifests” in order for kubelet to stop complaining that the directoy doesn’t exist. This was clearing up more of an annoyance than anything else.

Eventually, once the images had a chance to download 5–30min depending on connection) running “docker ps” on the windows node should reveal a couple running containers. One for the kube-proxy and another for the flannel network. To verify that the node is completely joined, type “kubectl get nodes” on the master node. It should show the status of “Ready” for the windows node.

NOTE: If the status is “Pending”, its most likely the case that windows is still downloading all of the images needed to run inside of Kubernetes (~6GB of images). One quick way to see if this is the case is to bring up the Resource monitor on windows and examine if disk writing is happening in the “c:\ProgramData\docker” folder area.

Final Verification of Installation

The true test is to see if the newly joined Windows node can accept a Kubernetes Pod. Create a basic deployment file that targets Windows nodes and apply it. The following is an example deployment that sets up a simple running web server.

Conclusion

And there you have it. Hopefully all of the stars aligned and you have a working Kubernetes cluster with windows nodes running on it.

One last word of caution:

Because you now have a working hybrid Kubernetes environment, you need to be careful when deploying additional services. Make sure that you always specify a “nodeSelector” element in the deployment YAML file for every service deployed. It should specify either “kubernetes.io/os: windows” or “kubernetes.io/os: linux”. Otherwise, you could end up with Kubernetes randomly picking the wrong type of nodes to run the pods from time to time.

My next article will focus on how to get linux and windows pods talking to windows fileshares using the new CSI module. Stay Tuned.

I am a longtime software builder, fly fisher and professional musician. Living in Montana allows me to follow all of my passions in life.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store