4 Tools to Automate Your Kubernetes Cluster Deployments
Kubernetes is hard
That is what you usually see voiced by community members when digging through obscure Github forums for the arcane k8s incantations to get your cluster nodes working smoothly. This challenge often leads engineers to wonder how to automate Kubernetes deployments.
This is probably why managed Kubernetes cluster services, with their emphasis on Kubernetes Automation, are such a hot topic, they promise to eliminate much of the deployment and management hassles, which can allow you to focus on the workloads you need to run. But if you are still going roll your own clusters, I’d first look through a few of the options covered in this article. They might provide insights into how to automate Kubernetes deployment, potentially saving you some headaches.
Before we look at the Kubernetes deployment tools, lets define a few deployment targets. At the bare minimum we are looking for an application that consumes declarative manifests of some sort and bootstraps fully functioning Kubernetes clusters. We will also want to ensure that a cluster is highly available.
Goal 1: Highly available
A highly available K8s cluster is more than just having 3 nodes. The nodes must be configured with the right roles to be considered fault tolerant and highly available. This is where the concept of automating Kubernetes cluster creation becomes crucial, as it streamlines the setup of these intricate configurations.
The bill of materials for an HA deployment include, at the minimum:
- 2 controller (master) nodes
- 3 etcd (config store) instances
- 2 worker nodes
This diagram shows HA components on nodes of a 3-node cluster (notice the included load balanced services).
NOTE: This does not do justice to the complications around truly HA K8s clusters. Read a great discussion on true HA here and an insightful article here.
Goal 2: Not highly available
A Kubernetes cluster is either highly available or it isn’t. Anything that is not highly available in Kubernetes may as well be a single node local deployment and is considerably easier to get working for the most part. This is important to know when exploring how to automate Kubernetes deployments.
Single node K8s clusters are useful for development, deployment smoke testing, and treading the Kubernetes waters a bit. They also serve as an initial step before advancing to more complex tasks like automating Kubernetes cluster creation. However, we will not delve into these multi-node, automated options here.
For single node clusters, I strongly recommend using Docker CE (edge) with its single node Kubernetes feature, minikube, or other purpose-built projects. These tools are specifically designed to simplify the process of setting up a single node cluster on almost any desktop OS. They offer a practical starting point for those new to Kubernetes, allowing users to focus on learning the basics before they tackle more advanced concepts such as automating Kubernetes cluster creation for larger, more resilient deployments.
Target platforms
Most of these tools target AWS or Azure VMs to deploy Kubernetes clusters, creating efficient clustering services. Some might also target other cloud providers or even local infrastructure as well. The general idea of all these solutions is to templatize the k8s cluster deployment process.
Some of the following are almost certainly required for your k8s cluster deployments regardless of the target platform:
- Shared/Clustered/HA backend persistent storage
- External Load Balancer Service/IP
- DNS zone for <clustername>.<zone>.<zoneroot> (Ideally external)
- Ability to create host VM instances and other IaaS for K8s cluster itself.
Opting for cloud-based Kubernetes Automated Deployment makes the process more straightforward, allowing you to leverage IaaS/PaaS offerings for added geo-redundancy and enhanced Kubernetes functionality.
Kubernetes Standard tools
The following are standard Kubernetes automation tools that may be used alongside other deployment tools to efficiently ‘get the job done’.
Kubeadm/kubectl
These are known by most and can be used to deploy an entire cluster once the base hosts/networking is setup and available.
Ansible
Configuration deployment and management platform. Ansible is a swiss army knife of configuration tools and can be used for every step of a k8s deployment. In fact some tools are simply complex Ansible playbooks. Ansible playbooks themselves can be particularly effective in automating Kubernetes cluster creation, which is a crucial step in a k8s deployment. In fact, some tools designed for Kubernetes deployments are essentially complex Ansible playbooks. These playbooks can be authored to be declarative and idempotent, which are key characteristics for efficiently automating Kubernetes cluster creation and management.
Terraform
A declarative infrastructure deployment tool that can greatly simplify the process to deploy Kubernetes clusters and other infrastructure to multiple platforms.
Cloudformation Templates
JSON templating language for cloud deployment to AWS.
ARM Templates
JSON templating language for cloud deployment to Azure.
Kubernetes Deployment tools
Each of these tools can be used to deploy Kubernetes clusters to various platforms.
KubeSpray
A prime example of Kubernetes automation tools, this Ansible-driven Kubernetes cluster deployment playbook showcases Kubernetes automation with deep customization and control. From their site you can deploy a production ready kubernetes cluster and has the following features:
- Can be deployed on AWS, GCE, Azure, OpenStack, vSphere, Oracle Cloud Infrastructure (Experimental), or Baremetal
- Highly available cluster
- Composable (Choice of the network plugin for instance)
- Supports most popular Linux distributions
- Continuous integration tests
Website: https://github.com/kubernetes-incubator/kubespray
Thoughts: I listed this one first as I consider it to be a good starting point. It has an included Vagrantfile definition for quick testing (it can take quite a while and may require you to run vagrant provision more than a few times to complete.). The entire kubespray project is an Ansible playbook geared to deploy Kubernetes clusters, offering extensive customization options.. That makes this project very flexible. As such it is also a bit more complex to get working properly in my experiences.
Conjure-up
From their site:
conjure-up lets you summon up a big-software stack as a “spell” — a model of the stack, combined with extra know-how to get you from an installed stack to a fully usable one. Start using your big software instead of learning how to deploy it.
Website: https://conjure-up.io/
Thoughts:
This one is relatively new to me. The spells listed in their registry are more difficult infrastructure deployments like OpenStack, Kubernetes, and Sparc stacks. This particular tool is designed to deploy Kubernetes clusters, specifically the Canonical distribution, across several cloud providers or locally.. That makes sense as the tool itself is a wrapper around Canonical’s juju framework. conjure-up gives you super pretty console prompts along the way to help guide the deployment but you can also easily automate the process with a ‘Conjourfile’.
While the whole process is easy and all, be careful as I somehow ended up with a cluster out in China when I had thought the process had failed against us-east-1. A Chinese east region k8s cluster built from conjure-up with mostly default values for a AWS will cost about 50 USD a day btw 🙂. This highlights the importance of carefully monitoring the process, especially when you automate Kubernetes cluster creation, to ensure deployments occur in the intended regions.
All joking aside, conjure-up is well thought out, has native AWS integration, and is totally worth keeping an eye on, particularly for those that want to streamline and automate Kubernetes cluster creation.
Kops
From their site:
Kubernetes Operations (kops) – Production Grade K8s Installation, Upgrades, and Management
- Automates the provisioning of Kubernetes clusters in AWS and GCE
- Deploys Highly Available (HA) Kubernetes Masters
- Built on a state-sync model for dry-runs and automatic idempotency
- Ability to generate Terraform
- Supports custom Kubernetes add-ons
- Command line autocompletion
- YAML Manifest Based API Configuration
- Templating and dry-run modes for creating Manifests
- Choose from eight different CNI Networking providers out-of-the-box
- Supports upgrading from kube-up
- Capability to add containers, as hooks, and files to nodes via a cluster manifest
Website: https://github.com/kubernetes/kops
Thoughts: I put this one on the list because they have an almost 800 line Makefile. Honestly, that is kind of impressive! Aside from that, I’ve actually used this to templatize and deploy, then destruct, a Kubernetes cluster in AWS flawlessly. It was super easy to setup and get working and I believe it is likely what I’ll be employing for rapid cluster deployments in future projects, especially when looking to automate Kubernetes deployment.
kube-aws
From their site:
kube-aws is a command-line tool to create/update/destroy Kubernetes clusters on AWS.
- Create, update and destroy Kubernetes clusters on AWS
- Highly available and scalable Kubernetes clusters backed by multi-AZ deployment and Node Pools
- Deployment to an existing VPC
- Powered by various AWS services including CloudFormation, KMS, Auto Scaling, Spot Fleet, EC2, ELB, S3, etc
Website: https://kubernetes-incubator.github.io/kube-aws/
Thoughts: I’ve not had the chance to deploy using this Kubernetes automation tool but the documentation is very well thought out around what the requirements are for existing VPCs, subnets, and other AWS cloud services and what will be required to get things running.
Fortunately, the project name clearly states its only drawback, that it only targets AWS.
Additional Kubernetes Deployment Tools
Before going further, it would be wise for the reader to pop over to the Cloud Native Computing Foundation and explore the many projects under this umbrella site, especially those that focus on how to automate Kubernetes deployment. All of these projects are likely to draw more attention and activity being backed by the CNCF.
Next, there are several other Kubernetes cluster deployment tools you may want to investigate as well. These tools not only facilitate cluster deployment but also offer capabilities to automate Kubernetes deployment. Here are very short descriptions and links to a few worth keeping on the radar:
BootKube – a tool for launching self-hosted Kubernetes clusters.
Kustomize – Template-free configuration customization for Kubernetes (Note: This is not about deploying a cluster but is extremely interesting as an arbitrary Kubernetes workload/configuration transform management tool).
Spread – a command line tool that makes it easy to version Kubernetes clusters, deploy to Kubernetes clusters in one command, and set up a local Kubernetes cluster.
Kubicorn – Very interesting deployment and lifecycle management tool for K8s. Goes beyond initial deployment with ‘cluster snapshot’.
Bonus! Here is a longer list of interesting Kubernetes projects of all kinds as well.
Conclusion
Kubernetes is hard, but Kubernetes automated deployment simplifies this with numerous advanced tools designed to address these complexity concerns.. I’ve listed a few purpose driven tools to get a production ready Kubernetes cluster up quickly. Each of these Kubernetes automation tools can be used to deploy Kubernetes clusters to various platforms to streamline the deployment process.. I recommend reviewing them all to see which will fit your own cluster deployment needs.
About the Author
Zachary Loeber is a Sr. Consultant with the Cloud practice within SPR. He is a passionate IT industry veteran in cloud automation, security, and design. Zach enjoys coding in PowerShell but is not adverse to Python, bash, groovy, or any form of declarative language.