Hightower et al. Kubernetes: Up and Running: Dive into the Future of Infrastructure
Introduction
Definition
Kubernetes is an open source orchestrator for deploying containerized applications. Kubernetes was originally developed by Google, inspired by a decade of experience deploying scalable, reliable systems in containers via application-oriented APIs.1
Kubernetes is a product that is suited not just to the needs of internet-scale companies but to cloud-native developers of all scales, from a cluster of Raspberry Pi computers to a warehouse full of the latest machines. Kubernetes provides the software necessary to successfully build and deploy reliable, scalable distributed systems.
(Hightower et al., 2017, p. 1)
Benefits
Four key benefits (Hightower et al., 2017, p. 2):
- Velocity
- Scaling (of both software and teams)
- Abstracting your infrastructure
- Efficiency
Velocity
Once upon a time, it was OK for a service to be down for maintenance at midnight every night. But today, our users expect constant uptime, even if the software they are running is changing constantly.
(Hightower et al., 2017, p. 2)
Velocity is measured not in terms of the raw number of features you can ship per hour or day, but rather in terms of the number of things you can ship while maintaining a highly available service.
(Hightower et al., 2017, p. 2)
The core concepts that enable [velocity] are immutability, declarative configuration, and online self-healing systems.
Immutability
(Hightower et al., 2017, p. 3)
- In an immutable system, rather than a series of incremental updates and changes, an entirely new, complete image is built, where the update simply replaces the entire image with the newer image in a single operation. There are no incremental changes.
- building a new image rather than modifying an existing one means the old image is still around, and can quickly be used for a rollback if an error occurs. In contrast, once you copy your new binary over an existing binary, such rollback is nearly impossible.
- Immutable container images are at the core of everything that you will build in Kubernetes.
Declarative Configuration
(Hightower et al., 2017, p. 4)
- Everything in Kubernetes is a declarative configuration object that represents the desired state of the system.
- Since the effects of declarative configuration can be understood before they are executed, declarative configuration is far less error-prone.
- The traditional tools of software development, such as source control, code review, and unit testing, can be used in declarative configuration in ways that are impossible for imperative instructions.
- Rollback of a change [is] trivially easy. It is simply restating the previous declarative state of the system.
Self-Healing System
(Hightower et al., 2017, pp. 4–5)
- When [Kubernetes] receives a desired state configuration, it does not simply take actions to make the current state match the desired state a single time. It continuously takes actions to ensure that the current state matches the desired state. This means that not only will Kubernetes initialize your system, but it will guard it against any failures or perturbations that might destabilize your system and affect reliability.
- [It reduces] the burden on operators and improve the overall reliability of the system by performing reliable repairs more quickly.
- Example: [if the desired state is to have three replicas], Kubernetes does not just create three replicas—it continuously ensures that there are exactly three replicas. If you manually create a fourth replica Kubernetes will destroy one to bring the number back to three. If you manually destroy a replica, Kubernetes will create one to again return you to the desired state.
Scaling Your Service and Your Teams
Kubernetes achieves scalability by favoring decoupled architectures. (Hightower et al., 2017, p. 5)
Decoupling
(Hightower et al., 2017, p. 5)
- In a decoupled architecture each component is separated from other components by defined APIs and service load balancers.
- Decoupling servers via APIs makes it easier to scale the development teams because each team can focus on a single, smaller microservice with a comprehensible surface area.
- Crisp APIs between microservices limit the amount of cross-team communication overhead required to build and deploy software.
Easy Scaling for Applications and Clusters
(Hightower et al., 2017, p. 6)
- Because each machine in a cluster is entirely identical to every other machine, and the applications themselves are decoupled from the details of the machine by containers, adding additional resources to the cluster is simply a matter of imaging a new machine and joining it into the cluster. This can be accomplished via a few simple commands or via a prebaked machine image.
- If instead you use Kubernetes to decouple the teams from the specific machines they are using, you can forecast growth based on the aggregate growth of all [teams]. Combining [all team’s] variable growth rates into a single growth rate reduces statistical noise and produces a more reliable forecast of expected growth.
Scaling Development Teams with Microservices
Kubernetes provides numerous abstractions and APIs that make it easier to build these decoupled microservice architectures. (Hightower et al., 2017, pp. 6–7)
- Pods: Pods, or groups of containers, can group together container images developed by different teams into a single deployable unit.
- load balancing, naming, and discovery to isolate one microservice from another.
- Namespaces provide isolation and access control, so that each microservice can control the degree to which other services interact with it.
- Ingress objects provide an easy-to-use frontend that can combine multiple microservices into a single externalized API surface area.
Separation of Concerns for Consistency and Scaling
(Hightower et al., 2017, pp. 7–8)
- Decoupling enables your operations function to scale to managing many machines with a single small, focused team.
- An important aspect of this decoupling is that the container orchestration API becomes a crisp contract that separates the responsibilities of the application operator from the cluster orchestration operator. We call this the “not my monkey, not my circus” line.
- The application developer relies on the service-level agreement (SLA) delivered by the container orchestration API, without worrying about the details of how this SLA is achieved. Likewise, the container orchestration API reliability engineer focuses on delivering the orchestration API’s SLA without worrying about the applications that are running on top of it.
- Kubernetes-as-a-Service (KaaS) is available on Azure, Google Cloud (Google Container Engine). On AWS there is no “as a service” support yet but the kops project helps setting up the basic infrastructure.
Abstracting Your Infrastructure
Key points: (Hightower et al., 2017, p. 9)
- Transferring your application between environments, or even running in hybrid environments, is simply a matter of sending the declarative config to a new cluster.
- Kubernetes has a number of plug-ins that can abstract you from a particular cloud. For example, Kubernetes services know how to create load balancers on all major public clouds as well as several different private and physical infrastructures.
- Kubernetes PersistentVolumes and PersistentVolumeClaims can be used to abstract your applications away from specific storage implementations.
- To achieve this portability you need to avoid cloud-managed services (e.g., Amazon’s DynamoDB or Google’s Cloud Spanner), which means that you will be forced to deploy and manage open source storage solutions like Cassandra, MySQL, or MongoDB.
Efficiency
Key points: (Hightower et al., 2017, pp. 9–10)
- Because developers no longer think in terms of machines, their applications can be colocated on the same machines without impacting the applications themselves. This means that tasks from multiple users can be packed tightly onto fewer machines.
- Running a server incurs a cost based on power usage, cooling requirements, data center space, and raw compute power. Once a server is racked and powered on (or clicked and spun up), the meter literally starts running. Any idle CPU time is money wasted.
- Kubernetes provides tools that automate the distribution of applications across a cluster of machines ensuring higher levels of utilization than are possible with traditional tooling.
- Testing:
- A developer’s test environment can be quickly and cheaply created as a set of containers running in a personal view of a shared Kubernetes cluster (using a feature called namespaces).
- The cost you incur for such testing is dramatically lower.
Docker Containers
Containers fall into two main categories: (Hightower et al., 2017, pp. 15–16)
- System containers: seek to mimic virtual machines and often run a full boot process. They often include a set of system services typically found in a VM, such as ssh, cron, and syslog.
- Application containers: they commonly run a single application. While running a single application per container might seem like an unnecessary constraint, it provides the perfect level of granularity for composing scalable applications, and is a design philosophy that is leveraged heavily by pods.
Pitfalls (Hightower et al., 2017, pp. 16–17)
- Savings passwords/secretes to images
- Being unaware that when a layer B removes a file x on layer A, file x still exists and it is transmitted through the network every time that the image is pulled. The most volatile layers should be the last to be added
Public vs Private Docker Registry Pitfalls (Hightower et al., 2017, p. 18)
- Public: ideal for sharing images with the world, because they allow for easy, unauthenticated use of the container images.
- Private: best for storing your applications that are private to your service and that you don’t want the world to use.
Examples of registries: Docker Hub, Google Container Registry.
Examples (Hightower et al., 2017, p. 19)
Limiting memory
Limiting CPU resources
Tips
- Remove images after using them with
docker rmi
- Contemplate using the
docker-gc
tool
Deploying a Kubernetes Cluster
(Hightower et al., 2017, Chapter 3)
Google Container Service
Set default zone
Create cluster
Credentials
Azure Container Service
Set default zone
Create cluster
Credentials
Install kubectl tool
AWS
No KaaS as of book publication:
Minikube (Local)
Minikube provides an easy-to-use way to get a local Kubernetes cluster up running in a VM on your local laptop or desktop. Though this is attractive, minikube only creates a single-node cluster, which doesn’t quite demonstrate all of the aspects of a complete Kubernetes cluster.
Examples
gcloud container clusters create rato-cluster
gcloud components install kubectl
gcloud container clusters get-credentials rato-cluster
gcloud container clusters list
kubectl run hello-web --image=gcr.io/google-samples/hello-app:1.0 --port=8080
kubectl expose deployment hello-web --type="LoadBalancer"
kubectl get service hello-web
Obtain IP address (if <pending> it may take a while)
Version
kubectl version
Both the API and Server version
Checking Cluster Status
C:\Users\Ernie>kubectl get componentstatuses
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-1 Healthy {"health": "true"}
etcd-0 Healthy {"health": "true"}
* controller-manager: top management component
* scheduler: placing of pods onto different nodes in the cluster
* etcd: storage of API objects
List worker nodes
C:\Users\Ernie>kubectl get nodes
NAME STATUS AGE VERSION
gke-rato-cluster-default-pool-2dfe4285-3mrs Ready 1h v1.7.6
gke-rato-cluster-default-pool-2dfe4285-kdbs Ready 1h v1.7.6
gke-rato-cluster-default-pool-2dfe4285-zbpx Ready 1h v1.7.6
kubectl get all --all-namespaces=true
kubectl get all --namespace default
kubectl get all --namespace kube-system
C:\Users\Ernie>kubectl logs hello-web-967542450-jg652
BibTeX
@book{hightower2017,
Author = {Kelsey Hightower and Brendan Burns and Joe Beda},
Title = {{Kubernetes: Up and Running: Dive into the Future of Infrastructure}},
Publisher = {O'Reilly Media},
Year = {2017},
ISBN = {978-1-491-93567-5}
}
References
Hightower, K., Burns, B., Beda, J., 2017. Kubernetes: Up and Running: Dive into the Future of Infrastructure. O’Reilly Media.
Brendan Burns et al., “Borg, Omega, and Kubernetes: Lessons Learned from Three Container-Management Systems over a Decade,” ACM Queue 14 (2016): 70–93, available at http://bit.ly/2vIrL4S.↩