Kubernetes Workloads - Deployments, StatefulSets, DaemonSets, Jobs and CronJobs

1. Deployments in Kubernetes

• Deployments are a fundamental building block for managing containerized applications in a Kubernetes cluster.

• A Deployment is an API resource and a higher-level abstraction that provides a declarative way to manage and scale a set of identical pods. You describe a desired state in a Deployment, and the Deployment Controller changes the actual state to the desired state at a controlled rate.

Common scenarios where deployments are used include the following:

• Create a Deployment to roll out a ReplicaSet – A Deployment manages a Replica Set, which, in turn, ensures that a specified number of pod replicas (instances) are running and healthy at all times. It allows you to scale your application by adjusting the desired number of replicas.

• Declare the new state of the Pods – Deployments are configured using declarative YAML or JSON files, which specify the desired state of the application. Kubernetes continually works to ensure that the actual state of the application matches this desired state.

• Scale up the Deployment to facilitate more load – You can easily scale the number of replicas up or down based on the desired workload. Deployments can automatically adjust the number of replicas to match the specified number.

• Pause the rollout of a Deployment – Deployments support rolling updates, allowing you to make changes to your application without causing downtime. During an update, new pods are gradually created, and old pods are scaled down, ensuring a smooth transition from one version to another.

• Rollback to an earlier Deployment revision – If a rolling update fails or results in issues, Kubernetes deployments provide automated rollback mechanisms to revert to the previous stable version of the application.

• Use the status of the Deployment – If a pod fails or becomes unresponsive, the Deployment will automatically replace it with a new pod, maintaining the desired number of replicas.

• Running nginx application using deployment.

• Deploy a simple Nginx application,

• Create POD using deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

• Apply deployment.yml file,

kubectl apply -f deployment.yml

• Create service for the application using service.yml file,

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
      nodePort: 30001

• Apply services.yml file,

kubectl apply -f service.yml

• Verify service status,

kubectl get svc -o wide

• Expose port to access application on browser,

kubectl port-forward svc/nginx-service 8081:80 --address=0.0.0.0 &

• Now try to access,

---

2. StatefulSets in Kubernetes

• A StatefulSet is the Kubernetes controller used to run the stateful application as containers (Pods) in the Kubernetes cluster. StatefulSets assign a sticky identity—an ordinal number starting from zero—to each Pod instead of assigning random IDs for each replica Pod.

• A new Pod is created by cloning the previous Pod’s data. If the previous Pod is in the pending state, then the new Pod will not be created. If you delete a Pod, it will delete the Pod in reverse order, not in random order. For example, if you had four replicas and you scaled down to three, it will delete the Pod numbered.

• The diagram below shows how the Pod is numbered from zero and how Kubernetes persistent volume is attached to the Pod in the StatefulSets.

When to Use StatefulSets?

There are several reasons to consider using StatefulSets. Here are two examples:

1. Assume you deployed a MySQL database in the Kubernetes cluster and scaled this to three replicas, and a frontend application wants to access the MySQL cluster to read and write data. The read request will be forwarded to three Pods. However, the write request will only be forwarded to the first (primary) Pod, and the data will be synced with the other Pods. You can achieve this by using StatefulSets.

2. Deleting or scaling down a StatefulSet will not delete the volumes associated with the stateful application. This gives you your data safety. If you delete the MySQL Pod or if the MySQL Pod restarts, you can have access to the data in the same volume.

Create a Headless Service

StatefulSets require a headless service for stable network identities. Create a nginx-headless.yml file,

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx

Apply this configuration,

kubectl apply -f nginx-headless.yml

Create the StatefulSet

Create a nginx-statefulset.yml file,

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx
spec:
  serviceName: "nginx"
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 1Gi

This StatefulSet will create 3 replicas of Nginx, each with its own persistent volume.

• Apply the StatefulSet,

kubectl apply -f nginx-statefulset.yml

• Verify the StatefulSet,

Check the StatefulSet status,

kubectl get statefulsets

• Check the pods created,

kubectl get pods -l app=nginx

You should see three pods with names like nginx-0, nginx-1, and nginx-2.

• Scale the StatefulSet,

Scale the StatefulSet to 5 replicas,

kubectl scale statefulset nginx --replicas=5

You should see nginx-3 and nginx-4 added.

To delete the StatefulSet and its associated resources,

kubectl delete statefulset nginx
kubectl delete service nginx

---

3. DaemonSets in Kubernetes

A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.

Some typical uses of a DaemonSet are:

• running a cluster storage daemon on every node

• running a logs collection daemon on every node

• running a node monitoring daemon on every node

In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon. A more complex setup might use multiple DaemonSets for a single type of daemon, but with different flags and/or different memory and cpu requests for different hardware types.

Creating a daemonset.yml,

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
spec:
  selector:
    matchLabels:
      name: fluentd
  template:
    metadata:
      labels:
        name: fluentd
    spec:
      containers:
        - name: fluentd-elasticsearch
          image: quay.io/fluentd_elasticsearch/fluentd:latest

• The manifest’s spec.selector field must reference the labels assigned to the Pod template in spec.template. The template is a regular Kubernetes Pod spec that defines the containers the DaemonSet will run.

• Apply daemonset.yml

kubectl apply -f daemonset.yml

• Wait while the DaemonSet’s Pods start, then use the kubectl get pods command with the -o wide option to list the Pods and the Nodes that they’re scheduled to,

kubectl get pods -o wide

You can see that Kubernetes has automatically scheduled a Fluentd Pod onto each of the three Nodes in your cluster.

---

4. Jobs in Kubernetes

• A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. As pods successfully complete, the Job tracks the successful completions.

• When a specified number of successful completions is reached, the task (ie, Job) is complete. Deleting a Job will clean up the Pods it created. Suspending a Job will delete its active Pods until the Job is resumed again.

• A simple case is to create one Job object in order to reliably run one Pod to completion. The Job object will start a new Pod if the first Pod fails or is deleted (for example due to a node hardware failure or a node reboot).

Here's a basic example of a Kubernetes Job that runs a single Pod to print "Hello, World!" and then exits.

• Create job.yml

apiVersion: batch/v1
kind: Job
metadata:
  name: hello-job
spec:
  template:
    spec:
      containers:
      - name: hello
        image: busybox
        command: ["echo", "Hello, World!"]
      restartPolicy: Never
  backoffLimit: 4

• Apply file,

kubectl apply -f job.yml

• List out Jobs,

kubectl get jobs

• Check logs on job,

kubectl logs jobs/<job_name>

---

5. CronJobs in Kubernetes

• A CronJob creates Jobs on a repeating schedule.

• CronJob is meant for performing regular scheduled actions such as backups, report generation, and so on. One CronJob object is like one line of a crontab (cron table) file on a Unix system. It runs a Job periodically on a given schedule, written in Cron format.

• Here’s an example of a CronJob that runs every minute and prints the current date and time,

apiVersion: batch/v1
kind: CronJob
metadata:
  name: date-cronjob
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: date
            image: busybox
            command: ["sh", "-c", "date; echo Hello from the Kubernetes CronJob"]
          restartPolicy: OnFailure

• Apply file,

kubectl apply -f cronjob.yml

• Check the Jobs created by the CronJob,

kubectl get jobs --watch

---

Connect With Me

Thank you for reading. I hope you were able to understand and learn something new from my blog.

Happy Learning!

LinkedIn | GitHub