Application Environment, Configuration, and Security (25%)

Discover and use resources that extend Kubernetes (CRD, Operators)
Understand authentication, authorization and admission control
Understand requests, limits, quotas
Understand ConfigMaps
Define resource requirements
Create & consume Secrets
Understand ServiceAccounts
Understand Application Security (SecurityContexts, Capabilities, etc.)

Discover and use resources that extend Kubernetes (CRD)

Complete Practical Example of Kubernetes CRD, Operator & Kubebuilder^Blog

Create a CustomResourceDefinition manifest file for an Operator with the following specifications :

Name : operators.stable.example.com
Group : stable.example.com
Schema: <email: string><name: string><age: integer>
Scope: Namespaced
Names: <plural: operators><singular: operator><shortNames: op>
Kind: Operator

show

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must match the spec fields below, and be in the form: <plural>.<group>
  name: operators.stable.example.com
spec:
  group: stable.example.com
  versions:
    - name: v1
      served: true
      # One and only one version must be marked as the storage version.
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                email:
                  type: string
                name:
                  type: string
                age:
                  type: integer
  scope: Namespaced
  names:
    plural: operators
    singular: operator
    # kind is normally the CamelCased singular type. Your resource manifests use this.
    kind: Operator
    shortNames:
    - op

Create the CRD resource in the K8S API

show

kubectl apply -f operator-crd.yml

Create custom object from the CRD

Name : operator-sample
Kind: Operator
Spec:
- email: operator-sample@stable.example.com
- name: operator sample
- age: 30

show

apiVersion: stable.example.com/v1
kind: Operator
metadata:
  name: operator-sample
spec:
  email: operator-sample@stable.example.com
  name: "operator sample"
  age: 30

kubectl apply -f operator.yml

Listing operator

show

Use singular, plural and short forms

kubectl get operators
or
kubectl get operator
or
kubectl get op

Understanding and defining resource requirements, limits, and quotas

Complete Practical Example of Kubernetes CRD, Operator & Kubebuilder^Blog

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Assign CPU Resources to Containers and Pods^{Kubernetes Documentation}

Create an nginx pod with requests cpu=100m,memory=256Mi and limits cpu=200m,memory=512Mi

show

kubectl run nginx --image=nginx --dry-run=client -o yaml > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "256Mi"
        cpu: "100m"
      limits:    
        memory: "512Mi"
        cpu: "200m"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Limit Ranges

kubernetes.io > Documentation > Concepts > Policies > Limit Ranges (https://kubernetes.io/docs/concepts/policy/limit-range/)^{Kubernetes Documentation}

Create a namespace named limitrange with a LimitRange that limits pod memory to a max of 500Mi and min of 100Mi

show

kubectl create ns one

vi 1.yaml

apiVersion: v1
kind: LimitRange
metadata:
  name: ns-memory-limit
  namespace: one
spec:
  limits:
  - max: # max and min define the limit range
      memory: "500Mi"
    min:
      memory: "100Mi"
    type: Container

kubectl apply -f 1.yaml

Describe the namespace limitrange

show

kubectl describe limitrange ns-memory-limit -n one

Create an nginx pod that requests 250Mi of memory in the limitrange namespace

show

vi 2.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: one
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "250Mi"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

kubectl apply -f 2.yaml

Resource Quotas

kubernetes.io > Documentation > Concepts > Policies > Resource Quotas (https://kubernetes.io/docs/concepts/policy/resource-quotas/)^{Kubernetes Documentation}

Create ResourceQuota in namespace `one` with hard requests `cpu=1`, `memory=1Gi` and hard limits `cpu=2`, `memory=2Gi`.

show

Create the namespace:

kubectl create ns one

Create the ResourceQuota

vi rq-one.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: my-rq
  namespace: one
spec:
  hard:
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi

kubectl apply -f rq-one.yaml

or

kubectl create quota my-rq --namespace=one --hard=requests.cpu=1,requests.memory=1Gi,limits.cpu=2,limits.memory=2Gi

Attempt to create a pod with resource requests `cpu=2`, `memory=3Gi` and limits `cpu=3`, `memory=4Gi` in namespace `one`

show

vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: one
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "3Gi"
        cpu: "2"
      limits:
        memory: "4Gi"
        cpu: "3"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

kubectl create -f pod.yaml

Expected error message:

Error from server (Forbidden): error when creating "pod.yaml": pods "nginx" is forbidden: exceeded quota: my-rq, requested: limits.cpu=3,limits.memory=4Gi,requests.cpu=2,requests.memory=3Gi, used: limits.cpu=0,limits.memory=0,requests.cpu=0,requests.memory=0, limited: limits.cpu=2,limits.memory=2Gi,requests.cpu=1,requests.memory=1Gi

Create a pod with resource requests `cpu=0.5`, `memory=1Gi` and limits `cpu=1`, `memory=2Gi` in namespace `one`

show

vi pod2.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: one
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "1Gi"
        cpu: "0.5"
      limits:
        memory: "2Gi"
        cpu: "1"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

kubectl create -f pod2.yaml

Show the ResourceQuota usage in namespace one

kubectl get resourcequota -n one

NAME    AGE   REQUEST                                          LIMIT
my-rq   10m   requests.cpu: 500m/1, requests.memory: 3Mi/1Gi   limits.cpu: 1/2, limits.memory: 4Mi/2Gi

Understand Secrets

Kubernetes Secrets: A complete guide to securely managing sensitive information^Blog
kubernetes.io > Documentation > Concepts > Configuration > Secrets^{Kubernetes Documentation}

Create a secret called mysecret with the values password=mypass

show

kubectl create secret generic mysecret --from-literal=password=mypass

Create a secret called mysecret2 that gets key/value from a file

Create a file called username with the value admin:

echo -n admin > username

show

kubectl create secret generic mysecret2 --from-file=username

Get the value of mysecret2

show

kubectl get secret mysecret2 -o yaml
echo -n YWRtaW4= | base64 -d # on MAC it is -D, which decodes the value and shows 'admin'

Alternative using --jsonpath:

kubectl get secret mysecret2 -o jsonpath='{.data.username}' | base64 -d  # on MAC it is -D

Alternative using --template:

kubectl get secret mysecret2 --template '{{.data.username}}' | base64 -d  # on MAC it is -D

Alternative using jq:

kubectl get secret mysecret2 -o json | jq -r .data.username | base64 -d  # on MAC it is -D

Create an nginx pod that mounts the secret mysecret2 in a volume on path /etc/foo

show

kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  volumes: # specify the volumes
  - name: foo # this name will be used for reference inside the container
    secret: # we want a secret
      secretName: mysecret2 # name of the secret - this must already exist on pod creation
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    volumeMounts: # our volume mounts
    - name: foo # name on pod.spec.volumes
      mountPath: /etc/foo #our mount path
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

kubectl create -f pod.yaml
kubectl exec -it nginx -- /bin/bash
ls /etc/foo  # shows username
cat /etc/foo/username # shows admin

Delete the pod you just created and mount the variable 'username' from secret mysecret2 onto a new nginx pod in env variable called 'USERNAME'

show

kubectl delete po nginx
kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    env: # our env variables
    - name: USERNAME # asked name
      valueFrom:
        secretKeyRef: # secret reference
          name: mysecret2 # our secret's name
          key: username # the key of the data in the secret
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

kubectl create -f pod.yaml
kubectl exec -it nginx -- env | grep USERNAME | cut -d '=' -f 2 # will show 'admin'

Create a Secret named 'ext-service-secret' in the namespace 'secret-ops'. Then, provide the key-value pair API_KEY=LmLHbYhsgWZwNifiqaRorH8T as literal.

show

export ns="-n secret-ops"
export do="--dry-run=client -oyaml"
k create secret generic ext-service-secret --from-literal=API_KEY=LmLHbYhsgWZwNifiqaRorH8T $ns $do > sc.yaml
k apply -f sc.yaml

Consuming the Secret. Create a Pod named 'consumer' with the image 'nginx' in the namespace 'secret-ops' and consume the Secret as an environment variable. Then, open an interactive shell to the Pod, and print all environment variables.

show

export ns="-n secret-ops"
export do="--dry-run=client -oyaml"
k run consumer --image=nginx $ns $do > nginx.yaml
vi nginx.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: consumer
  name: consumer
  namespace: secret-ops
spec:
  containers:
  - image: nginx
    name: consumer
    resources: {}
    env:
    - name: API_KEY
      valueFrom:
        secretKeyRef:
          name: ext-service-secret
          key: API_KEY
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

k exec -it $ns consumer -- /bin/sh
#env

Create a Secret named 'my-secret' of type 'kubernetes.io/ssh-auth' in the namespace 'secret-ops'. Define a single key named 'ssh-privatekey', and point it to the file 'id_rsa' in this directory.

show

#Tips, export to variable
export do="--dry-run=client -oyaml"
export ns="-n secret-ops"

#if id_rsa file didn't exist.
ssh-keygen

k create secret generic my-secret $ns --type="kubernetes.io/ssh-auth" --from-file=ssh-privatekey=id_rsa $do > sc.yaml
k apply -f sc.yaml

Create a Pod named 'consumer' with the image 'nginx' in the namespace 'secret-ops', and consume the Secret as Volume. Mount the Secret as Volume to the path /var/app with read-only access. Open an interactive shell to the Pod, and render the contents of the file.

show

#Tips, export to variable
export ns="-n secret-ops"
export do="--dry-run=client -oyaml"
k run consumer --image=nginx $ns $do > nginx.yaml
vi nginx.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: consumer
  name: consumer
  namespace: secret-ops
spec:
  containers:
    - image: nginx
      name: consumer
      resources: {}
      volumeMounts:
        - name: foo
          mountPath: "/var/app"
          readOnly: true
  volumes:
    - name: foo
      secret:
        secretName: my-secret
        optional: true
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

k exec -it $ns consumer -- /bin/sh
# cat /var/app/ssh-privatekey
# exit

Understand ServiceAccounts

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure Service Accounts for Pods^{Kubernetes Documentation}

See all the service accounts of the cluster in all namespaces

show

kubectl get sa --all-namespaces

Alternatively

kubectl get sa -A

Create a new serviceaccount called 'myuser'

show

kubectl create sa myuser

Alternatively:

# let's get a template easily
kubectl get sa default -o yaml > sa.yaml
vim sa.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: myuser

kubectl create -f sa.yaml

Create an nginx pod that uses 'myuser' as a service account

show

kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  serviceAccountName: myuser # we use pod.spec.serviceAccountName
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

or

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  serviceAccount: myuser # we use pod.spec.serviceAccount
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

kubectl create -f pod.yaml
kubectl describe pod nginx # will see that a new secret called myuser-token-***** has been mounted

Generate an API token for the service account 'myuser'

show

kubectl create token myuser

Understand ConfigMaps

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure a Pod to Use a ConfigMap^{Kubernetes Documentation}

Real World Example: Configuring Redis with ConfigMap^Blog

Create a configmap named config with values foo=lala,foo2=lolo

show

kubectl create configmap config --from-literal=foo=lala --from-literal=foo2=lolo

Display its values

show

kubectl get cm config -o yaml
# or
kubectl describe cm config

Create and display a configmap from a file

Create the file with

echo -e "foo3=lili\nfoo4=lele" > config.txt

show

kubectl create cm configmap2 --from-file=config.txt
kubectl get cm configmap2 -o yaml

Create and display a configmap from a .env file

Create the file with the command

echo -e "var1=val1\n# this is a comment\n\nvar2=val2\n#anothercomment" > config.env

show

kubectl create cm configmap3 --from-env-file=config.env
kubectl get cm configmap3 -o yaml

Create and display a configmap from a file, giving the key 'special'

Create the file with

echo -e "var3=val3\nvar4=val4" > config4.txt

show

kubectl create cm configmap4 --from-file=special=config4.txt
kubectl describe cm configmap4
kubectl get cm configmap4 -o yaml

Create a configMap called 'options' with the value var5=val5. Create a new nginx pod that loads the value from variable 'var5' in an env variable called 'option'

show

kubectl create cm options --from-literal=var5=val5
kubectl run nginx --image=nginx --restart=Never --dry-run=client -o yaml > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    env:
    - name: option # name of the env variable
      valueFrom:
        configMapKeyRef:
          name: options # name of config map
          key: var5 # name of the entity in config map
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

kubectl create -f pod.yaml
kubectl exec -it nginx -- env | grep option # will show 'option=val5'

Create a configMap 'anotherone' with values 'var6=val6', 'var7=val7'. Load this configMap as env variables into a new nginx pod

show

kubectl create configmap anotherone --from-literal=var6=val6 --from-literal=var7=val7
kubectl run --restart=Never nginx --image=nginx -o yaml --dry-run=client > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    envFrom: # different than previous one, that was 'env'
    - configMapRef: # different from the previous one, was 'configMapKeyRef'
        name: anotherone # the name of the config map
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

kubectl create -f pod.yaml
kubectl exec -it nginx -- env

Create a configMap 'cmvolume' with values 'var8=val8', 'var9=val9'. Load this as a volume inside an nginx pod on path '/etc/lala'. Create the pod and 'ls' into the '/etc/lala' directory.

show

kubectl create configmap cmvolume --from-literal=var8=val8 --from-literal=var9=val9
kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  volumes: # add a volumes list
  - name: myvolume # just a name, you'll reference this in the pods
    configMap:
      name: cmvolume # name of your configmap
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    volumeMounts: # your volume mounts are listed here
    - name: myvolume # the name that you specified in pod.spec.volumes.name
      mountPath: /etc/lala # the path inside your container
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

kubectl create -f pod.yaml
kubectl exec -it nginx -- /bin/sh
cd /etc/lala
ls # will show var8 var9
cat var8 # will show val8

Understand SecurityContexts

Update pod ubuntu-sleeper to run as Root user and with the SYS_TIME capability.Note: Only make the necessary changes. Do not modify the name of the pod.

show

kubectl delete po ubuntu-sleeper
vi  <file-name>.yaml

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu-sleeper
  namespace: default
spec:
  containers:
  - command:
    - sleep
    - "4800"
    image: ubuntu
    name: ubuntu-sleeper
    securityContext:
      capabilities:
        add: ["SYS_TIME"]

kubectl apply -f <file-name>.yaml

Files

b.application_environment_configuration_security.md

Latest commit

History

b.application_environment_configuration_security.md

File metadata and controls

Application Environment, Configuration, and Security (25%)

Discover and use resources that extend Kubernetes (CRD)

Create a CustomResourceDefinition manifest file for an Operator with the following specifications :

Create the CRD resource in the K8S API

Create custom object from the CRD

Listing operator

Understanding and defining resource requirements, limits, and quotas

Create an nginx pod with requests cpu=100m,memory=256Mi and limits cpu=200m,memory=512Mi

Limit Ranges

Create a namespace named limitrange with a LimitRange that limits pod memory to a max of 500Mi and min of 100Mi

Describe the namespace limitrange

Create an nginx pod that requests 250Mi of memory in the limitrange namespace

Resource Quotas

Create ResourceQuota in namespace one with hard requests cpu=1, memory=1Gi and hard limits cpu=2, memory=2Gi.

Attempt to create a pod with resource requests cpu=2, memory=3Gi and limits cpu=3, memory=4Gi in namespace one

Create a pod with resource requests cpu=0.5, memory=1Gi and limits cpu=1, memory=2Gi in namespace one

Understand Secrets

Create a secret called mysecret with the values password=mypass

Create a secret called mysecret2 that gets key/value from a file

Get the value of mysecret2

Create an nginx pod that mounts the secret mysecret2 in a volume on path /etc/foo

Delete the pod you just created and mount the variable 'username' from secret mysecret2 onto a new nginx pod in env variable called 'USERNAME'

Create a Secret named 'ext-service-secret' in the namespace 'secret-ops'. Then, provide the key-value pair API_KEY=LmLHbYhsgWZwNifiqaRorH8T as literal.

Consuming the Secret. Create a Pod named 'consumer' with the image 'nginx' in the namespace 'secret-ops' and consume the Secret as an environment variable. Then, open an interactive shell to the Pod, and print all environment variables.

Create a Secret named 'my-secret' of type 'kubernetes.io/ssh-auth' in the namespace 'secret-ops'. Define a single key named 'ssh-privatekey', and point it to the file 'id_rsa' in this directory.

Create a Pod named 'consumer' with the image 'nginx' in the namespace 'secret-ops', and consume the Secret as Volume. Mount the Secret as Volume to the path /var/app with read-only access. Open an interactive shell to the Pod, and render the contents of the file.

Understand ServiceAccounts

See all the service accounts of the cluster in all namespaces

Create a new serviceaccount called 'myuser'

Create an nginx pod that uses 'myuser' as a service account

Generate an API token for the service account 'myuser'

Understand ConfigMaps

Create a configmap named config with values foo=lala,foo2=lolo

Display its values

Create and display a configmap from a file

Create and display a configmap from a .env file

Create and display a configmap from a file, giving the key 'special'

Create a configMap called 'options' with the value var5=val5. Create a new nginx pod that loads the value from variable 'var5' in an env variable called 'option'

Create a configMap 'anotherone' with values 'var6=val6', 'var7=val7'. Load this configMap as env variables into a new nginx pod

Create a configMap 'cmvolume' with values 'var8=val8', 'var9=val9'. Load this as a volume inside an nginx pod on path '/etc/lala'. Create the pod and 'ls' into the '/etc/lala' directory.

Understand SecurityContexts

Update pod ubuntu-sleeper to run as Root user and with the SYS_TIME capability.Note: Only make the necessary changes. Do not modify the name of the pod.

Create ResourceQuota in namespace `one` with hard requests `cpu=1`, `memory=1Gi` and hard limits `cpu=2`, `memory=2Gi`.

Attempt to create a pod with resource requests `cpu=2`, `memory=3Gi` and limits `cpu=3`, `memory=4Gi` in namespace `one`

Create a pod with resource requests `cpu=0.5`, `memory=1Gi` and limits `cpu=1`, `memory=2Gi` in namespace `one`