Kubernetes Kustomize:
Kustomize Overview:
- As a standalone tool, customizes Kubernetes objects through a kustomization file
- Makes declarative changes to application configurations and components and preserve the original base YAML files
- Group in a directory the Kubernetes resources that constitute your application, and then use Kustomize to copy and adapt these resource files to your environments and clusters.
- Starting with Kubernetes 1.14, kubectl supported declarative management of Kubernetes objects using kustomization files. oc integrates the Kustomization tool, as well.
- Features:
- - Generating resources from other sources
- - Setting cross-cutting fields for resources
- - Composing and customizing collections of resources
Kustomize File Structure:
Add to your code's base directory a kustomization.yaml file. The kustomization.yaml file has a list resource field to include all resource files. As the name implies, all resources in the base directory are a common resource set. The kustomization.yaml file can create a base application by composing all common resources by referring to one or more directories as bases.
Below is a very basic example showing the contents of a kustomization:
[admin@rocp ~]$ tree ./
base folder
├── configmap.yaml
├── deployment.yaml
├── secret.yaml
├── service.yaml
├── route.yaml
└── kustomization.yaml
[admin@rocp ~]$ cat ~/kustomcodedemo/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- configmap.yaml
- deployment.yaml
- secret.yaml
- service.yaml
- route.yaml
To render/view resources contained in kustomization:
[admin@rocp ~]$ kubectl kustomize ~/kustomcodedemo/base/
<review resources/yaml>
To apply the kustomation(s), use kubectl apply:
[admin@rocp ~]$ kubectl apply -k ~/kustomcodedemo/base/
Kustomize Overlays:
- Overlays declarative YAML artifacts or patches that override the general settings without modifying the original files
Overlay example:
Notes:
- Overlay folder is peer to the base folder
- The overlay folders contain relative references to the base code (e.g. ../../base).
- The dev overlay kustomization.yaml changes the namespace to dev-env.
- The test overlay kustomization.yaml changes the namespace to test-env, and contains the testing patches
- The prod overlay kustomization.yaml contains one patch loading the file patch.yaml, and has allowNameChange: true, so that patch.yaml can change the name.
[admin@rocp ~]$ tree ~/kustomcodedemo/
base (folder)
├── configmap.yaml
├── deployment.yaml
├── secret.yaml
├── service.yaml
├── route.yaml
└── kustomization.yaml
overlay (folder)
└── development
└── kustomization.yaml
└── testing
└── kustomization.yaml
└── production
├── kustomization.yaml
└── patch.yaml
[admin@rocp ~]$ cat ~/kustomcodedemo/overlay/development/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev-env
resources:
- ../../base
[admin@rocp ~]$ cat ~/kustomcodedemo/overlay/testing/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: test-env
patches:
- patch: |-
- op: replace
path: /metadata/name
value: mysql-pod-test
target:
kind: Deployment
name: frontend
- patch: |-
- op: replace
path: /spec/replicas
value: 15
target:
kind: Deployment
name: mysql-pod
resources:
- ../../base
commonLabels:
env: test
[admin@rocp ~]$ cat ~/kustomcodedemo/overlay/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prod-env
patches:
- path: patch.yaml
target:
kind: Deployment
name: mysql-pod
options:
allowNameChange: true
resources:
- ../../base
commonLabels:
env: prod
[admin@rocp ~]$ cat ~/kustomcodedemo/overlay/production/patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql-pod-prod
spec:
replicas: 5
Apply the production overlay:
[admin@rocp ~]$ kubectl apply -k ~/kustomcodedemo/overlay/production
deployment.apps/mysql-pod-prod created
...
Delete the testing overlay:
[admin@rocp ~]$ oc delete -k ~/kustomcodedemo/overlay/testing
<view pod containers deleted>
Configuration and Sensitive Data with Kustomize:
ConfigMaps and Secrets are used to store configuration or sensitive data that are used by other Kubernetes objects, such as Deployment Pods. The source of truth of ConfigMaps or Secrets are usually external to a cluster, such as a .properties file or an SSH keyfile. Kustomize has secretGenerator and configMapGenerator, which generate Secret and ConfigMap from files or literals.
Using ConfigMaps Notes:
- All entries in an application.properties become a single key in the ConfigMap generated.
- Each variable in the .env file becomes a separate key in the ConfigMap generated.
- ConfigMaps can also be generated from literal key-value pairs; add an entry to the literals list in configMapGenerator.
Using Secrets Notes:
- Generate Secrets from files or literal key-value pairs.
- To generate a Secret from a file, add an entry to the files list in secretGenerator
Using ConfigMaps:
Example configMapGenerator that loads from an application.properties file:
[admin@rocp ~]$ vi ~/kustomcodemap/base/application.properties
FOO=Bar
<esc>:wq (to save)
[admin@rocp ~]$ vi ~/kustomcodemap/base/kustomization.yaml
configMapGenerator:
- name: example-configmap-1
files:
- application.properties
<esc>:wq (to save)
To validate your configMap generation:
[admin@rocp ~]$ kubectl kustomize ~/kustomcodemap/base/
apiVersion: v1
data:
application.properties: |
FOO=Bar
kind: ConfigMap
metadata:
name: example-configmap-1-1abcd7891e
Example configMapGenerator that loads from an .env file:
[admin@rocp ~]$ vi ~/kustomcodemap/base/.env
FOO=Bar
<esc>:wq (to save)
[admin@rocp ~]$ vi ~/kustomcodemap/base/kustomization.yaml
configMapGenerator:
- name: example-configmap-1
envs:
- .env
<esc>:wq (to save)
To validate your configMap generation:
[admin@rocp ~]$ kubectl kustomize ~/kustomcodemap/base/
apiVersion: v1
data:
FOO: Bar
kind: ConfigMap
metadata:
name: example-configmap-1-23abcd123a
Example of a configMap from a literal key-value pair:
[admin@rocp ~]$ vi ~/kustomcodemap/base/kustomization.yaml
configMapGenerator:
- name: example-configmap-2
literals:
- FOO=Bar
<esc>:wq (to save)
To validate your configMap generation:
[admin@rocp ~]$ kubectl kustomize ~/kustomcodemap/base/
apiVersion: v1
data:
FOO: Bar
kind: ConfigMap
metadata:
name: example-configmap-2-a1abcde2ab
Example of application.properties, deployment, with a generated ConfigMap:
[admin@rocp ~]$ vi ~/kustomcodemap/base/application.properties
FOO=Bar
<esc>:wq (to save)
[admin@rocp ~]$ vi ~/kustomcodemap/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
labels:
app: my-app
spec:
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: app
image: my-app
volumeMounts:
- name: config
mountPath: /config
volumes:
- name: config
configMap:
name: example-configmap-1
<esc>:wq (to save)
[admin@rocp ~]$ vi ~/kustomcodemap/base/kustomization.yaml
resources:
- deployment.yaml
configMapGenerator:
- name: example-configmap-1
files:
- application.properties
<esc>:wq (to save)
To validate your configMap generation:
[admin@rocp ~]$ kubectl kustomize ~/kustomcodemap/base/
apiVersion: v1
data:
application.properties: |
FOO=Bar
kind: ConfigMap
metadata:
name: example-configmap-1-g4hk9g2ff8
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: my-app
name: my-app
spec:
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
- image: my-app
name: app
volumeMounts:
- mountPath: /config
name: config
volumes:
- configMap:
name: example-configmap-1-a1bc2d3ef4
name: config
Example of configMapGenerator with all three types (file, envs, and literal key):
[admin@rocp ~]$ cd ~/kustomconfigmaps/
[admin@rocp ~]$ vi application.properties
Day=Monday
Enabled=True
<esc>:wq (to save)
[admin@rocp ~]$ vi configmap2.env
Greet=Welcome
Enable=True
<esc>:wq (to save)
[admin@rocp ~]$ cat kustomization.yaml
...
configMapGenerator:
- name: configmap-props
files:
- application.properties
- name: configmap-envs
envs:
- configmap2.env
- name: configmap-literals
literals:
- name="configmap-literal"
- description="literal key-value pair"
...
[admin@rocp ~]$ kubectl kustomize .
apiVersion: v1
data:
application.properties: |
Day=Monday
Enable=True
kind: ConfigMap
metadata:
name: configmap-1-5g2mh569b5 1
---
apiVersion: v1
data:
Enable: "True"
Greet: Welcome
kind: ConfigMap
metadata:
name: configmap-2-92m84tg9kt 2
---
apiVersion: v1
data:
description: literal key-value pair
name: configmap-3
kind: ConfigMap
metadata:
name: configmap-3-k7g7d5bffd 3
---
...
Using Secrets:
Example Secret that loads from an password.txt file:
[admin@rocp ~]$ vi ~/kustomcodeshh/base/password.txt
username=admin
password=reallygoodpassword
<esc>:wq (to save)
[admin@rocp ~]$ vi ~/kustomcodeshh/base/kustomization.yaml
secretGenerator:
- name: example-secret-1
files:
- password.txt
<esc>:wq (to save)
Example Secret that loads from literal list:
[admin@rocp ~]$ vi ~/kustomcodeshh/base/password.txt
[admin@rocp ~]$ vi ~/kustomcodeshh/base/kustomization.yaml
- name: example-secret-2
literals:
- username=admin
- password=reallygoodpassword
<esc>:wq (to save)
To validate your configMap generation:
[admin@rocp ~]$ kubectl kustomize ~/kustomcodeshh/base/
apiVersion: v1
data:
password: reallygoodpassword
username: admin
kind: Secret
metadata:
name: example-secret-2-t52t6g96d8
type: Opaque
To apply:
[admin@rocp ~]$ kubectl apply -k ~/kustomcodeshh/base/
<confirm created okay>
View the created pod:
[admin@rocp ~]$ oc get all
<view containers>
Example of secret generators with all three types (file, envs, and literal key):
[admin@rocp ~]$ cd ~/kustomsecrets2/
[admin@rocp ~]$ cat kustomization.yaml
...
secretGenerator:
- name: secret-file
files:
- password.txt
- name: secret-envs
envs:
- secret-mysql.env
- name: secret-literal
literals:
- MYSQL_ADMIN_PASSWORD=postgres
- MYSQL_DB=mysql
- MYSQL_USER=user
- MYSQL_PASS=reallygoodpassword
configMapGenerator:
- name: db-config
literals:
- DB_HOST=database
- DB_PORT=5432
...
ConfigMap generatorOptions:
- Alter the default behavior of Kustomize generators.
- Workload resources, e.g. deployments, do not detect changes to configuration maps and secrets unless the name changes.
- By default, a kustomize configMapGenerator and a secretGenerator append a hash suffix to the name of the generated resource(s), which means the name changes each apply, which means the deployment updates.
- Use generated-disable-suffix to disable the hash suffix
Example of configMapGenerator without a hash each time:
[admin@rocp ~]$ vi ~/kustomdisablegeneratedhash/kustomization.yaml
...
configMapGenerator:
- name: my-configmap
literals:
- name="configmap-nohash"
- description="literal key-value pair"
generatorOptions:
disableNameSuffixHash: true
labels:
type: generated-disabled-suffix
annotations:
note: generated-disabled-suffix
...
[admin@rocp ~]$ kubectl kustomize ~/kustomdisablegeneratedhash/
apiVersion: v1
data:
description: literal key-value pair
name: configmap-nohash
kind: ConfigMap
metadata:
annotations:
note: generated-disabled-suffix
labels:
type: generated-disabled-suffix
name: my-configmap
...
previous page
|