Kubelet configuration files

2-minute read

Intro

This is an effort to bring all configuration files for kubelet together, to see a full picture.

It is based on official documentation: https://v1-16.docs.kubernetes.io/docs/setup/production-environment/tools/kubeadm/kubelet-integration/#the-kubelet-drop-in-file-for-systemd

Versions:

  • Ubuntu 16.04.6 LTS (Xenial Xerus)
  • Kubernetes 1.16 - currently used for CKA exam, although 1.17 is already released

Files

/etc/kubernetes/kubelet.conf

Client certificates that allow the kubelet to communicate with the API server.

Written during:

  • kubeadm init - with baseline cluster-wide configuration for all kubelets in the cluster

  • kubeadm join - with unique credential for a node created during TLS Bootstrap based on bootstrap-kubelet.conf file (see below)

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ...
    server: https://172.28.128.1:6443
  name: default-cluster
contexts:
- context:
    cluster: default-cluster
    namespace: default
    user: default-auth
  name: default-context
current-context: default-context
kind: Config
preferences: {}
users:
- name: default-auth
  user:
    client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem
    client-key: /var/lib/kubelet/pki/kubelet-client-current.pem

/etc/kubernetes/bootstrap-kubelet.conf

KubeConfig file written by kubeadm join, contains a CA certificate and Bootstrap Token.
kubelet uses it to perform the TLS Bootstrap and obtain a unique credential, which is then stored in /etc/kubernetes/kubelet.conf.

/var/lib/kubelet/config.yaml

File created during kubeadm init, containing the kubelet’s ComponentConfig.
It is also uploaded as a ConfigMap in the cluster.

address: 0.0.0.0
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
...

/var/lib/kubelet/kubeadm-flags.env

A list of flags to pass to the kubelet when it starts.

KUBELET_KUBEADM_ARGS="--cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1"

/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

This is a configuration for how systemd should run the kubelet.
kubeadm CLI command never touches this drop-in file.
It specifies the default locations for all of the files managed by kubeadm for the kubelet.

# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

Resulting process will look like (as seen by ps -ef ...):

/usr/bin/kubelet \
    --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf \
    --kubeconfig=/etc/kubernetes/kubelet.conf \
    --config=/var/lib/kubelet/config.yaml \
    --cgroup-driver=systemd \
    --network-plugin=cni \
    --pod-infra-container-image=k8s.gcr.io/pause:3.1

/etc/default/kubelet (for DEBs)

Can contain user-specified flag overrides via KUBELET_EXTRA_ARGS env var.
This var is the last in the flag chain (see ExecStart above).
So it has the highest priority in the event of conflicting settings.

References