Node taint

Node taint taints the node by applying the desired effect. Only the resources that contain the corresponding tolerations can bypass the taints.

Use cases

• Node taint fault verifies the resilience of applications when a certain taint is added to a node.
• It simulates loss of critical services (or node-crash).
• It verifies resource budgeting on cluster nodes (whether request(or limit) settings are honored on the available nodes).
• It verifies whether topology constraints are adhered to (node selectors, tolerations, zone distribution, affinity(or anti-affinity) policies) or not.

Permissions required

Below is a sample Kubernetes role that defines the permissions required to execute the fault.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: hce
  name: node-taint
spec:
  definition:
    scope: Cluster
permissions:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "delete", "get", "list", "patch", "deletecollection", "update"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "get", "list", "patch", "update"]
  - apiGroups: [""]
    resources: ["chaosEngines", "chaosExperiments", "chaosResults"]
    verbs: ["create", "delete", "get", "list", "patch", "update"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["get", "list", "create"]
  - apiGroups: ["batch"]
    resources: ["jobs"]
    verbs: ["create", "delete", "get", "list", "deletecollection"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "patch", "update"]
  - apiGroups: [""]
    resources: ["pod eviction"]
    verbs: ["get", "list", "create"]

Prerequisites

• Kubernetes > 1.16
• Node specified in the TARGET_NODE environment variable should be cordoned before executing the chaos fault. This ensures that the fault resources are not scheduled on it (or subject to eviction). This is achieved by the following steps:
  • Get node names against the applications pods using command kubectl get pods -o wide.
  • Cordon the node using command kubectl cordon NODENAME.
• The target nodes should be in the ready state before and after injecting chaos.

Mandatory tunables

Tunable	Description	Notes
TARGET_NODE	Name of the node to be tainted.	For more information, go to target node.
NODE_LABEL	It contains node label, which will be used to filter the target nodes if TARGET_NODES ENV is not set	It is mutually exclusive with the TARGET_NODES environment variable. If both are provided, the fault uses the TARGET_NODES. For more information, go to node label.
TAINT_LABEL	Label and the effect to be tainted on the application node.	For more information, go to taint label.

Optional tunables

Tunable	Description	Notes
TOTAL_CHAOS_DURATION	Duration that you specify, through which chaos is injected into the target resource (in seconds).	Default: 60 s. For more information, go to duration of the chaos.
RAMP_TIME	Period to wait before and after injecting chaos (in seconds).	For example, 30 s. For more information, go to ramp time.

Taint label

Label and the effect to taint on the application node. Tune it by using the TAINT_LABEL environment variable.

The following YAML snippet illustrates the use of this environment variable:

# node tainted with provided key and effect
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  annotationCheck: "false"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: node-taint
    spec:
      components:
        env:
        # label and effect to be tainted on the targeted node
        - name: TAINT_LABEL
          value: 'key=value:effect'
        - name: TOTAL_CHAOS_DURATION
          VALUE: '60'