Skip to main content

Configure a Kubernetes build farm to use self-signed certificates

CI build infrastructure pods can interact with servers using self-signed certificates. This option is useful for organizations that prefer to use internal certificates instead of certificates generated by a public Certificate Authority (CA).

info
  • This topic assumes that you are familiar with how to implement SSL in Kubernetes. General information about implementing SSL is outside the scope of this topic.
  • With a Kubernetes cluster build infrastructure, all Build and Push steps use kaniko. Kaniko reads certificates from /kaniko/ssl/certs/additional-ca-cert-bundle.crt.
  • Harness uses a UBI image for the Git Clone step. UBI reads certificates from /etc/ssl/certs/ca-bundle.crt.
  • Different base images use different paths as their default certificate location. For example, Alpine images use the path /etc/ssl/certs/ca-certificates.crt. For other images, make sure you verify the default certificate path.

Enable self-signed certificates​

  1. Create a Kubernetes secret or config map with the required certificates in the same namespace used by the Harness Delegate. For example:

    apiVersion: v1
    kind: Secret
    metadata:
      name: addcerts
      namespace: harness-delegate-ng
    type: Opaque
    stringData:
      ca.bundle: |
        -----BEGIN CERTIFICATE-----
        XXXXXXXXXXXXXXXXXXXXXXXXXXX
        -----END CERTIFICATE-------
        -----BEGIN CERTIFICATE-----
        XXXXXXXXXXXXXXXXXXXXXXXXXXX
        -----END CERTIFICATE-------

  2. Mount the secret as a volume on the delegate pod and add a volumeMount to your certificate files at /opt/harness-delegate/ca-bundle.

    In the delegate pod, do the following:

    1. Provide a comma-separated list of paths in the build pod where you want the certs to be mounted.
    2. Mount your certificate files to /opt/harness-delegate/ca-bundle.

    To mount the secret, go to the delegate Deployment YAML and update volumeMounts and volumes. For an example, expand the section below.

    info

    Secrets are retrieved by the delegate. For Kubernetes build infrastructure, secrets are set as Kubernetes secrets and passed to the pod.

    YAML example: Mount the SSL certificate
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         labels:
           harness.io/name: abc-ca-kubernetes-delegate
         name: abc-ca-kubernetes-delegate
         namespace: abc-ca-kubernetes-delegate
       spec:
         replicas: 1
         minReadySeconds: 120
         selector:
           matchLabels:
             harness.io/name: abc-ca-kubernetes-delegate
         template:
           metadata:
             labels:
               harness.io/name: abc-ca-kubernetes-delegate
             annotations:
               # ...
           spec:
             terminationGracePeriodSeconds: 600
             restartPolicy: Always
             containers:
             - image: harness/delegate:yy.mm.verno
               imagePullPolicy: Always
               name: delegate
               securityContext:
                 # ...
               ports:
                 # ...
               resources:
                 # ...
               livenessProbe:
                 # ...
               startupProbe:
                 # ...
               envFrom:
                 # ...
               env:
                 # ...
               volumeMounts:
               ### ADD THE VOLUME MOUNT HERE
               - name: certvol
                 mountPath: /opt/harness-delegate/ca-bundle/ca.bundle
                 subPath: ca.bundle
             volumes:
             ### ADD THE VOLUME HERE
             - name: certvol
               secret:
                 secretName: addcerts
                 items:
                   - key: ca.bundle
                     path: ca.bundle

    Both CI build pods and the SCM client on the delegate support this method.

    warning

    • Make sure the destination path is not same as the default CA certificate path of the corresponding container image.

    • If you override the default certificate file, make sure the Kubernetes secret or config map (from step one) includes all certificates required by the pipelines that will use this build infrastructure.

  3. Restart the delegate. Once it is up and running, exec into the container and ensure that the volume exists at the mounted path and contains your certificates.

Enable self-signed certificates with a self-hosted image registry (advanced)​

important

This workflow is applicable only if you're using a self-hosted registry to store your container images. For all other workflows, go to Enable self-signed certificates.

  1. Create a Kubernetes secret or config map with the required certificates in the same namespace used by the Harness delegate. For example:

    apiVersion: v1
    kind: Secret
    metadata:
      name: addcerts
      namespace: harness-delegate-ng
    type: Opaque
    stringData:
      ca.bundle: |
        -----BEGIN CERTIFICATE-----
        XXXXXXXXXXXXXXXXXXXXXXXXXXX
        -----END CERTIFICATE-------
        -----BEGIN CERTIFICATE-----
        XXXXXXXXXXXXXXXXXXXXXXXXXXX
        -----END CERTIFICATE-------

  2. Mount the secret as a volume on the delegate pod.

    In the delegate pod, do the following:

    1. Add DESTINATION_CA_PATH to the environment.
    2. Provide a comma-separated list of paths in the build pod where you want the certs to be mounted.
    3. Mount your certificate files to /opt/harness-delegate/ca-bundle.

    To add DESTINATION_CA_PATH and mount the secret, go to the delegate Deployment YAML and update env, volumeMounts, and volumes. For an example, expand the section below.

    YAML example: Add DESTINATION_CA_PATH and mount the SSL certificate
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         labels:
           harness.io/name: abc-ca-kubernetes-delegate
         name: abc-ca-kubernetes-delegate
         namespace: abc-ca-kubernetes-delegate
       spec:
         replicas: 1
         minReadySeconds: 120
         selector:
           matchLabels:
             harness.io/name: abc-ca-kubernetes-delegate
         template:
           metadata:
             labels:
               harness.io/name: abc-ca-kubernetes-delegate
             annotations:
               # ...
           spec:
             terminationGracePeriodSeconds: 600
             restartPolicy: Always
             containers:
             - image: harness/delegate:yy.mm.verno
               imagePullPolicy: Always
               name: delegate
               securityContext:
                 # ...
               ports:
                 # ...
               resources:
                 # ...
               livenessProbe:
                 # ...
               startupProbe:
                 # ...
               envFrom:
                 # ...
               env:
               ### ADD DESTINATION_CA_PATH HERE
               - name: DESTINATION_CA_PATH
                 value: "/etc/ssl/certs/ca-bundle.crt,/kaniko/ssl/certs/additional-ca-cert-bundle.crt"
               volumeMounts:
               ### ADD THE VOLUME MOUNT HERE
               - name: certvol
                 mountPath: /opt/harness-delegate/ca-bundle/ca.bundle
                 subPath: ca.bundle
             volumes:
             ### ADD THE VOLUME HERE
             - name: certvol
               secret:
                 secretName: addcerts
                 items:
                   - key: ca.bundle
                     path: ca.bundle

    Both CI build pods and the SCM client on the delegate support this method.

    warning

    • Make sure the destination path is not same as the default CA certificate path of the corresponding container image.

    • If you override the default certificate file, make sure the Kubernetes secret or config map (from step one) includes all certificates required by the pipelines that will use this build infrastructure.

    Legacy: CI_MOUNT_VOLUMES

    Prior to the introduction of DESTINATION_CA_PATH, you used ADDITIONAL_CERTS_PATH and CI_MOUNT_VOLUMES to mount certs.

    The legacy method is still supported, but Harness recommends DESTINATION_CA_PATH. If you include both, DESTINATION_CA_PATH takes precedence. If Harness can't resolve DESTINATION_CA_PATH, it falls back to CI_MOUNT_VOLUMES and ADDITIONAL_CERTS_PATH.

    You must specify both ADDITIONAL_CERTS_PATH and CI_MOUNT_VOLUMES.

    For ADDITIONAL_CERTS_PATH, provide the path to the certificates in the delegate, such as /tmp/ca.bundle.

    For CI_MOUNT_VOLUMES, provide a comma-separated list of source:destination mappings where source is the certificate path on the delegate, and destination is the path where you want to expose the certificates on the build containers. For example:

    /tmp/ca.bundle:/etc/ssl/certs/ca-bundle.crt,/tmp/ca.bundle:/kaniko/ssl/certs/additional-ca-cert-bundle.crt

    The CI_MOUNT_VOLUMES list must include all certificates that your build containers need to interact with external services.

          spec:
            containers:
              - image: harness/delegate:yy.mm.verno
                # ...
                env:
                # ADD ADDITIONAL_CERTS_PATH AND CI_MOUNT_VOLUMES HERE
                  - name: ADDITIONAL_CERTS_PATH
                    value: /tmp/ca.bundle
                  - name: CI_MOUNT_VOLUMES
                    value: "/tmp/ca.bundle:/etc/ssl/certs/ca-bundle.crt,/tmp/ca.bundle:/kaniko/ssl/certs/additional-ca-cert-bundle.crt"
                volumeMounts:
                # ADD VOLUME MOUNT HERE
                  - name: certvol
                    mountPath: /tmp/ca.bundle
                    subPath:  ca.bundle
            volumes:
              # ADD VOLUME HERE
              - name: certvol
                secret:
                  secretName: addcerts
                  items:
                  - key: ca.bundle
                    path: ca.bundle

  3. If you're storing your certificates in a local registry and need to run Docker-in-Docker, specify the local certificate path on the delegate.

    For example, if your self-signed certs are stored at https://my-registry.local.org:799 and you log in with docker login my-registry.local.org:799, then you add the path to your DESTINATION_CA_PATH environment like this:

     spec:
       containers:
       - image: harness/delegate:yy.mm.verno
         env:
         - name: DESTINATION_CA_PATH
           value: "/etc/docker/certs.d/my-registry.local.org:799/ca.crt,/etc/ssl/certs/ca-bundle.crt,/kaniko/ssl/certs/additional-ca-cert-bundle.crt"
         volumeMounts:
           - name: certvol
             mountPath: /opt/harness-delegate/ca-bundle/ca.bundle
             subPath:  ca.bundle
       volumes:
       - name: certvol
         secret:
             secretName: addcerts
              items:
              - key: ca.bundle
                path: ca.bundle

  4. Restart the delegate. Once it's up and running, exec into the container and ensure that the volume exists at the mounted path and contains your certificates.

Troubleshoot Kubernetes cluster build infrastructures​

Go to the CI Knowledge Base for questions and issues related to Kubernetes cluster build infrastructures, including use of self-signed certificates, such as: