Issue
When creating a new Managed Master, some manual configuration is necessary to be able to connect an external agent.
Is there a way to automate this Managed Master creation when using an Helm setup?
Environment
- CloudBees CI (CloudBees Core)
- CloudBees CI (CloudBees Core) on modern cloud platforms - Managed Master
- Helm based installation (Helm version 2 or 3)
Resolution
Prerequisite:
- Ensure you replace
cloudbees-coreby the namespace used by your ingress-controller
The following bash script will:
- Take a backup of your current configuration and save it under
backup.yaml(note, you can also rollback with helm). - Create the new rules and save them under
rules.yaml. - Apply the newly created rules to the installation.
Important notes
This script will update several Kubernetes objects in your cluster.
From a Kubernetes point of view, thanks to the mechanism of Rolling Updates, there should not be any service interruptions.
However, the script also updates nginx configuration, which means that most likely the nginx service will be restarted.
Therefore all active connections to the cluster might get dropped (eg an external agent connected to the master).
We therefore recommend this script to run only at a quiet time.
The script is provided AS IS and you should review what it is doing before applying it.
Type of rules generated
The script is generating rules for a backing Load Balancer using the proxy protocol (like an AWS ELB).
If your Load Balancer is not using the proxy protocol, you should modify the rule creation to not append :PROXY
This means that:
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}")
becomes:
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{'\"\n'}{end}")
If you are using Helm 2
Prerequisites
- Helm 2
- kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
- jq
Script Parameters
You need to adjust the RELEASE_NAME to your current release name.
Run the script once as a dry run (no changes actually applied).
Make sure the output is the expected one.
Then update DRY_RUN to false.
Script
#!/bin/bash
set -euo pipefail
# Prerequisites of this script:
# * kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
# * helm 2
# * jq
# Script will:
# 1. Take a backup of your current configuration and save it under backup.yaml
# 2. Create the new rules and save them under rules.yaml
# 3. Apply the newly created rules to the installation
# Parameters
# * RELEASE_NAME: the name of the helm release for the CloudBees Core installation, defaults to cloudbees-core.
# You can find this value by running helm ls
# * DRY_RUN: set to something else than true to run the actual update, defaults to true.
# BEGIN PARAMETERS
RELEASE_NAME='cloudbees-core'
DRY_RUN=false
# END PARAMETERS
printf 'Backing up the current configuration for configuration %s under backup.yaml\n' "${RELEASE_NAME}"
helm get values "${RELEASE_NAME}" >backup.yaml
printf 'Done\n'
printf 'Creating the rules\n'
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}")
cat >rules.yaml <<EOF
nginx-ingress:
tcp:
${ROUTES}
EOF
printf 'Done\n'
printf 'Will write the following rules to your ingress controller:\n%s\n' "$(cat rules.yaml)"
CHART_NAME=$(helm list "${RELEASE_NAME}" --output json | jq -j '.Releases[0].Chart')
printf "Found the following chart to update %s\n" "${CHART_NAME}"
if [ "${DRY_RUN}" = true ] ; then
helm upgrade --dry-run --debug --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --description 'Ingress routes generation' --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
else
helm upgrade --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --description 'Ingress routes generation' --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
fi
If you are using Helm 3
Prerequisites
- Helm 3
- kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
- jq
Script Parameters
You need to adjust the RELEASE_NAME to your current release name.
Run the script once as a dry run (no changes actually applied).
Make sure the output is the expected one.
Then update DRY_RUN to false.
Script
#!/bin/bash
set -euo pipefail
# Prerequisites of this script:
# * kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
# * helm 3. It is assumed the CloudBees repository is named cloudbees (ie you added it with helm repo add cloudbees https://charts.cloudbees.com/public/cloudbees
# * jq
# Script will:
# 1. Take a backup of your current configuration and save it under backup.yaml
# 2. Create the new rules and save them under routes.yaml
# 3. Apply the newly created rules to the installation
# Parameters
# * RELEASE_NAME: the name of the helm release for the CloudBees Core installation, defaults to cloudbees-core.
# You can find this value by running helm ls
# * DRY_RUN: set to something else than true to run the actual update, defaults to true.
# BEGIN PARAMETERS
RELEASE_NAME='cloudbees-core'
DRY_RUN=true
# END PARAMETERS
printf 'Backing up the current configuration for configuration %s under backup.yaml\n' "${RELEASE_NAME}"
helm get values "${RELEASE_NAME}" >backup.yaml
printf 'Done\n'
printf 'Creating the rules\n'
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{' \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}")
cat >rules.yaml <<EOF
nginx-ingress:
tcp:
${ROUTES}
EOF
printf 'Done\n'
printf 'Will write the following rules to your ingress controller:\n%s\n' "$(cat rules.yaml)"
CHART_NAME=$(helm list --filter "${RELEASE_NAME}" --output json | jq -j '.[0].chart')
printf "Found the following chart to update %s\n" "${CHART_NAME}"
if [ "${DRY_RUN}" = true ] ; then
helm upgrade --dry-run --debug --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
else
helm upgrade --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
fi
Tested product/plugin versions
- CloudBees CI (CloudBees Core) 2.204.1.3
- Helm 2.16.1 and Helm 3.0.1
0 Comments