Generate Ingress Routes For External JNLP Agents

Issue

When creating a new Managed controller, some manual configuration is necessary to be able to connect an external agent.
Is there a way to automate this Managed controller creation when using an Helm setup?

Environment

Resolution

Prerequisite:

  1. Ensure you replace cloudbees-core by the namespace used by your ingress-controller

The following bash script will:

  1. Take a backup of your current configuration and save it under backup.yaml (note, you can also rollback with helm).
  2. Create the new rules and save them under rules.yaml.
  3. Apply the newly created rules to the installation.

Important notes

This script will update several Kubernetes objects in your cluster.
From a Kubernetes point of view, thanks to the mechanism of Rolling Updates, there should not be any service interruptions.
However, the script also updates nginx configuration, which means that most likely the nginx service will be restarted.
Therefore all active connections to the cluster might get dropped (eg an external agent connected to the controller).
We therefore recommend this script to run only at a quiet time.

The script is provided AS IS and you should review what it is doing before applying it.

Type of rules generated

The script is generating rules for a backing Load Balancer using the proxy protocol (like an AWS ELB).
If your Load Balancer is not using the proxy protocol, you should modify the rule creation to not append :PROXY

This means that:

ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{'    \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}")

becomes:

ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{'    \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{'\"\n'}{end}")

If you are using Helm 2

Prerequisites

  • Helm 2
  • kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
  • jq

Script Parameters

You need to adjust the RELEASE_NAME to your current release name.
Run the script once as a dry run (no changes actually applied).
Make sure the output is the expected one.
Then update DRY_RUN to false.

Script

#!/bin/bash
set -euo pipefail

# Prerequisites of this script:
# * kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
# * helm 2
# * jq

# Script will:
# 1. Take a backup of your current configuration and save it under backup.yaml
# 2. Create the new rules and save them under rules.yaml
# 3. Apply the newly created rules to the installation

# Parameters
# * RELEASE_NAME: the name of the helm release for the CloudBees Core installation, defaults to cloudbees-core.
# You can find this value by running helm ls
# * DRY_RUN: set to something else than true to run the actual update, defaults to true.

# BEGIN PARAMETERS
RELEASE_NAME='cloudbees-core'
DRY_RUN=false
# END PARAMETERS


printf 'Backing up the current configuration for configuration %s under backup.yaml\n' "${RELEASE_NAME}"
helm get values "${RELEASE_NAME}" >backup.yaml
printf 'Done\n'

printf 'Creating the rules\n'
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=master --all-namespaces -o jsonpath="{range .items[*]}{'    \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}")

cat >rules.yaml <<EOF
nginx-ingress:
  tcp:
${ROUTES}
EOF
printf 'Done\n'

printf 'Will write the following rules to your ingress controller:\n%s\n' "$(cat rules.yaml)"

CHART_NAME=$(helm list "${RELEASE_NAME}" --output json | jq -j '.Releases[0].Chart')

printf "Found the following chart to update %s\n" "${CHART_NAME}"

if [ "${DRY_RUN}" = true ] ; then
    helm upgrade --dry-run --debug --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}")  --description 'Ingress routes generation' --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
else
  helm upgrade --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}")  --description 'Ingress routes generation' --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
fi

If you are using Helm 3

Prerequisites

  • Helm 3
  • kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
  • jq

Script Parameters

You need to adjust the RELEASE_NAME to your current release name.
Run the script once as a dry run (no changes actually applied).
Make sure the output is the expected one.
Then update DRY_RUN to false.

Script

#!/bin/bash
set -euo pipefail

# Prerequisites of this script:
# * kubectl configured to connect to the Kubernetes Cluster containing your CloudBees Core installation
# * helm 3. It is assumed the CloudBees repository is named cloudbees (ie you added it with helm repo add cloudbees https://charts.cloudbees.com/public/cloudbees
# * jq

# Script will:
# 1. Take a backup of your current configuration and save it under backup.yaml
# 2. Create the new rules and save them under routes.yaml
# 3. Apply the newly created rules to the installation

# Parameters
# * RELEASE_NAME: the name of the helm release for the CloudBees Core installation, defaults to cloudbees-core.
# You can find this value by running helm ls
# * DRY_RUN: set to something else than true to run the actual update, defaults to true.

# BEGIN PARAMETERS
RELEASE_NAME='cloudbees-core'
DRY_RUN=true
# END PARAMETERS


printf 'Backing up the current configuration for configuration %s under backup.yaml\n' "${RELEASE_NAME}"
helm get values "${RELEASE_NAME}" >backup.yaml
printf 'Done\n'

printf 'Creating the rules\n'
ROUTES=$(kubectl get svc -l com.cloudbees.cje.type=controller --all-namespaces -o jsonpath="{range .items[*]}{'    \"'}{.spec.ports[?(@.name=='agent')].port}{'\": \"'}{.metadata.namespace}{'/'}{.metadata.name}{':'}{.spec.ports[?(@.name=='agent')].port}{':PROXY\"\n'}{end}")

cat >rules.yaml <<EOF
nginx-ingress:
  tcp:
${ROUTES}
EOF
printf 'Done\n'

printf 'Will write the following rules to your ingress controller:\n%s\n' "$(cat rules.yaml)"

CHART_NAME=$(helm list --filter "${RELEASE_NAME}" --output json | jq -j '.[0].chart')

printf "Found the following chart to update %s\n" "${CHART_NAME}"

if [ "${DRY_RUN}" = true ] ; then
    helm upgrade --dry-run --debug --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
else
  helm upgrade --version $(sed "s/${RELEASE_NAME}-//" <<< "${CHART_NAME}") --values backup.yaml --values rules.yaml "${RELEASE_NAME}" cloudbees/cloudbees-core
fi

Tested product/plugin versions

Was this article helpful?
0 out of 0 found this helpful

Have more questions?

0 Comments

Please sign in to leave a comment.