Issue

I would like to create dedicated Agents that run in a Kubernetes Cluster.

Environment

CloudBees Jenkins Enterprise - Managed Master (CJE-MM)
CloudBees Jenkins Platform - Client Master (CJP-CM)
CloudBees Jenkins Team (CJT)
Jenkins LTS
Kubernetes

Resolution

While there is a solution - i.e. the Kubernetes plugin - to provision on-demand Kubernetes agents, there might be scenarios where a dedicated agent is suitable. Following are some examples on how to deploy dedicated Agents in Kubernetes for different launcher:

Dedicated JNLP Agent

The deployment of such agent requires two steps:

Create a Permanent Agent in Manage Jenkins > Manage Nodes and select the JNLP launcher:

Deploy the agent in kubernetes. Here is an example of a ReplicaSet:

---
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata: 
name: "jenkins-jnlp-agent"
labels: 
  name: "jenkins-jnlp-agent"
spec: 
replicas: 1
template: 
  metadata: 
    name: "jenkins-jnlp-agent"
    labels: 
      name: "jenkins-jnlp-agent"
  spec: 
    containers:
    - name: jnlp-slave
      image: jenkinsci/jnlp-slave
      env:
        - name: JENKINS_URL
          value: "http://cje.example.com"
        - name: JENKINS_AGENT_NAME
          value: "dedicatedJNLPAgent"
        - name: JENKINS_SECRET
          value: c02454dd29892ba194dc5f98fb68f83463a8518da7be06254b460058114ac21d
      livenessProbe:
        exec:
          command:
          - uname
          - -a
        initialDelaySeconds: 60
        timeoutSeconds: 1

The values for the variable JENKINS_URL, JENKINS_AGENT_NAME and JENKINS_SECRET must be set accordingly. Note that If Jenkins is running in the kubernetes cluster, the JENKINS_URL can be set to the name / kubernetes DNS name of the service exposing the Jenkins master.

For more information about the configuration, have a look at the base image jenkinsci/jnlp-slave.

Dedicated SSH Agent

This scenario can be used for connecting SSH Agents with either the CloudBees SSH Build Agents plugin or the SSH Slaves plugin.

The deployment of such agent requires two steps:

Deploy the agent in kubernetes. Here is an example of a ReplicaSet:

---
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata: 
name: "jenkins-ssh-agent"
labels: 
  name: "jenkins-ssh-agent"
spec: 
replicas: 1
template: 
  metadata: 
    name: "jenkins-ssh-agent"
    labels: 
      name: "jenkins-ssh-agent"
  spec: 
    containers:
    - name: ssh-slave
      image: jenkinsci/ssh-slave
      env:
        - name: JENKINS_SLAVE_SSH_PUBKEY
          value: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3lXl4ODAXtdb6NBVXx8VCOtTT0lKKcNyokxAsE47eb5HjwV/UF9izc+DgDKPt6bC4R6PEeSuYiUzoYsi7MzA2o3NURDH0Fb8XKWLwCykJznVHoqWXITloKxabtFOMR8FyNiV0qIn49NeQqGkLFld2WL4/dJj76DLV6AEqVsB52GVyJjt+TJnMzz9x0dXzRUDM5DIaLDBW3NGSMMpo1DxhJL4HrXGbUjF0TLaqBU8CqxOWmH88EU8yedbBBTSDNTY+8Qk9WWpI2E/ssAWjnLLsrJFxduTg5/iTsF/FWcOHrFmjcsf3ojpy2/PYYA5+jfgHKDn82zcLYdhyqKkxBzIq84/vwMASzgSUc3ONtxirPmIVgCHQixeGHzlAC4ea/6h66QoMyssozyAz3/X4byoG6gqEwDxhJyb36ntL/81wLFt5dZWqg/+OvBZ2XJMetAAjulh3KCPx5gc2G9XWb0JxzwoAiUGsOLONzGPNU/Lu/eMVKJopNY2ZibmwTvR63x7bMiRhe/cO2/5D/LKDhEpy2EAdS8j8wPyTeFZPoxhQPGtiwb4eAB2BudAjAnPTZELk7Hv6pHXrMMCebZhemHToX4q5hOUUZeA5/j+9ynMptCrouUW9v/QmTvmJu0of0vQ2Ywrh7tzDODbov1M55CklLgqkERSiYYIN4ZIbRpsPbw== cje.example.com"
      livenessProbe:
        tcpSocket:
          port: 22
        initialDelaySeconds: 60
        timeoutSeconds: 1

---
apiVersion: v1
kind: Service
metadata:
name: jenkins-ssh-agent
namespace: cje-on-support-cje2-aburdajewicz
spec:
selector:
  name: jenkins-ssh-agent
ports:
- name: ssh
  port: 22
  protocol: TCP

The value of the JENKINS_SLAVE_SSH_PUBKEY is the SSH public key of the Jenkins server.

Create a Permanent Agent in Manage Jenkins > Manage Nodes, select an SSH launcher and specify the Jenkins SSH Credentials holding the private key:

Note: If Jenkins is running in the kubernetes cluster, the Host can be set to the name / kubernetes DNS name of the service (here jenkins-ssh-agent). Otherwise, the jenkins-ssh-agent would need to be exposed externally from the Kubernetes cluster. For that matter, the SSH launcher is more suitable when Jenkins is running aside in the same Kubernetes cluster.

For more information about the configuration, have a look at the base image jenkinsci/ssh-slave.

Dedicated Swarm Agent

This scenario can be used for connecting Swarm Agents with the Self-Organizing Swarm Modules plugin. The advantage of that solution is that you can scale with more than 1 replicas.

The deployment of such agent requires only one step, to deploy the agent in Kubernetes. Here is an example of a ReplicaSet:

---
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata: 
  name: "jenkins-swarm-agent"
  labels: 
    name: "jenkins-swarm-agent"
spec: 
  replicas: 3
  template: 
    metadata: 
      name: "jenkins-swarm-agent"
      labels: 
        name: "jenkins-swarm-agent"
    spec: 
      containers:
      - name: swarm-slave
        image: csanchez/jenkins-swarm-slave
        env:
          - name: MASTER
            value: "http://cje.example.com"
          - name: USERNAME
            value: admin
          - name: PASSWORD
            value: supersecretpassword
          - name: NAME
            value: dedicatedSwarmAgent
          - name: EXECUTORS
            value: "1"
          - name: FSROOT
            value: /home/jenkins-slave
        livenessProbe:
          exec:
            command:
            - sh
            - -c
            - "netstat -tan | grep ESTABLISHED"
          initialDelaySeconds: 60
          timeoutSeconds: 1

The values for the different variable MASTER, NAME, EXECUTORS, USERNAME, PASSWORD, FSROOT must be set accordingly. Note that If Jenkins is running in the kubernetes cluster, the MASTER variable can be set to the name / kubernetes DNS name of the service exposing the Jenkins master.

There are many more environment variables that are possible. For more information about the configuration, have a look at the base image csanchez/jenkins-swarm-slave and also the source code of that image carlossg/jenkins-swarm-slave-docker.

Note: The above example is meant to provide the basics. It is best to use a Kubernetes secret for the value of USERNAME and PASSWORD.

References

Following are useful resources on that topic:

Create dedicated agents running Kubernetes