Create dedicated agents running Kubernetes

Issue

  • I would like to create dedicated Agents that run in a Kubernetes Cluster.

Environment

Resolution

While there is a solution - i.e. the Kubernetes plugin - to provision on-demand Kubernetes agents, there might be scenarios where a dedicated agent is suitable. Following are some examples on how to deploy dedicated Agents in Kubernetes for different launcher:

Dedicated JNLP Agent

The deployment of such agent requires two steps:

  • Create a Permanent Agent in Manage Jenkins > Manage Nodes and select the JNLP launcher:

  • Deploy the agent in kubernetes. Here is an example of a ReplicaSet:

    ---
    apiVersion: extensions/v1beta1
    kind: ReplicaSet
    metadata: 
      name: "jenkins-jnlp-agent"
      labels: 
        name: "jenkins-jnlp-agent"
    spec: 
      replicas: 1
      template: 
        metadata: 
          name: "jenkins-jnlp-agent"
          labels: 
            name: "jenkins-jnlp-agent"
        spec: 
          containers:
          - name: jnlp-slave
            image: jenkinsci/jnlp-slave
            env:
              - name: JENKINS_URL
                value: "http://cje.example.com"
              - name: JENKINS_AGENT_NAME
                value: "dedicatedJNLPAgent"
              - name: JENKINS_SECRET
                value: c02454dd29892ba194dc5f98fb68f83463a8518da7be06254b460058114ac21d
            livenessProbe:
              exec:
                command:
                - uname
                - -a
              initialDelaySeconds: 60
              timeoutSeconds: 1
    

    The values for the variable JENKINS_URL, JENKINS_AGENT_NAME and JENKINS_SECRET must be set accordingly. Note that If Jenkins is running in the kubernetes cluster, the JENKINS_URL can be set to the name / kubernetes DNS name of the service exposing the Jenkins master.

For more information about the configuration, have a look at the base image jenkinsci/jnlp-slave.

Dedicated SSH Agent

This scenario can be used for connecting SSH Agents with either the CloudBees SSH Build Agents plugin or the SSH Slaves plugin.

The deployment of such agent requires two steps:

  • Deploy the agent in kubernetes. Here is an example of a ReplicaSet:

    ---
    apiVersion: extensions/v1beta1
    kind: ReplicaSet
    metadata: 
      name: "jenkins-ssh-agent"
      labels: 
        name: "jenkins-ssh-agent"
    spec: 
      replicas: 1
      template: 
        metadata: 
          name: "jenkins-ssh-agent"
          labels: 
            name: "jenkins-ssh-agent"
        spec: 
          containers:
          - name: ssh-slave
            image: jenkinsci/ssh-slave
            env:
              - name: JENKINS_SLAVE_SSH_PUBKEY
                value: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3lXl4ODAXtdb6NBVXx8VCOtTT0lKKcNyokxAsE47eb5HjwV/UF9izc+DgDKPt6bC4R6PEeSuYiUzoYsi7MzA2o3NURDH0Fb8XKWLwCykJznVHoqWXITloKxabtFOMR8FyNiV0qIn49NeQqGkLFld2WL4/dJj76DLV6AEqVsB52GVyJjt+TJnMzz9x0dXzRUDM5DIaLDBW3NGSMMpo1DxhJL4HrXGbUjF0TLaqBU8CqxOWmH88EU8yedbBBTSDNTY+8Qk9WWpI2E/ssAWjnLLsrJFxduTg5/iTsF/FWcOHrFmjcsf3ojpy2/PYYA5+jfgHKDn82zcLYdhyqKkxBzIq84/vwMASzgSUc3ONtxirPmIVgCHQixeGHzlAC4ea/6h66QoMyssozyAz3/X4byoG6gqEwDxhJyb36ntL/81wLFt5dZWqg/+OvBZ2XJMetAAjulh3KCPx5gc2G9XWb0JxzwoAiUGsOLONzGPNU/Lu/eMVKJopNY2ZibmwTvR63x7bMiRhe/cO2/5D/LKDhEpy2EAdS8j8wPyTeFZPoxhQPGtiwb4eAB2BudAjAnPTZELk7Hv6pHXrMMCebZhemHToX4q5hOUUZeA5/j+9ynMptCrouUW9v/QmTvmJu0of0vQ2Ywrh7tzDODbov1M55CklLgqkERSiYYIN4ZIbRpsPbw== cje.example.com"
            livenessProbe:
              tcpSocket:
                port: 22
              initialDelaySeconds: 60
              timeoutSeconds: 1
    
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: jenkins-ssh-agent
      namespace: cje-on-support-cje2-aburdajewicz
    spec:
      selector:
        name: jenkins-ssh-agent
      ports:
      - name: ssh
        port: 22
        protocol: TCP
    

    The value of the JENKINS_SLAVE_SSH_PUBKEY is the SSH public key of the Jenkins server.

  • Create a Permanent Agent in Manage Jenkins > Manage Nodes, select an SSH launcher and specify the Jenkins SSH Credentials holding the private key:

    Note: If Jenkins is running in the kubernetes cluster, the Host can be set to the name / kubernetes DNS name of the service (here jenkins-ssh-agent). Otherwise, the jenkins-ssh-agent would need to be exposed externally from the Kubernetes cluster. For that matter, the SSH launcher is more suitable when Jenkins is running aside in the same Kubernetes cluster.

For more information about the configuration, have a look at the base image jenkinsci/ssh-slave.

Dedicated Swarm Agent

This scenario can be used for connecting Swarm Agents with the Self-Organizing Swarm Modules plugin. The advantage of that solution is that you can scale with more than 1 replicas.

The deployment of such agent requires only one step, to deploy the agent in Kubernetes. Here is an example of a ReplicaSet:

---
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata: 
  name: "jenkins-swarm-agent"
  labels: 
    name: "jenkins-swarm-agent"
spec: 
  replicas: 3
  template: 
    metadata: 
      name: "jenkins-swarm-agent"
      labels: 
        name: "jenkins-swarm-agent"
    spec: 
      containers:
      - name: swarm-slave
        image: csanchez/jenkins-swarm-slave
        env:
          - name: MASTER
            value: "http://cje.example.com"
          - name: USERNAME
            value: admin
          - name: PASSWORD
            value: supersecretpassword
          - name: NAME
            value: dedicatedSwarmAgent
          - name: EXECUTORS
            value: "1"
          - name: FSROOT
            value: /home/jenkins-slave
        livenessProbe:
          exec:
            command:
            - sh
            - -c
            - "netstat -tan | grep ESTABLISHED"
          initialDelaySeconds: 60
          timeoutSeconds: 1

The values for the different variable MASTER, NAME, EXECUTORS, USERNAME, PASSWORD, FSROOT must be set accordingly. Note that If Jenkins is running in the kubernetes cluster, the MASTER variable can be set to the name / kubernetes DNS name of the service exposing the Jenkins master.

There are many more environment variables that are possible. For more information about the configuration, have a look at the base image csanchez/jenkins-swarm-slave and also the source code of that image carlossg/jenkins-swarm-slave-docker.

Note: The above example is meant to provide the basics. It is best to use a Kubernetes secret for the value of USERNAME and PASSWORD.

References

Following are useful resources on that topic:

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.