Issue

• I am using the YAML field in the master’s configuration
• I am using CloudBees Core 2.138.1.2 or 2.138.2.2
• When starting / restarting a master, the provisioning fails and the provisioning log in CJOC shows:

Failure executing: POST at: https://10.55.240.1/apis/apps/v1/namespaces/cje-support-general/statefulsets. Message: StatefulSet.apps "master-test" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"com.cloudbees.cje.tenant":"master-test", "type":"master", "tenant":"master-test", "com.cloudbees.pse.type":"master", "com.cloudbees.cje.type":"master", "com.cloudbees.pse.tenant":"master-test"}: `selector` does not match template `labels`]. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.selector, message=Required value, reason=FieldValueRequired, additionalProperties={}), StatusCause(field=spec.template.metadata.labels, message=Invalid value: map[string]string{"tenant":"master-test", "com.cloudbees.pse.type":"master", "com.cloudbees.cje.type":"master", "com.cloudbees.pse.tenant":"master-test", "com.cloudbees.cje.tenant":"master-test", "type":"master"}: `selector` does not match template `labels`, reason=FieldValueInvalid, additionalProperties={})], group=apps, kind=StatefulSet, name=master-test, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=StatefulSet.apps "master-test" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"com.cloudbees.cje.tenant":"master-test", "type":"master", "tenant":"master-test", "com.cloudbees.pse.type":"master", "com.cloudbees.cje.type":"master", "com.cloudbees.pse.tenant":"master-test"}: `selector` does not match template `labels`], metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).

Environment

• CloudBees CI (CloudBees Core) on Modern Cloud Platforms 2.138.1.2 and 2.138.2.2
• Master Provisioning Plugin 2.2.2 and 2.2.3

Related Issues

• CPLT2-5255: statefulset/spec/selector/matchLabels are not generated when the yaml field is used in the configuration
• CPLT2-5191: Master Provisioning fails with 2.138.1.1 & Master Provisioning Plugin 2.2.2

Explanation

In version 2.138.1.2 and 2.138.2.2 of CloudBees Core, when using the YAML field in the Master’s Advanced configuration the statefulset generated is wrong as it is lacking the spec.selector. Resulting in the following error in the master’s page:

This is caused by a bug in the Master Provisioning Plugin.

Resolution

Solution

The issue has been fixed in Master Provisioning version 2.2.4 available since CloudBees CI 2.138.3.1

The solution is to upgrade to 2.138.3.1 or later.

Workaround

The workaround if running an affected version is to define the spec.selector in the master configuration:

kind: "StatefulSet"
spec:
  selector:
    matchLabels:
      type: "master"
      tenant: "<masterName>"
      com.cloudbees.pse.type: "master"
      com.cloudbees.cje.type: "master"
      com.cloudbees.pse.tenant: "<masterName>"
      com.cloudbees.cje.tenant: "<masterName>"

As an example, the following YAML in a master configuration adds a volume to the master’s container:

---
kind: "StatefulSet"
spec:
  template:
    spec:
      containers:
        - name: "jenkins"
          volumeMounts:
          - name: volume-ca-bundle
            mountPath: /var/certs
      volumes:
      - name: volume-ca-bundle
        configMap:
          name: ca-bundle

But when provisioning it fails due to the above mentioned issue. The workaround is to add the spec.selector (the name of the master in that case is master-test):

---
kind: "StatefulSet"
spec:
  selector:
    matchLabels:
      type: "master"
      tenant: "master-test"
      com.cloudbees.pse.type: "master"
      com.cloudbees.cje.type: "master"
      com.cloudbees.pse.tenant: "master-test"
      com.cloudbees.cje.tenant: "master-test"
  template:
    spec:
      containers:
        - name: "jenkins"
          volumeMounts:
          - name: volume-ca-bundle
            mountPath: /var/certs
      volumes:
      - name: volume-ca-bundle
        configMap:
          name: ca-bundle

Resources

• Statefulset spec