Issue

• I am using the YAML field in the controller’s configuration
• I am using CloudBees Core 2.138.1.2 or 2.138.2.2
• When starting / restarting a controller, the provisioning fails and the provisioning log in CJOC shows:

Failure executing: POST at: https://10.55.240.1/apis/apps/v1/namespaces/cje-support-general/statefulsets. Message: StatefulSet.apps "controller-test" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"com.cloudbees.cje.tenant":"controller-test", "type":"master", "tenant":"controller-test", "com.cloudbees.pse.type":"master", "com.cloudbees.cje.type":"master", "com.cloudbees.pse.tenant":"controller-test"}: `selector` does not match template `labels`]. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=spec.selector, message=Required value, reason=FieldValueRequired, additionalProperties={}), StatusCause(field=spec.template.metadata.labels, message=Invalid value: map[string]string{"tenant":"controller-test", "com.cloudbees.pse.type":"master", "com.cloudbees.cje.type":"master", "com.cloudbees.pse.tenant":"controller-test", "com.cloudbees.cje.tenant":"controller-test", "type":"master"}: `selector` does not match template `labels`, reason=FieldValueInvalid, additionalProperties={})], group=apps, kind=StatefulSet, name=controller-test, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=StatefulSet.apps "controller-test" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"com.cloudbees.cje.tenant":"controller-test", "type":"master", "tenant":"controller-test", "com.cloudbees.pse.type":"master", "com.cloudbees.cje.type":"master", "com.cloudbees.pse.tenant":"controller-test"}: `selector` does not match template `labels`], metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).

Environment

• CloudBees CI (CloudBees Core) on Modern Cloud Platforms 2.138.1.2 and 2.138.2.2
• Master Provisioning Plugin 2.2.2 and 2.2.3

Related Issues

• CPLT2-5255: statefulset/spec/selector/matchLabels are not generated when the yaml field is used in the configuration
• CPLT2-5191: Master Provisioning fails with 2.138.1.1 & Master Provisioning Plugin 2.2.2

Explanation

In version 2.138.1.2 and 2.138.2.2 of CloudBees Core, when using the YAML field in the controller’s Advanced configuration the statefulset generated is wrong as it is lacking the spec.selector. Resulting in the following error in the controller’s page:

This is caused by a bug in the Master Provisioning Plugin.

Resolution

Solution

The issue has been fixed in Master Provisioning version 2.2.4 available since CloudBees CI 2.138.3.1

The solution is to upgrade to 2.138.3.1 or later.

Workaround

The workaround if running an affected version is to define the spec.selector in the controller configuration:

kind: "StatefulSet"
spec:
  selector:
    matchLabels:
      type: "master"
      tenant: "<masterName>"
      com.cloudbees.pse.type: "master"
      com.cloudbees.cje.type: "master"
      com.cloudbees.pse.tenant: "<masterName>"
      com.cloudbees.cje.tenant: "<masterName>"

As an example, the following YAML in a controller configuration adds a volume to the controller’s container:

---
kind: "StatefulSet"
spec:
  template:
    spec:
      containers:
        - name: "jenkins"
          volumeMounts:
          - name: volume-ca-bundle
            mountPath: /var/certs
      volumes:
      - name: volume-ca-bundle
        configMap:
          name: ca-bundle

But when provisioning it fails due to the above mentioned issue. The workaround is to add the spec.selector (the name of the controllerin that case is controller-test):

---
kind: "StatefulSet"
spec:
  selector:
    matchLabels:
      type: "master"
      tenant: "controller-test"
      com.cloudbees.pse.type: "master"
      com.cloudbees.cje.type: "master"
      com.cloudbees.pse.tenant: "controller-test"
      com.cloudbees.cje.tenant: "controller-test"
  template:
    spec:
      containers:
        - name: "jenkins"
          volumeMounts:
          - name: volume-ca-bundle
            mountPath: /var/certs
      volumes:
      - name: volume-ca-bundle
        configMap:
          name: ca-bundle

Resources

• Statefulset spec