Issue
- We have a CJE AWS Cluster not setup with volume encryption and we want to enable it
- We have a CJE AWS Cluster already setup with volume encryption and we want to change the encryption key
Environment
Resolution
New Installation
Volume encryption can be enabled when initializing the cluster by setting the property encrypted to yes. A KMS key can be provided by setting the property kms_key_id:
## Enable storage encryption (yes/no)
#
# This setting applies to JENKINS_HOME volumes *only*.
# To use encrypted volumes for root volumes, please create an encrypted copy of CloudBees-provided AMI and pass it as `default_ami` (above) during this operation.
#
# encrypted =
## KMS Key ID to use for encryption (optional)
#
# If encryption is enabled, a default CMK is created automatically.
# A custom CMK can be specified here.
#
# example: arn:aws:kms:us-east-1:123456789012:key/aabbccdd-1234-5678-abcd-987cba012345
#
# kms_key_id =
Existing Installation
To enable volume encryption to an already existing cluster, CloudBees Jenkins enterprise provides the operation cje prepare castle-update since version 1.11.2. However this applies to newly created masters only and existing Jenkins volumes (masters and CJOC) must be encrypted / re-encrypted manually.
1. Setup the Cluster Encryption Configuration
To activate or change the volume encryption in CJE, follow these steps:
1) Use the operation cje prepare castle-update
2) In the file castle-update.config:
- set the value
encrypted = yes - optionally set the value of an existing
kms_key_id
3) cje verify
4) cje apply
When applying the changes, castle is restarted in each master worker with the new configuration. This implies that:
- the volumes of newly created Masters will be encrypted according to the configuration provided
- the volumes of existing Masters remains unchanged:
- if they were not yet encrypted, they remain non encrypted
- if they were already encrypted, they remains encrypted with the key specified in the previous configuration.
More details about the behavior of AWS Snapshots and Volumes can be found at How Amazon Elastic Block Store (Amazon EBS) Uses AWS KMS.
2. Encryption of Existing Volume
When enabling / changing encryption for an already initialized cluster, existing Jenkins volumes must be encrypted manually.
To encrypt or re-encrypt a Master/CJOC with a new key, follow these steps:
1) Stop the Master (via CJOC UI) / CJOC (via dna stop cjoc)
2) Go to EC2 console, wait for the corresponding volume (tag: cloudbees:pse:account: <nameofmaster>) to be in state ‘available’
3) Create a snapshot:
4) Wait until the snapshot is completed, then copy it, and tick the encrypt checkbox, provide the KMS key here.
5) Tag the new snapshot with the tags Castle expects (copy the tags based on the existing snapshot):
account: <nameofmaster>cloudbees:pse:account: <nameofmaster>cloudbees:pse:cluster: <name-of-cluster>cluster: <name-of-cluster>
6) Delete the account volume (this step is required, if castle found an existing volume it uses it but if not it creates a volume from the latest snapshot):
7) Start the Master (via CJOC UI) / CJOC (via dna start cjoc)
When the master / cjoc is restarted, then all volumes and snapshots for this master will be encrypted.
Permissions to use the KMS Key
The workers must have the required permission to use the KMS key provided.
When using IAM roles, ensure that the worker role (the instance_profile in .dna/project.config) is added to the list of roles that can use the KMS key. In AWS, go to IAM > Encryption keys > $MY_KMS_KEY > Key Users:
If this is not the case, castle would not be able to provision new volumes and would likely fail with an error like the following:
com.cloudbees.dac.castle.VolumeException: java.util.concurrent.ExecutionException: com.amazonaws.services.ec2.model.AmazonEC2Exception: The volume '<volumeId>' does not exist. (Service: AmazonEC2; Status Code: 400; Error Code: InvalidVolume.NotFound; Request ID: <requestId>)
0 Comments