Migrating Jenkins instances

Issue

  • I’d like to migrate all of my Jenkins jobs (or my entire Jenkins configuration) from one instance to another.
  • I’d like to horizontally scale my current CJP configuration by moving jobs between masters in order to distribute the load between multiple masters.

Environment

  • CloudBees Jenkins Enterprise (CJE)
  • Jenkins
  • CloudBees Jenkins Operations Center (CJOC)

Resolution

There are two ways to do this. Prior to attempting any Jenkins migration, please ensure your destination (new) Jenkins
instance meets the prerequisites, including the CloudBees Jenkins Platform Supported Java Versions,
Network Requirements, Hardware Recommendations

Case A - Migrate Jobs Only

This is the typical use case when you are planning to scale horizontally.

Use option A rather that B when the following circumstances are met:

  • You are moving the full pile of jobs between one Jenkins instance to another.
  • There is not a huge complexity folder tree among Jenkins instances.

Important: The configuration ( System, Credentials, Plug-in, Nodes, etc.) of the destination jenkins instances must be similar to the origin at the point of migrating jobs. Otherwise, jobs could fail in case the have any missing configuration dependencies (for instance, missing credential secret.key or artifactory server). After migrating jobs, there is not problem in having different configuration among your Jenkins according to your needs.

Option A - Relocating $JENKINS_HOME/jobs

1 . Copy the items in the $JENKINS_HOME/jobs folder from the source to the destination instance.
Ensure permissions are preserved for the ‘jenkins’ user when doing this, to avoid issues.

2 . Restart Jenkins (or “Reload configuration from disk” from the “Manage Jenkins” screen) on the destination instance.
Once Jenkins is up and running, you should be able to view/access/configure the newly copied jobs.
Note that job configurations may depend on a particular Jenkins version or plugins/versions being present…so they may not necessarily function the same way once migrated to a new instance.

Optional: Immediately run this script to disable all jobs (particularly if you have copied from a “Production” instance, and/or have a lot of scheduled jobs or jobs configured to poll from SCM repository:

Jenkins.instance.getAllItems().each { j ->
if (j instanceof com.cloudbees.hudson.plugins.folder.Folder) { return }
j.disable()
}

3 . Please see the Deleting Old Builds - Best Strategy for Cleanup and disk space management article for recommendations on configuring job settings following migration to your destination (new) instance.

Option B - Using Move/Copy/Promote

From a CJOC:

  1. Create a client master with your original Jenkins instance you want to migrate the jobs from.
  2. Create as many client master as you need, proportional to your job load.
  3. Use Copy/Move/Promote plugin to redistribute your job load from your original jenkins (1) to your other instances (2).

Case B - Migrate Entire Jenkins Configuration, Including Jobs

Note: In this example, we’re cloning a PROD Jenkins master, and cloning it to a new TEST Jenkins master.
(For reference please see the Test Instances section on our documentation site)

This will preserve most (but not all) of your Jenkins configuration and other settings. Some items (such as Slave connections) will need to be re-created on the TEST instance. Note that in using this method, it will be necessary to contact CloudBees Support (Licensing) to obtain an updated license key. The instance ID (and associated license string) from your source/original Jenkins instance will not work on the newly cloned TEST instance.
Please follow the following procedure:

Note: If possible, enable Quiet Mode in the PROD master -> then shutdown PROD master -> then take a backup of $JENKINS_HOME. After the backup is complete and copied over to TEST instance you can start SOURCE instance and disable QuietMode in PROD. This will help starting the TEST clone instance in Quiet mode by default initially.
Start new instance in Quiet Mode - https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/chapter-quiet-start.html#quiet-start

1 . Backup Prod Master $JENKINS_HOME

2 . Copy $JENKINS_HOME to Test master

3 . Reset Identity - https://cloudbees.zendesk.com/hc/en-us/articles/204713750-Copying-Jenkins-home-gives-duplicate-instance-ids

4 . Remove the following:
- $JENKINS_HOME/secret.key*
- $JENKINS_HOME/license.xml
- $JENKINS_HOME/identity.key.enc
- $JENKINS_HOME/jgroups/ (subdirectory) This will only be present if the SOURCE destination is member of HA cluster

Any JOC connection files (these likely won’t be present if you are running CloudBees Jenkins Enterprise only, without CloudBees Jenkins Operations Center), but may include:
- $JENKINS_HOME/operations-center-cloud*
- $JENKINS_HOME/operations-center-client*
- $JENKINS_HOME/com.cloudbees.opscenter.client.plugin.OperationsCenterRootAction.xml

5 . Check (and edit, if needed) config.xml for URL specific options. Remove any settings which contains references to the
source detination’s URL.

6 . Remove any slave connections and generate new connections to them. You can do this by just removing them from $JENKINS_HOME/nodes and removing the contents of $JENKINS_HOME/jgroups

7 . Start new instance in Quiet Mode - https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/chapter-quiet-start.html#quiet-start
You can enable this via the URLs /quietDown and /cancelQuietDown
e.g: https://<jenkins-url>/quietDown

8 . Upon startup, you will be prompted for CloudBees Jenkins registration.
- If you are connecting your new Test instance to a CloudBees Jenkins Operations Center (which is configured to centrally manage your licenses), you should be able to establish the connection.
- If you would like to run your newly configured CloudBees Jenkins Enterprise instance in standalone (not connected to CloudBees Jenkins Operations Center), please contact CloudBees and provide your instance ID to obtain a new license/registration. In the meantime, you may select the “Evaluation” option to proceed with the remaining steps prior to obtaining the license string.

9 . Immediately run this script to disable all jobs

Jenkins.instance.getAllItems().each { j ->
if (j instanceof com.cloudbees.hudson.plugins.folder.Folder) { return }
j.disable()
}

10 . Navigate to ‘Manage Jenkins’ -> ‘Configure System’ and ‘Configure Security’ and update any settings as needed. Some settings (such
as the Jenkins URL) are specific to each instance, and may need to be re-configured for your destination instance.

If needed, disable Security in CloudBees Jenkins Enterprise by editing the config.xml file in the $JENKINS_HOME folder:

Locate this line: <useSecurity>true</useSecurity>

And change it to: <useSecurity>false</useSecurity>

11 . It will be necessary to re-create the nodes, including Slave or Shared Slave connections. If any issues arise, please check (and repeat, if needed) step #4. If any of these files are corrupted, or the original copies weren’t replaced with new versions once the destination instance was started…issues might arise in creating Slave connections.

12 . Uncheck Quiet Restart

13 . Start enabling jobs for testing (note: only enable jobs that will not have side effects on production platform - ie don’t update SCMs, publish artifacts to repositories, send notifications etc)

14 . Please see the Deleting Old Builds - Best Strategy for Cleanup and disk space management
article for recommendations on configuring job settings following migration to your newly cloned TEST instance.

15 . If the Jenkins instance is not up-to-date with the most recent Jenkins version, plugin updates, and Jenkins Security Advisories, it’s strongly recommended to upgrade following migration. Please check the CloudBees Products site for current versions, and see the How to upgrade Jenkins guide to do this.

Case C - Migrate just credentials

As explained in How to migrate credentials to a new Jenkins instance

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.