Migration Guide: CloudBees CI

Issue

  • I would like to migrate all of my Jenkins jobs (or my entire Jenkins configuration) from one instance to another.
  • I would like to horizontally scale my current CloudBees CI configuration by moving jobs between Controllers (formerly known as master) in order to distribute the workload.

Environment

Resolution

There are two ways to do this. Prior to attempting any Jenkins migration, please ensure your destination (new) Jenkins instance meets with Prepare Jenkins for Support – CloudBees Support.

Case A - Migrate Jobs Only

This is the typical use case when a Jenkins instance exceedsthe reocmmendation from CloudBees Master Sizing Guidelines. Please, refer also to Right-sizing Jenkins controllers

Use option A rather that B when the following circumstances are met:

  • You are moving the full pile of jobs between one Jenkins instance to another.
  • There is not a huge complexity folder tree among Jenkins instances.

Important: The configuration ( System, Credentials, Plug-in, Nodes, etc.) of the destination jenkins instances must be similar to the origin at the point of migrating jobs. Otherwise, jobs could fail in case the have any missing configuration dependencies (for instance, missing credential secret.key or artifactory server). After migrating jobs, there is not problem in having different configuration among your Jenkins according to your needs.

Option A - Relocating $JENKINS_HOME/jobs

  1. Copy the items in the $JENKINS_HOME/jobs folder from the source to the destination instance.
    Ensure permissions are preserved for the ‘jenkins’ user when doing this, to avoid issues.

  2. Restart Jenkins (or “Reload configuration from disk” from the “Manage Jenkins” screen) on the destination instance.
    Once Jenkins is up and running, you should be able to view/access/configure the newly copied jobs.
    Note that job configurations may depend on a particular Jenkins version or plugins/versions being present…so they may not necessarily function the same way once migrated to a new instance.

    Optional: Immediately run this script to disable all jobs (particularly if you have copied from a “Production” instance, and/or have a lot of scheduled jobs or jobs configured to poll from SCM repository. You can run scripts at the url, https://<jenkins-url>/script or by navigating to Manage Jenkins -> Script Console. This requires Admin permissions.

        Jenkins.instanceOrNull.getAllItems().each { j ->
        if (j instanceof com.cloudbees.hudson.plugins.folder.Folder) { return }
            j.setDisabled(true)
        }
    
  3. Please see the Deleting Old Builds - Best Strategy for Cleanup and disk space management article for recommendations on configuring job settings following migration to your destination (new) instance.

Option B - Using Move/Copy/Promote

From the Operation Center

  1. Create a Client/Managed/Team master with your original Jenkins instance you want to migrate the jobs from.
  2. Create as many Client/Managed/Team master as you need, proportional to your job load.
  3. Use Copy/Move/Promote plugin to redistribute your job load from your original jenkins (step 1) to your other instances (step 2) .

Case B - Migrate Entire Jenkins Configuration, Including Jobs

Option A - Just Migration

This will preserve most (but not all) of your Jenkins configuration and other settings. Some items (such as Agent connections) will need to be re-created on the destination instance. Note, that in using this method, it will be necessary to contact CloudBees Support (Licensing) to obtain an updated license key. The instance ID (and associated license string) from your source/original Jenkins instance will not work on the newly cloned TEST instance.

Please follow this procedure:

Note: If possible, enable Quiet Mode in the Origin Controller (formerly known as Master) -> then shutdown Origin Controller (formerly known as Master) -> then take a backup of $JENKINS_HOME. After the backup is complete and copied over to Destination instance you can start SOURCE instance and disable QuietMode in the Origin. This will help starting the Destination clone instance in Quiet mode by default initially. Start new instance in Quiet Mode.

If you are not using a CloudBees CI, you will not have access to the Quiet Start Plugin, instead refer to this article.

  1. Backup Prod Controller (formerly known as Master) $JENKINS_HOME. Refer to Backup and restore | CloudBees Docs

  2. Copy $JENKINS_HOME to Test Controller (formerly known as Master)

  3. Reset Identity

  4. Remove the following:

    • $JENKINS_HOME/secret.key*
    • $JENKINS_HOME/license.xml
    • $JENKINS_HOME/identity.key.enc
    • $JENKINS_HOME/jgroups/ (subdirectory) This will only be present if the SOURCE destination is member of HA cluster

    Any JOC connection files (these likely won’t be present if you are running CloudBees Jenkins Enterprise only, without CloudBees Jenkins Operations Center), but may include:

    • $JENKINS_HOME/operations-center-cloud*
    • $JENKINS_HOME/operations-center-client*
    • $JENKINS_HOME/com.cloudbees.opscenter.client.plugin.OperationsCenterRootAction.xml
  5. Check (and edit, if needed) config.xml for URL specific options. Remove any settings which contains references to the source detination’s URL.

  6. Remove any agent connections and generate new connections to them. You can do this by just removing them from $JENKINS_HOME/nodes and removing the contents of $JENKINS_HOME/jgroups

  7. Start new instance in Quiet Mode - https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/chapter-quiet-start.html#quiet-start.
    You can toggle this via the URLs /quietDown and /cancelQuietDown. e.g: https://<jenkins-url>/quietDown.
    If you are not using a CloudBees distributed release, you will not have access to the Quiet Start Plugin, instead refer to this article.

  8. Upon startup, you will be prompted for Registration.

    • If you are connecting your new Test instance to a Operation Center (which is configured to centrally manage your licenses), you should be able to establish the connection.
    • If you would like to run your newly configured Controller (formerly known as Master) in standalone (not connected to Operation Center), please contact CloudBees and provide your instance ID to obtain a new license/registration. In the meantime, you may select the “Evaluation” option to proceed with the remaining steps prior to obtaining the license string.
  9. Immediately run this script to disable all jobs. You can run scripts at the url, https://<jenkins-url>/script or by navigating to Manage Jenkins -> Script Console. This requires Admin permissions.

    Jenkins..instanceOrNull.getAllItems().each { j ->
    if (j instanceof com.cloudbees.hudson.plugins.folder.Folder) { return }
    j.setDisabled(true)
    }
    
  10. Navigate to ‘Manage Jenkins’ -> ‘Configure System’ and ‘Configure Security’ and update any settings as needed. Some settings (such
    as the Jenkins URL) are specific to each instance, and may need to be re-configured for your destination instance.

    If needed, disable Security

  11. It will be necessary to re-create the nodes, including Agent or Shared Agent connections. If any issues arise, please check (and repeat, if needed) step #4. If any of these files are corrupted, or the original copies weren’t replaced with new versions once the destination instance was started…issues might arise in creating Agent connections.

  12. Uncheck Quiet Restart - https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/chapter-quiet-start.html#quiet-start
    You can toggle this via the URLs /quietDown and /cancelQuietDown. e.g: https://<jenkins-url>/cancelQuietDown.
    If you are not using a CloudBees distributed release, you will not have access to the Quiet Start Plugin, instead refer to this article.

  13. Start enabling jobs for testing (note: only enable jobs that will not have side effects on production platform - e.g. don’t update SCMs, publish artifacts to repositories, send notifications etc)

  14. Please see the Deleting Old Builds - Best Strategy for Cleanup and disk space management
    article for recommendations on configuring job settings following migration to your newly cloned TEST instance.

Option B - Migration and Upgrade

  1. Fresh installation to latest the version of the product in [DESTINATION]. Important: Enroll this instance in the CloudBees Assurance Program and Allow automatic upgrades of plugins on restart.
  2. Verify the product is running fine in [DESTINATION].
  3. Stop [DESTINATION] instance.
  4. Backup [DESTINATION] JENKINS_HOME
  5. Apply steps from previous option Migrate Entire Jenkins Configuration up to step #6.
  6. [DESTINATION] Update the variable JENKINS_HOME to the “Migrated JENKINS_HOME”.
  7. Apply steps in Migrate Entire Jenkins Configuration starting from step #7.

The content A. Before Upgrading in Upgrade Guide: CloudBees Jenkins Platform and Team is applicable here too.

Have more questions?

0 Comments

Please sign in to leave a comment.