Why is my multibranch project cloning the whole repository on the master?

Issue

  • My multibranch pipeline clones the entire repository on the master to obtain the Jenkinsfile
  • My multibranch pipeline shows a timeout error when cloning the repository to obtain the Jenkinsfile:
Branch indexing
23:43:32 Connecting to https://api.github.com using my-credentials/****** (my-credentials GitHub personal access token)
Checking out git https://github.com/org/repo.git into /var/jenkins_home/workspace/_repo_PR-123-P7DALB4WUZUT3GSAR3HPW2RCNJME43BF5FNLWOCKUK55BAHQV6SA@script to read Jenkinsfile
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
Cloning repository https://github.com/org/repo.git
 > git init /var/jenkins_home/workspace/_repo_PR-123-P7DALB4WUZUT3GSAR3HPW2RCNJME43BF5FNLWOCKUK55BAHQV6SA@script # timeout=10
Fetching upstream changes from https://github.com/org/repo.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials my-credentials GitHub personal access token
 > git fetch --no-tags --progress https://github.com/org/repo.git +refs/pull/123/head:refs/remotes/origin/PR-123 +refs/heads/master:refs/remotes/origin/master
ERROR: Timeout after 10 minutes
[...]

Environment

Related Issues

  • JENKINS-33273: Optimize Jenkinsfile loading and branch detection
  • JENKINS-43194: Lightweight checkout for PR merge jobs
  • JENKINS-42518: Provide lightweight checkout capability for bitbucket to avoid repository clone for multi-branch pipeline jobs

Resolution

Pipeline from SCM used to perform several checkout as explained in JENKINS-33273. Since version 2.13 of Pipeline Multibranch Plugin and the resolution of JENKINS-33273, Pipeline from SCM offers the ability to perform a lightweight checkout - if the SCM supports it - to retrieve the Jenkinsfile. With a lightweight checkout, no clone is needed to retrieve the Jenkinsfile anymore. The Jenkinsfile is instead retrieved using an API.

At the time of this article, the following plugin provide support for lightweight checkout:

There are still cases for which the lightweight checkout is not supported.

  • PR merge jobs (PR merged with the base branch)

In this scenario, it falls back to the previous behavior - a heavyweight checkout - which mean that a full clone is performed to retrieve the Jenkinsfile.

For such cases, there are workarounds that can be used until further support for a lightweight checkout is provided.

Workaround 1: Use a reference Repository

This workaround requires SCM API v2.2.0: Prior to that version, only the behavior of the checkout step can be adjusted - see [How to Customize Checkout for Pipeline Multibranch? - but not the behavior of the checkout carried out to obtain the Jenkinsfile.

A reference repository is a general tip to speed up the checkout/cloning of large Git repository. You can set up a reference repository on the master that will serve as a cache for your repository. It also saves disk space and reduce the traffic between Jenkins and the Git server.

First, have a look at the article Using a Git reference repository to understand how to set up a reference repository.

Then, in the configuration of the organization / multibranch project, add an SCM Trait of type Advanced Clone Behaviors and specify the path to the reference repository:

Workaround 2: Increase the Git timeout

If affected by the timeout error, it is possible to increase the Git timeout.

You can increase the timeout globally by setting the system property -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=<number of minutes> on startup. Upon restart, you can check that the timeout is properly set by running the groovy script return org.jenkinsci.plugins.gitclient.CliGitAPIImpl.TIMEOUT under Manage Jenkins > Script Console.

SCM API v2.2.0 and later

You can set the timeout per organization / multibranch. In the configuration of organization / multibranch project, add the SCM Traits of type Advanced Clone Behaviors and Advanced Checkout Behaviors and set a suitable value for the timeout (in minutes):

About SCM API v2.2.0

Since version 2.2.0 of the SCM API Plugin, it is now possible to add SCM Traits - like the git additional behaviors - at the organization / multibranch level. These traits apply to:

  • the checkout performed to retrieve the Jenkinsfile
  • the checkout scm step

SCM API v2.2+ along with GitHub Branch Source 2.2+ and Bitbucket Branch Source 2.2+ are available under the CloudBees Assurance Program since CJE 2.73.3.1.

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.