How to Trigger Multibranch Jobs from Bitbucket Server?

Issue

  • I want to configure Jenkins and Bitbucket Server to automatically trigger Multibranch or Bitbucket Team/Project jobs on changes

Environment

Resolution

The Bitbucket Branch Source plugin provides support for integration of Bitbucket with multibranch project such as Pipeline Multibranch. The API listens for notification to the endpoint /bitbucket-scmsource-hook/notify.

Configuration in Jenkins

Configuration in Bitbucket Server

Configuration in Jenkins

Global Configuration

(This is not applicable to versions older than Bitbucket Branch Source 2.2.0)

Configure a Bitbucket Endpoint under Manage Jenkins > Configure System > Bitbucket Endpoints:

Multibranch Project

To track a single repository, create an item of type Multibranch Pipeline and use the Bitbucket branch source:

Configure the SCM Source (refer to the help tooltips of each Behavior to better understand what it does):

Older Versions

Prior to Bitbucket Branch Source 2.2.0, there is nothing to configure in Manage Jenkins > Configure System. Configure the SCM as explained in the documentation of the Bitbucket Branch Source Plugin. In the Advanced section, specify the Bitbucket server URL - as well as the SSH port if using SSH authentication for checkout credentials:

Bitbucket Team/Project

To track multiple repositories of a Bitbucket project, create an item of type Bitbucket Team/Project and use Bitbucket Team/Project repository source:

Configure the SCM Source (refer to the help tooltips of each Behavior to better understand what it does):

Older Versions

Prior to Bitbucket Branch Source 2.2.0, there is nothing to configure in Manage Jenkins > Configure System. Configure the SCM as explained in the documentation of the Bitbucket Branch Source Plugin. In the Advanced section, specify the Bitbucket server URL - as well as the SSH port if using SSH authentication for checkout credentials:

Further Configuration: Hook Management

Once a project is saved, it automatically listens for events - commonly sent by webhooks - on the endpoint /bitbucket-scmsource-hook/notify.

The option Manage Hooks may be enabled for a specific endpoint under Manage Jenkins > Configure System > Bitbucket Endpoints to allow Jenkins to automatically register webhooks to that endpoint. This is the recommended, quicker, easiest configuration.

Notice that there are two different implementations to use for managing webhooks

  • Native: It will automatically create the webhook in the Webhooks section, which is the implementation of the native Bitbucket Webhooks
  • Plugin: It will automatically create the webhook in the Post Webhooks section, which is the implementation of the Bitbucket WebPost Hooks Plugin

Both options could be used, but the Native option is the recommended one in case you are running Bitbucket Server >= 5.4.0, version in which the native webhook integration was added. Since this version it is not really needed to use the Bitbucket WebPost Hooks Plugin - but you can still use it if you would like.

(Note: Since version 2.3.0, the Bitbucket Server Native Webhook API is supported. The global configuration provides an option to choose whether to use the native API - option native - or the Post Webhooks for Bitbucket add-on - option plugin.)

Older Versions

Between Bitbucket Branch Source 2.1.1 and Bitbucket Branch Source 2.2.0 and since Post Webhooks for Bitbucket 1.4.1 (bitbucket add-on), the option Auto-register webhook may be enabled to let Jenkins automatically register a webhook for scanned repository/ies. This is the recommended, quicker, easiest configuration.

Even Older Versions

Prior to Bitbucket Branch Source 2.1.1, the only workaround is to configure each repository hook manually in Bitbucket Server. However the Bitbucket REST API can be used to facilitate that process, have a look at the article Generate webhooks in Bitbucket Server via REST API for Pipeline Multibranch.

Configuration in Bitbucket Server

If Jenkins is not setup to automatically manage webhooks, see below to understand how to manually create the webhooks for a repository:

Bitbucket Branch Source 2.3.0+ / Bitbucket Server 5.4+

Bitbucket Server 5.4 introduced a native Webhook API. Support for this API has been implemented in Bitbucket Branch Source version 2.3.0.

To create a webhook, go to the Settings of a repository, section Webhooks and click on “Create webhook”:

Give it any “title” and specify the Bitbucket Branch Source endpoint $JENKINS_URL/bitbucket-scmsource-hook/notify?server_url=<BITBUCKET_URL>. Use the Test Connection feature to ensure that the webhook is able to ping the Jenkins URL:

Note In case you are using the Bitbucket WebPost Hooks Plugin, then the Branch Source endpoint should be just $JENKINS_URL/bitbucket-scmsource-hook/notify as this plugin is already injecting server_url.

Select any Repository and Pull Request event that should be processed by Jenkins. The above example shows the events that are automatically selected when generated by Bitbucket Branch Source as of version 2.4.0. See the supported events com.cloudbees.jenkins.plugins.bitbucket.hooks.HookEventType.java.

Save and ensure that the webhook has been properly created and is showing as “Active”:

Important If the manual hook is not working, we encourage you to configure the Bitbucket Branch Source plugin to automatically create the webhooks in order to check which is the right URL to be used.

Any Version

Starting from Bitbucket Branch Source 2.3.0, you can choose between two webhook implementations: native or plugin.

For plugin implementation only:

The add-on Post Webhooks for Bitbucket can be installed to work with any version of Bitbucket Branch Source or Bitbucket Server. This add-on has been designed by the community to provide support to the Bitbucket Branch Source plugin with Bitbucket Server as part of JENKINS-33507. The source code of this add-on can be found on GitHub at https://github.com/topicusfinan/bitbucket-webhooks-plugin.

Once installed, this add-on adds a Post Webhooks section (Webhooks section for older version of the add-on) in the Settings of a repository. To create a Webhook for a repository, click on Add WebHook on the top right:

Give it any “title” and specify the Bitbucket Branch Source endpoint $JENKINS_URL/bitbucket-scmsource-hook/notify:

Select any Repository and Pull Request event that should be processed by Jenkins. The above example shows the events that are automatically selected when generated by Bitbucket Branch Source as of version 2.4.0. See the supported events com.cloudbees.jenkins.plugins.bitbucket.hooks.HookEventType.java.

Note: In older versions of the Post Webhooks for Bitbucket, the Repository and Pull requests events could not be selected and were defaulted to most of pull requests updates and repository pushes

Save and ensure that the webhook has been properly created and is not showing as “Inactive”:

Troubleshooting

Please have a look at Bitbucket Webhooks Troubleshooting to troubleshoot issues with these solutions.

References

Bitbucket Integration

Have more questions?

15 Comments

  • 0
    Avatar
    K Caola

    The article is titiled "How to trigger Multibranch Jobs from BitBucket server" but the issue is "I want to configure Jenkins and BitBucket Cloud to automatically trigger Multibranch or Bitbcuket Team/Project jobs on changes". Then environment also lists BitBucket Server.

    Is that a typo?

    Or is this actually a how to for BitBucket Cloud AND NOT BitBucket Server?

  • 0
    Avatar
    Allan Burdajewicz

    Hi,

    You are right. This is a typo issue. This will be corrected soon.

    Thanks!

  • 0
    Avatar
    Nitul Kukadia

    @Allan Burdajewicz Please correct this typo issue. 

  • 0
    Avatar
    Denys Digtiar

    Hi Nitul,

    All the typos have already been corrected.

  • 0
    Avatar
    Nitul Kukadia

    Ok Allan Burdajewicz.

    Is this details are for BitBucket Cloud or Server?

  • 0
    Avatar
    Allan Burdajewicz

    Hi Nitul,

    This article is for Bitbucket Server integration.

  • 0
    Avatar
    Nitul Kukadia

    Thanks Allan,

    I configured integration successfully with server.

    Is there any official documentation for endpoint?URL: ENKINS_URL/bitbucket-scmsource-hook/notify

    I found JENKINS_URL/bitbucket-hook/  only.

  • 0
    Avatar
    K Caola

    Hi Nitul,

    I managed to get webhooks working using the "Bitbucket Server Webhook to Jenkins" plugin . When you use that you don't need a specific notify URL, just the main url for your jenkins server (such as http://<YourJenkinsServer>.com/). You can find it in one of the admin level panels, "Manage Jenkins" > "Configure System" > "Jenkins Location: Jenkins URL".

    If you have your jenkinsfile in a repository that does not have the webhooks (example, the webhook is in ProjectName repo and the jenkinsfile is in ProjectNameCI) then you need to check the "Omit Branch Name" box. If you have your jenkinsfile in a repository that has the webhooks then leave "Omit Branch Name" box unchecked.

    This is the setup that worked for us.
    For reference, I'm using Jenkins 2.46.1 and BitBucket server 5.5.1
    I hope this helps you!

     

    Edited by K Caola
  • 0
    Avatar
    David Norton

    Will support be added for X-Auth-Token header instead of HTTP Basic auth? Our BitBucket server is behind a single-sign-on provider and username/password auth is not available.

    UPDATE: our Bitbucket server uses the Token Authentication plugin here: https://marketplace.atlassian.com/plugins/com.thundermoose.plugins.stash-token-auth/server/overview

    Edited by David Norton
  • 0
    Avatar
    Roy Mammen

    Does the newer version of plugin supports ssh authentication? I would like to use ssh url (ssh://git@bitbucket.mycompany.com:7999/myproject/myrepo.git) instead of http url along with ssh private key from Jenkins credential store for authentication. Apparently the new plugin doesn’t allow me to choose ssh credentials from the credential drop down, and I also dont see the advance ssh port options within the Branch Source setup page within Jenkins. Is it by design? Or I am not looking at the right place?

    Thanks

  • 1
    Avatar
    Allan Burdajewicz

    Hello @Nitul, the endpoint `/bitbucket-scmsource-hook/notify` is not documented. The plugin was first designed to automatically create the webhook (and only for Bitbucket Cloud). And the documentation did not include reference to this endpoint. Something to improve.

    Hello @Roy, the Bitbucket Branch Source plugin uses the Bitbucket REST API to scan team project (Organization Scan) and branches (Branch Indexing). You can however use the SCM Trait "Checkout Over SSH" to use SSH for pipeline checkout.

    Hello @David, I believe Bitbucket Branch Source supports Personal Access Token authentication (added in Bitbucket Bucket Server 5.5). I am not sure about the Token Authenticator though. If this is a requirement, you could create an improvement issue in JIRA.

  • 0
    Avatar
    Deepak Narayanarao

    Hi,

    How is the authentication handled for webhook API call from bitbucket to cloudbees jenkins server?

  • 0
    Avatar
    Francisco Vieira

    Hello. I'm trying to configure Jenkins to build upon commit on BitBucket. So far I was only able to do this with the Git Plugin. 

    I followed the steps before, created the multibranch at the folder level, added the BitBucket Branch Source Plugin, created the hook on bitbucket and even tried to use the Bitbucket Server Webhook to Jenkins but nothing worked.

    Any thoughts on this?

  • 0
    Avatar
    Alexander Moiseenko

    Hi,

    Does sending webhook to /bitbucket-scmsource-hook/notify endpoint triggers every multibranch job to rescan it's repository for new commits or it works like git/notifyCommit endpoint with repo url parameter?

  • 0
    Avatar
    Allan Burdajewicz

    @Deepak At the moment, there is no authentication from Bitbucket to Jenkins. Since Bitbucket Branch Source 2.3.0, there is support for the Bitbucket Server Webhook API (see this PR). The Bitbucket native webhook API enables to provide a Secret to authenticate the payload. This could well be an improvement ticket for the Bitbucket Branch Source plugin.

    @Francisco I would suggest that you have a look at Bitbucket Webhooks Troubleshooting and open a support request if you need more help.

    @Alexander Unless you are using an old version of the SCM API plugin (prior to 2.0), a commit will trigger the respective branch job(s) only. The SCM AMI provide an event processing mechanism for fine grained notification.

Please sign in to leave a comment.