Bitbucket Webhooks Troubleshooting

Symptoms

  • Jobs are not automatically triggered when a changes are pushed to Bitbucket

Diagnostic/Treatment

  • Pre-condition: Jenkins (resp. Bitbucket) can reach out to Bitbucket (resp. Jenkins)

Non-Multibranch

In Jenkins for non-multibranch jobs, you are either using the Git Plugin or the Bitbucket Plugin.

Git Plugin

1) Endpoint

This applies to repository configured with a hook to the endpoint $JENKINS_URL/git/notifyCommit

2) Add-on (for Bitbucket Server)

This endpoint is used by the add-on Webhook to Jenkins for Bitbucket in Bitbucket Server:

3) Review Configuration

Review the articles:

4) Troubleshooting

To troubleshoot request processing on that endpoint, please add a logger to the following class:

hudson.plugins.git.GitStatus

Expected Logs

With branch and/or sha specified:

2017-03-10 08:07:09.273+0000 [id=831118]    FINE    h.p.g.GitStatus$JenkinsAbstractProjectListener#onNotifyCommit: Received notification for uri = https://my.bitbucket.server/myuser/myrepo.git ; sha1 = e3baa50a43479d6d0246fa2ba5d921c0e06fc707 ; branches = [develop]
2017-03-10 08:07:09.273+0000 [id=831118]    FINE    h.p.g.GitStatus$JenkinsAbstractProjectListener#onNotifyCommit: Branch Spec origin/develop matches modified branch develop for mybitbucket » myrepo.
2017-03-10 08:07:09.273+0000 [id=831118]    INFO    h.p.g.GitStatus$JenkinsAbstractProjectListener#onNotifyCommit: Scheduling folder-bitbucket » myrepo to build commit e3baa50a43479d6d0246fa2ba5d921c0e06fc707

Without branch and/or sha specified:

2017-03-13 02:39:51.532+0000 [id=106781]	FINE	h.p.g.GitStatus$JenkinsAbstractProjectListener#onNotifyCommit: Received notification for uri = https://my.bitbucket.server/myuser/myrepo.git ; sha1 = null ; branches = []
2017-03-13 02:39:51.532+0000 [id=106781]	INFO	h.p.g.GitStatus$JenkinsAbstractProjectListener#onNotifyCommit: Triggering the polling of mybitbucket » myrepo

Bitbucket Plugin

1) Endpoint

This applies to Bitbucket repositories configured with a hook to the endpoint $JENKINS_URL/bitbucket-hook/.

2) Add-on (for Bitbucket Server)

There are currently some issues using the to process payload from Bitbucket Server which is captured in JENKINS-28877

Bitbucket Plugin integrates with Bitbucket Server via the add-on Bitbucket Server Web Post Hooks Plugin in Bitbucket server:

3) Review Configuration

Review the articles:

4) Troubleshooting

To troubleshoot request processing on that endpoint, please add a logger to the following class:

com.cloudbees.jenkins.plugins.BitbucketJobProbe
com.cloudbees.jenkins.plugins.BitBucketTrigger
com.cloudbees.jenkins.plugins.BitbucketPayloadProcessor
com.cloudbees.jenkins.plugins.BitbucketHookReceiver

Expected Logs

2017-03-13 02:32:36.138+0000 [id=127609]	FINE	c.c.j.p.BitbucketHookReceiver#doIndex: Received commit hook notification : {<payload>}
2017-03-13 02:32:36.207+0000 [id=127609]	INFO	c.c.j.p.BitbucketPayloadProcessor#processPayload: Processing new Webhooks payload
2017-03-13 02:32:36.208+0000 [id=127609]	INFO	c.c.j.p.BitbucketPayloadProcessor#processWebhookPayload: Received commit hook notification for {<simplified-payload>}
2017-03-13 02:32:36.307+0000 [id=127609]	FINE	c.c.j.plugins.BitbucketJobProbe#triggerMatchingJobs: Considering candidate job freestyle-push
2017-03-13 02:32:36.307+0000 [id=127609]	FINE	c.c.j.plugins.BitbucketJobProbe#triggerMatchingJobs: Considering to poke support-team » Bucket » CE-424 » BitbucketCloud » freestyle-push
2017-03-13 02:32:36.311+0000 [id=127609]	INFO	c.c.j.plugins.BitbucketJobProbe#triggerMatchingJobs: Triggering BitBucket job freestyle-push

Multibranch

In Jenkins, for multibranch jobs, you should be using the Bitbucket Branch Source Plugin.

1) Endpoint

This applies to repositories configured with a hook to the endpoint $JENKINS_URL/bitbucket-scmsource-hook/notify/.

2) Add-on (for Bitbucket Server)

Bitbucket Branch Source Plugin integrates with Bitbucket Server via the add-on Post Webhooks for Bitbucket in Bitbucket server:

3) Review Configuration

Review the articles:

4) Troubleshooting

To troubleshoot request processing on that endpoint, please add a logger to the following class:

com.cloudbees.jenkins.plugins.bitbucket.hooks

Expected Logs

Registering Hooks:

2017-03-12 23:50:10.659+0000 [id=28857]	INFO	c.c.j.p.b.h.WebhookAutoRegisterListener#registerHooks: Registering hook for jenkinssupport/simple-project
2017-03-13 00:15:57.702+0000 [id=28857]	FINE	c.c.j.p.b.h.WebhookAutoRegisterListener#removeHooks: NOT removing hook for jenkinssupport/simple-project because does not exists or its used in other project

Processing Reindexing:

2017-03-13 02:33:27.697+0000 [id=132556]	INFO	c.c.j.p.b.h.PushHookProcessor#process: Received hook from Bitbucket. Processing push event on jenkinssupport/simple-project
2017-03-13 02:33:27.699+0000 [id=132556]	INFO	c.c.j.p.b.hooks.HookProcessor$1#run: Multibranch project found, reindexing jenkinssupport
2017-03-13 02:33:27.699+0000 [id=132556]	INFO	c.c.j.p.b.hooks.HookProcessor$1#run: Multibranch project found, reindexing simple-project
2017-03-13 02:33:27.700+0000 [id=132556]	INFO	c.c.j.p.b.hooks.HookProcessor$1#run: Multibranch project found, reindexing pipeline-multibranch

5) Indexing Logs

Any event received on this endpoint should trigger a Branch indexing. But only the logs of the last branch indexing are persisted. If an issue seems intermittent and it is difficult to catch the Branch Indexing that exposes it. In order to catch the issue, you can set the system property com.cloudbees.hudson.plugins.folder.computed.FolderComputation.BACKUP_LOG_COUNT to for example 10 in order to keep the logs of the last 10 branch indexing.

You can set this property on startup by adding it to Jenkins Java arguments or run the following Groovy script in Manage Jenkins > Script Console:

System.setProperty('com.cloudbees.hudson.plugins.folder.computed.FolderComputation.BACKUP_LOG_COUNT','10')

The branch indexing files are located in the File System in $JENKINS_HOME/jobs/<pathToMultibranch>/indexing/*
The folder computation files are located in the File System in $JENKINS_HOME/jobs/<pathToMultibranch>/computation/*

Troubleshooting Further

If for some reason the request is not received by Jenkins and improved logging did not help, you can troubleshoot this even further by enabling Access Logging for Jenkins.

Expected Logs

Bitbucket Cloud

192.168.0.1 - - [13/Mar/2017:13:23:28 +1000] "POST /bitbucket-scmsource-hook/notify HTTP/1.1" 200 0 "-" "Bitbucket-Webhooks/2.0"
192.168.0.1 - - [13/Mar/2017:13:23:28 +1000] "POST /bitbucket-hook/ HTTP/1.1" 200 0 "-" "Bitbucket-Webhooks/2.0"

Bitbucket Server

192.168.0.2 - - [13/Mar/2017:13:23:52 +1000] "POST /bitbucket-scmsource-hook/notify HTTP/1.1" 200 0 "-" "Atlassian HttpClient 0.23.0 / Bitbucket-4.13.0 (4013000) / Default"
192.168.0.2 - - [13/Mar/2017:13:23:53 +1000] "GET /git/notifyCommit HTTP/1.1" 200 110 "-" "Apache-HttpClient/4.5.1 (Java/1.8.0_102)"
192.168.0.2 - - [13/Mar/2017:13:23:53 +1000] "POST /bitbucket-hook/ HTTP/1.1" 500 3348 "-" "Bitbucket 4.13.0"

What to attach to a support ticket

Open a Support ticket if you are stuck at this point, adding a the following information:

  • Screenshot of the hook configured in Bitbucket
  • Support Bundle generated after adding logging and reproducing the issue
  • config.xml of the Jenkins job (for multibranch/organization folder, we need the config.xml of the multibranch/organization project)
  • If applicable, the branch indexing logs

References

Bitbucket Integration

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.