Issue
- Jenkins instance and users are in the same Timezone but Jenkins System keeps triggering builds randomly.
- On logs can be found a similar trace to the following:
2017-02-06 12:16:53.815-0600 [id=5089154] SEVERE s.s.stash.StashApiClient$1#call: Failing to get response from Stash PR GET/rest/api/1.0/projects/INNODELOC/repos/InNode-location-performance-ci/pull-requests
2017-02-06 12:16:53.816-0600 [id=37] WARNING hudson.triggers.Trigger#checkTriggers: stashpullrequestbuilder.stashpullrequestbuilder.StashBuildTrigger.run() failed for hudson.maven.MavenModuleSet@6d32c80b[Walmart-Technology-Reston/Enterprise-Location-Service-Performance-Test-CI]java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Didn't get a 200 response from Stash PR GET! Response; 'Not Found' with message; null
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getRequest(StashApiClient.java:262)
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getPullRequests(StashApiClient.java:86)
at stashpullrequestbuilder.stashpullrequestbuilder.StashRepository.getTargetPullRequests(StashRepository.java:72)
at stashpullrequestbuilder.stashpullrequestbuilder.StashPullRequestsBuilder.run(StashPullRequestsBuilder.java:32)
at stashpullrequestbuilder.stashpullrequestbuilder.StashBuildTrigger.run(StashBuildTrigger.java:299)
at hudson.triggers.Trigger.checkTriggers(Trigger.java:273)
at hudson.triggers.Trigger$Cron.doRun(Trigger.java:222)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Didn't get a 200 response from Stash PR GET! Response; 'Not Found' with message; null
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getRequest(StashApiClient.java:254)
... 14 more
Caused by: java.lang.RuntimeException: Didn't get a 200 response from Stash PR GET! Response; 'Not Found' with message; null
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient$1.call(StashApiClient.java:234)
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient$1.call(StashApiClient.java:221)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 1 more
- On thread dumps can be found a similar trace to the following which proves that Stash pullrequest builder plugin is doing too much work in the timer trigger thread, delaying all subsequent executions.
"jenkins.util.Timer [#1]" id=37 (0x25) state=TIMED_WAITING cpu=40%
- waiting on <0x1c6062eb> (a java.util.concurrent.FutureTask)
- locked <0x1c6062eb> (a java.util.concurrent.FutureTask)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:426)
at java.util.concurrent.FutureTask.get(FutureTask.java:204)
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getRequest(StashApiClient.java:254)
at stashpullrequestbuilder.stashpullrequestbuilder.stash.StashApiClient.getPullRequestMergeStatus(StashApiClient.java:150)
at stashpullrequestbuilder.stashpullrequestbuilder.StashRepository.isPullRequestMergable(StashRepository.java:209)
at stashpullrequestbuilder.stashpullrequestbuilder.StashRepository.isBuildTarget(StashRepository.java:261)
at stashpullrequestbuilder.stashpullrequestbuilder.StashRepository.getTargetPullRequests(StashRepository.java:75)
at stashpullrequestbuilder.stashpullrequestbuilder.StashPullRequestsBuilder.run(StashPullRequestsBuilder.java:32)
at stashpullrequestbuilder.stashpullrequestbuilder.StashBuildTrigger.run(StashBuildTrigger.java:299)
at hudson.triggers.Trigger.checkTriggers(Trigger.java:273)
at hudson.triggers.Trigger$Cron.doRun(Trigger.java:222)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked synchronizers: count = 1
- java.util.concurrent.ThreadPoolExecutor$Worker@2b0a3762
Environment
- Jenkins
- CloudBees Jenkins Enterprise
- Stash pullrequest builder plugin 1.7
JENKINS
Resolution
First, disable the Stash pullrequest builder plugin because it is blocking the scheduler and preventing all timer triggers.
Then, think about your jobs making use of other plugins which enable to build pull requests from an Atlassian Stash instance.
- Bitbucket Branch Source Plugin for Bitbucket 4.0 or later; Stash 3.x and earlier are not supported.
- Otherwise, Pipeline Multibranch Plugin
0 Comments