How to abort a build that is stuck waiting for an available executor?

Issue

I have a job stuck waiting for an available executor. It turns out the required label of the agent is spelled incorrectly, therefore the build will never finish. Is there any way to timeout a build in this case?

Environment

Resolution

We recommmend using script Pipeline and placing the timeout option at the top level, for example:

timeout(time: 30, unit: 'MINUTES') {
  node {
    sh 'echo hello world'
  }
}

If you are using Declarative Pipeline, the timeout option at the top-agent level only takes effect after the agent becomes available. If you must enforce timeout before the agent is ready, you can workaround it by using agent none at the job level and specify your agent for each stage. See example under Using multiple agents.

If you can’t use agent none for any reason, please vote the RFE (#41660) in Pendo and add your use case.

For Freestye jobs, if the build is in the queue then it has not really started. That means we will need to abort the job from Jenkins.

You can use this groovy script to abort any builds. To detect long running builds, you can create an alert with one of the following metrics:

  • Local metric gause within rage with the gauge: jenkins.queue.stuck.value.
    This alert will be active when a lable cannot be found (tested working for both Freestyle and Pipeline jobs)

  • Local metric gause within rage with the gauge jenkins.queue.time.value.
    This gauge is only available if the CloudBees DevOptics Plugin is installed. This monitors how long each build has been in the queue.

Configure the alert to execute the groovy script to abort the builds.

Tested product/plugin versions

  • CloudBees CI Managed Master 2.235.2.3 / CloudBees Jenkins Enterprise 2.222.42.0.1-fixed
  • CloudBees DevOptics Plugin 1.1934

References

Have more questions?

0 Comments

Please sign in to leave a comment.