Why are jobs on the queue when there are dedicated agents with free executors

Issue

Jobs stay on the Build queue when there are dedicated agents with free executors.

  • Note: This KB article is not applied for shared agents leased from CJOC.

Environment

Resolution

There are multiple causes which might produce this issue, but if you want to diagnosis this issue you might want to try running in /script:

for (i in Jenkins.instance.queue.buildableItems) {
  println "considering ${i}"
  for (c in Jenkins.instance.computers) {
    println "found computer ${c}"
    EXEC: for (e in c.executors) {
      if (e.interrupted || !e.parking) continue
      println "with executor ${e}"
      def o = new Queue.JobOffer(e)
      if (!o.canTake(i)) {
        println "${o} refused ${i}"
        def node = o.node
        if (node == null) {
          println "no node associated with ${c}"
          continue
        }
        def cob = node.canTake(i)
        if (cob != null) {
          println "because of ${cob}"
          continue
        }
        for (d in hudson.model.queue.QueueTaskDispatcher.all()) {
          cob = d.canTake(node, i)
          if (cob != null) {
            println "because of ${cob} from ${d}"
            continue EXEC
          }
        }
        if (!o.available) {
          println "${o} not available"
          if (o.workUnit != null) println "has a workUnit ${o.workUnit}"
          if (c.offline) println "${c} is offline"
          if (!c.acceptingTasks) println "${c} is not accepting tasks"
        }
      }
    }
  }
}

Note: This script was proposed in JENKINS-38514

Known causes

Misconfiguration in Authorize Project

In one reported case, the root issue was that the Authorize Project plugin was configured, so Node.canTake was returning anonymous doesn’t have a permission to run on [sic]; yet the build queue (and support bundle) displayed only Waiting for next available executor.

jobs-stay-on-queue.png

considering hudson.model.Queue$BuildableItem:hudson.model.FreeStyleProject@1878318e[test-a]:1
found computer hudson.model.Hudson$MasterComputer@e9bc860
with executor Thread[Executor #0 for master,5,main]
JobOffer[ #0] refused hudson.model.Queue$BuildableItem:hudson.model.FreeStyleProject@1878318e[test-a]:1
because of anonymous doesn’t have a permission to run on
with executor Thread[Executor #1 for master,5,main]
JobOffer[ #1] refused hudson.model.Queue$BuildableItem:hudson.model.FreeStyleProject@1878318e[test-a]:1
because of anonymous doesn’t have a permission to run on
Have more questions? Submit a request

2 Comments

  • 0
    Avatar
    Mark Kenneally

     

    Thanks, this was very helpful.

    Regards

    -mark

     

     

    Edited by Mark Kenneally
  • 0
    Avatar
    Manuel Cardenas

    Hi Mark,

    We hope that we provide the information required in the support ticket.

    Best regards,

    Manuel

Please sign in to leave a comment.