Pipeline - Parallel execution of tasks

Issue

  • Want to execute parallel tasks in Pipeline

Environment

Resolution

Following a couple of options to achieve it.

Disclaimer: Please note that the above script just has been proved/tested within the specified environment. In the case a particular step does work within your environment, please go to <JENKINS_URL>/pipeline-syntax/ validate the syntax of the problematic step with the output from the Snippet Generator.

Option 1

def tasks = [:]

tasks["task_1"] = {
  stage ("task_1"){    
    node('label_example1') {  
        sh 'echo $NODE_NAME'
    }
  }
}
tasks["task_2"] = {
  stage ("task_2"){    
    node('label_example2') {  
        sh 'echo $NODE_NAME'
    }
  }
}

parallel tasks

Option 2

def testList = ["a", "b", "c", "d"]
def branches = [:] 

for (int i = 0; i < 4 ; i++) {
       int index=i, branch = i+1
       stage ("branch_${branch}"){ 
            branches["branch_${branch}"] = { 
                node ('label_example'){
                    sh "echo 'node: ${NODE_NAME},  index: ${index}, i: ${i}, testListVal: " + testList[index] + "'"
                }
            }
      }
}

parallel branches

Note: Variables define outside the for block are not local, but global to the script. Testing the option 2, you will notice that variable i prints always value 4, whereas index increases from 0 to 3 and branch from 1 to 4.

References

Have more questions? Submit a request

3 Comments

  • 0
    Avatar
    Serosset Serosset

    Does "parallel" have options to control what happens in case of failure of one of the parallel tasks? What is the default behavior? Fail the "parallel" statement as soon as one of the tasks fail?

    We are using a pipeline job with a combination of "parallel" and downstream jobs, what happens when a downstream job fails? (similar to the example at https://support.cloudbees.com/hc/en-us/articles/221400287-How-to-pass-parameter-to-downstream-job-in-Pipeline-job)

    Edited by Serosset Serosset
  • 0
    Avatar
    Pushpa Buzz

    I have used option-1. I am not passing any branch details or anything. Just defined the ubuntu image and running scripts in stage under sh''' .. '''. 

    I see "Failed in branch" and hung the job as well.

    Can you help me?

     

    [task_1] [Pipeline] [task_1] // stage
    [Pipeline] [task_1] }
    [task_1] Failed in branch task_1
  • 0
    Avatar
    Anand Sridharan

    With option1 I am getting the following error

    ERROR: Build step failed with exception groovy.lang.MissingMethodException: No signature of method: Script1.parallel() is applicable for argument types: (java.util.LinkedHashMap) values: [[task_1:Script1$_run_closure1@535b4b7, task_2:Script1$_run_closure2@73eb3b5a]]

    I am running Jenkins 2.89.4 with the latest version of pipeline. I am running this as a "Execute System groovy script" in a job. 

     EDIT: 

    Executing this code as "Pipeline script" works. I reckon libraries relating to pipeline doesn't load when i execute this as groovy script. 

    Edited by Anand Sridharan
Please sign in to leave a comment.