0

Pipeline fails to loop inside a NonCPS method in a master-slave setup

Can any jenkins expert help?

 

Where i am faulting. this have owned me for days now :( ... i am trying to generate an XML file. based on the data provided by a JSON file. 
 
 
The pipeline snippet is 
 
import groovy.xml.*
import groovy.json.JsonSlurper
 
node('slave1') {
  deleteDir()
  
  stage('Checkout') {
    checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [],
    userRemoteConfigs: [[credentialsId: 'b27f7cb2-efa8-496a-90d8-825b9332bf44', url: 'git@somerepo.git']]])
  }
  
  writeFile file: "MyFile.XML", text: GenerateXML()    //calling the method to generate the XML
  
  println "Generated the manifest XML"
}
 
 
@NonCPS
def GenerateXML() {
  /*
  parsing the obj.json file
  */
  def currentws = pwd()
  println currentws
  def jsonSlurper = new JsonSlurper();  
  //def fileReader = new BufferedReader(new FileReader("${currentws}/objects.json"))  //the file location need to change in the actual implementation
  def fileReader = readFile "${currentws}/objects.json"  // ^^avoided the above line to use the pipeline DSL 
  def parsedData = jsonSlurper.parse(fileReader)
  /*
  creating the xml
  */
  // def writer = new FileWriter("${currentws}/sampleManifest123.XML")
  def builder = new StreamingMarkupBuilder()
  builder.encoding = 'UTF-8'
  def xml = builder.bind {
    mkp.xmlDeclaration()
    mkp.declareNamespace('udm.DeploymentPackage' :'http://www.w3.org/2001/XMLSchema')
    mkp.declareNamespace('powercenter.PowercenterXml' :'http://www.w3.org/2001/XMLSchema')
    delegate."udm.DeploymentPackage"(version:'$BUILD_NUMBER', application: "sampleApp"){
      delegate.deployables {
        parsedData.each { index, obj ->
          it."powercenter.PowercenterXml"(name:obj.name, file:obj.file) {
            delegate.scanPlaceholders(true)
            delegate.sourceRepository(obj.sourceRepository)
            delegate.folderNameMap {
              obj.folderNames.each { name, value ->
                it.entry(key:name, value)
              }
            }
            delegate.objectNames {
              delegate.value(obj.objectName)
            }
            delegate.objectTypes {
              delegate.value(obj.objectType)
            }
          }
        }
      }
      delegate.dependencyResolution('LATEST')
      delegate.undeployDependencies(false)
    }
  }
  
}
 
 
 
 
 
The input json file looks like:
 
{"workflows1":
  {
    "name": "/wf_multifolder",
    "file": "release1/wf_multifolder.XML",
    "objectName": "wf_multifolder.XML",
    "objectType": "workflow",
    "sourceRepository": "DEV2",
    "folderNames": { "multifolder": "{{multifolder}}","agent1": "{{agentx}}" }
  },
  "workflows1":
    {
      "name": "/wf_multifolder",
      "file": "release1/wf_multifolder.XML",
      "objectName": "wf_multifolder.XML",
      "objectType": "workflow",
      "sourceRepository": "DEV2",
      "folderNames": { "multifolder": "{{multifolder}}","agent1": "{{agentx}}" }
    }
}
 
 
 
 
 
 
The current behaviour is that, the method just works fine in my groovyConsole but fails big time in jenkins, The file "MyFile.XML"  is created in the workspace. But it have the same content as the input json file... Its not performing the loop. 
 

5 comments

  • 0
    Avatar
    Ivan Fernandez Calvo

    If you take a look at https://jenkins.io/blog/2017/02/01/pipeline-scalability-best-practice/  it is not recommended the use of .each{} for loops this is a common problem. You need to use C-style loop, i.e. for(int i=0; i<max; i++){ }

  • 0
    Avatar
    Niristotle Okram

    Thanks for the pointer. I have modified the pipeline.. But there seems to be another issue. The for loops works, until i use another FOR loop inside the first FOR loop (a nested FOR loop).. Is this a bug?

  • 0
    Avatar
    Niristotle Okram

    Please ignore, i had a typo 

  • 0
    Avatar
    Ivan Fernandez Calvo

    Did you resolve it changing to FOR loops?

  • 0
    Avatar
    Niristotle Okram

    Yes, i have to change the JSON file to use List for the the FOR loop to work. My test was in a jenkins 2.x env. But unfortunately when i put the script into a jenkins running in ver1.6; things keeps failing... 

    Looks like there is a namespace class between the 'invokeMethod' in the StreamMarkupBuilder() and something else in Jenkins. 

    I am not sure on what exactly needs to be upgraded to fix this.  

Please sign in to leave a comment.