Upgrade Cloudbees(AWS EKS)
Hello.,
We are trying to upgrade cloudbees jenkins that is running on top of AWS based EKS (kubernetes) and this jenkins environment or "cloudbees core cloud" is running having jenkins master. We are trying to upgrade from 2.190.x to 2.204.x and it is failing on all multiple jenkins master with below exception stack, please advise resolution to fix upgrade on multiple jenkins. PS : we have two EKS, one for production running 2.190.x and non production running 2.190.x, on this non-prod, we are trying to upgrade this one to 2.204.x
+ eval 'exec java -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Duser.home=/var/jenkins_home -Xmx2867m -Xms2867m -Djenkins.model.Jenkins.slaveAgentPort="50000" -Dhudson.TcpSlaveAgentListener.port="50001" -Dhudson.lifecycle="hudson.lifecycle.ExitLifecycle" -DMASTER_NAME="toolseng" -Dcb.BeekeeperProp.autoInstallIncremental="true" -Djenkins.model.Jenkins.slaveAgentPortEnforce="true" -DMASTER_GRANT_ID="1e4d35ff-938e-4525-bf98-a722409112fe" -DMASTER_DOMAIN="teams-toolseng" -Dcb.IMProp.warProfiles.cje="kubernetes.json" -DMASTER_INDEX="0" -Dcb.IMProp.warProfiles="bluesteel-core.json" -DMASTER_OPERATIONSCENTER_ENDPOINT="http://cjoc.cje.svc.cluster.local/cjoc" -Dcb.BeekeeperProp.noFullUpgrade="true" -DMASTER_ENDPOINT="https://build-test.devsnc.com/teams-toolseng/" -XshowSettings:vm -XX:MaxRAMFraction=1 -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.DNSMultiCast.disabled=true -Dcasc.reload.token=tagalongs65oz -Dcb.BeekeeperProp.autoInstallIncremental=true -Dcb.BeekeeperProp.noFullUpgrade=true -Dhudson.model.ParametersAction.keepUndefinedParameters=true -Dhudson.model.ParametersAction.safeParameters=priority,branch -Dfile.encoding=UTF-8 -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteArtifacts=true -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteStashes=true -Djenkins.security.ApiTokenProperty.showTokenToAdmins=true -Djenkins.security.ApiTokenProperty.adminCanGenerateNewTokens=true -Dpermissive-script-security.enabled=no_security -Duser.timezone=America/Los_Angeles -Dsun.jnu.encoding=UTF-8 -jar -Dcb.distributable.name="Docker Common CJE" -Dcb.distributable.commit_sha=832ca28b7039b60c8530ba2496003cc062863891 /usr/share/jenkins/jenkins.war --webroot=/tmp/jenkins/war --pluginroot=/tmp/jenkins/plugins --prefix=/teams-toolseng/ "$@"'
++ exec java -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Duser.home=/var/jenkins_home -Xmx2867m -Xms2867m -Djenkins.model.Jenkins.slaveAgentPort=50000 -Dhudson.TcpSlaveAgentListener.port=50001 -Dhudson.lifecycle=hudson.lifecycle.ExitLifecycle -DMASTER_NAME=toolseng -Dcb.BeekeeperProp.autoInstallIncremental=true -Djenkins.model.Jenkins.slaveAgentPortEnforce=true -DMASTER_GRANT_ID=1e4d35ff-938e-4525-bf98-a722409112fe -DMASTER_DOMAIN=teams-toolseng -Dcb.IMProp.warProfiles.cje=kubernetes.json -DMASTER_INDEX=0 -Dcb.IMProp.warProfiles=bluesteel-core.json -DMASTER_OPERATIONSCENTER_ENDPOINT=http://cjoc.cje.svc.cluster.local/cjoc -Dcb.BeekeeperProp.noFullUpgrade=true -DMASTER_ENDPOINT=https://build-test.devsnc.com/teams-toolseng/ -XshowSettings:vm -XX:MaxRAMFraction=1 -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.DNSMultiCast.disabled=true -Dcasc.reload.token=tagalongs65oz -Dcb.BeekeeperProp.autoInstallIncremental=true -Dcb.BeekeeperProp.noFullUpgrade=true -Dhudson.model.ParametersAction.keepUndefinedParameters=true -Dhudson.model.ParametersAction.safeParameters=priority,branch -Dfile.encoding=UTF-8 -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteArtifacts=true -Dio.jenkins.plugins.artifact_manager_jclouds.s3.S3BlobStoreConfig.deleteStashes=true -Djenkins.security.ApiTokenProperty.showTokenToAdmins=true -Djenkins.security.ApiTokenProperty.adminCanGenerateNewTokens=true -Dpermissive-script-security.enabled=no_security -Duser.timezone=America/Los_Angeles -Dsun.jnu.encoding=UTF-8 -jar '-Dcb.distributable.name=Docker Common CJE' -Dcb.distributable.commit_sha=832ca28b7039b60c8530ba2496003cc062863891 /usr/share/jenkins/jenkins.war --webroot=/tmp/jenkins/war --pluginroot=/tmp/jenkins/plugins --prefix=/teams-toolseng/
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
VM settings:
Min. Heap Size: 2.80G
Max. Heap Size: 2.80G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM
Running from: /usr/share/jenkins/jenkins.war
2020-02-17 06:22:45.642+0000 [id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging initialized @2104ms to org.eclipse.jetty.util.log.JavaUtilLog
2020-02-17 06:22:45.764+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file
2020-02-17 06:22:49.542+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-9.4.z-SNAPSHOT; built: 2019-05-02T00:04:53.875Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 1.8.0_232-b09
2020-02-17 06:22:49.872+0000 [id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /teams-toolseng, did not find org.eclipse.jetty.jsp.JettyJspServlet
2020-02-17 06:22:49.965+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0
2020-02-17 06:22:49.966+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults
2020-02-17 06:22:49.968+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 660000ms
2020-02-17 06:22:50.054+0000 [id=1] INFO c.c.j.i.b.RestoreServletListener#contextInitialized: Jenkins restore listener found no infradna-backup-restore.properties, continuing without restore.
2020-02-17 06:22:50.571+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
2020-02-17 06:22:50.762+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started w.@2d2acd89{CloudBees Core Managed Master,/teams-toolseng,file:///tmp/jenkins/war/,AVAILABLE}{/tmp/jenkins/war}
2020-02-17 06:22:50.839+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStart: Started ServerConnector@132ddbab{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-02-17 06:22:50.839+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: Started @7301ms
2020-02-17 06:22:50.840+0000 [id=21] INFO winstone.Logger#logInternal: Winstone Servlet Engine v4.0 running: controlPort=disabled
2020-02-17 06:22:52.356+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Started initialization
2020-02-17 06:22:54.438+0000 [id=26] WARNING hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created /tmp/jenkins/plugins/greenballs/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness
2020-02-17 06:22:55.861+0000 [id=26] WARNING hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created /tmp/jenkins/plugins/heavy-job/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness
2020-02-17 06:22:59.036+0000 [id=26] WARNING hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created /tmp/jenkins/plugins/job-dsl/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness
2020-02-17 06:23:01.579+0000 [id=26] WARNING hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created /tmp/jenkins/plugins/purge-build-queue-plugin/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness
2020-02-17 06:23:02.796+0000 [id=26] WARNING hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created /tmp/jenkins/plugins/groovy-label-assignment/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness
2020-02-17 06:23:03.844+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: Listed all plugins
2020-02-17 06:23:24.857+0000 [id=26] INFO jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
2020-02-17 06:23:26.452+0000 [id=27] INFO c.c.j.p.k.KubernetesCredentialProvider#startWatchingForSecrets: retrieving secrets with selector: jenkins.io/credentials-type, LabelSelector(matchExpressions=[], matchLabels={}, additionalProperties={})
2020-02-17 06:23:27.043+0000 [id=27] INFO o.internal.platform.Platform#log: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
2020-02-17 06:23:28.535+0000 [id=26] INFO h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component com.cloudbees.hudson.plugins.modeling.integrations.promoted_builds.PromotionProcessTemplateProperty$DescriptorImpl; skipping
2020-02-17 06:23:32.856+0000 [id=26] INFO h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component hudson.plugins.templateproject.ProxySCM$DescriptorImpl; skipping
2020-02-17 06:23:33.639+0000 [id=27] WARNING c.c.j.p.k.KubernetesCredentialProvider#convertSecret: Failed to convert Secret 'github-commit-status' of type secretText due to secretText credential is missing the text
2020-02-17 06:23:33.733+0000 [id=41] INFO o.internal.platform.Platform#log: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
2020-02-17 06:23:34.037+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Started all plugins
2020-02-17 06:23:34.237+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
2020-02-17 06:23:34.443+0000 [id=46] INFO c.c.j.p.a.CloudBeesAssurance#installExtension: Installing the Plugin Catalog
2020-02-17 06:23:34.443+0000 [id=46] INFO c.c.j.p.a.CloudBeesAssurance#validateExtension: Validating the Plugin Catalog
2020-02-17 06:23:34.443+0000 [id=46] INFO c.c.j.p.a.CloudBeesAssurance#loadAndExpandExtension: Loading and expanding the Plugin Catalog to retrieve all the information
2020-02-17 06:23:34.446+0000 [id=46] WARNING c.c.j.p.a.CloudBeesAssurance#installExtension: The Plugin Catalog cannot be validated
2020-02-17 06:23:35.746+0000 [id=27] WARNING i.j.p.casc.BaseConfigurator#createAttribute: Can't handle class javaposse.jobdsl.plugin.GlobalJobDslSecurityConfiguration#metaClass: type is abstract but not Describable.
2020-02-17 06:23:35.751+0000 [id=27] WARNING i.j.p.casc.BaseConfigurator#createAttribute: Can't handle class javaposse.jobdsl.plugin.GlobalJobDslSecurityConfiguration#metaClass: type is abstract but not Describable.
2020-02-17 06:23:36.258+0000 [id=26] INFO c.c.o.c.c.ConfigurationSnippetRefresher#<init>: ConfigurationSnippetRefresher configured with recurrence period of 60,000ms
2020-02-17 06:23:37.859+0000 [id=26] INFO j.a.PrioritySorterPlugin#init1: Configuring the Priority Sorter ...
2020-02-17 06:23:38.565+0000 [id=27] WARNING i.j.p.casc.BaseConfigurator#createAttribute: Can't handle class jenkins.plugins.git.GitSCMSource#owner: type is abstract but not Describable.
2020-02-17 06:23:38.569+0000 [id=27] WARNING i.j.p.casc.BaseConfigurator#createAttribute: Can't handle class jenkins.plugins.git.GitSCMSource#owner: type is abstract but not Describable.
2020-02-17 06:23:39.960+0000 [id=27] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed ConfigurationAsCode.init
io.jenkins.plugins.casc.ConfiguratorException: Invalid configuration elements for type class jenkins.model.GlobalConfigurationCategory$Unclassified : splunkJenkinsInstallation.
Available attributes : administrativeMonitorsConfiguration, alerter, ansiColorBuildWrapper, artifactManager, bitbucketEndpointConfiguration, buildStepOperation, casCGlobalConfig, cloud, contentFilters, defaultView, descriptionSetterWrapper, dropDownGlobalConfiguration, envInjectNodeProperty, experimentalPlugins, extendedEmailPublisher, findBugsReporter, findbugs, gitHubConfiguration, gitHubPluginConfig, gitSCM, globalConfigFiles, globalConfigurationImpl, globalDefaultFlowDurabilityLevel, globalLibraries, globalSettings, httpRequestGlobalConfig, jiraGlobalConfiguration, location, mailer, masterBuild, mavenModuleSet, metricsAccessKey, myView, naginatorPublisher, nodeProperties, openTasks, operationsCenterRootAction, pipeline-model, plugin, pollSCM, prioritySorterConfiguration, projectNamingStrategy, provisioningLimitConfiguration, quietPeriod, rebuildDescriptor, resourceRoot, scmRetryCount, sdlcBuildNotifierAdv, shell, simple-theme-plugin, singleConditionalBuilder, sonarGlobalConfiguration, tasksReporter, testResultsAnalyzerExtension, throttleJobProperty, timestamperConfig, usageStatistics, viewsTabBar, warningsParsers, warningsPlugin
at io.jenkins.plugins.casc.BaseConfigurator.handleUnknown(BaseConfigurator.java:377)
at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:366)
at io.jenkins.plugins.casc.BaseConfigurator.check(BaseConfigurator.java:287)
at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$checkWith$7(ConfigurationAsCode.java:746)
at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:696)
Caused: io.jenkins.plugins.casc.ConfiguratorException: unclassified: error configuring 'unclassified' with class io.jenkins.plugins.casc.impl.configurators.GlobalConfigurationCategoryConfigurator configurator
at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:702)
at io.jenkins.plugins.casc.ConfigurationAsCode.checkWith(ConfigurationAsCode.java:746)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:731)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:611)
at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:292)
at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:284)
Caused: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
Caused: java.lang.Error
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1121)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2020-02-17 06:23:39.962+0000 [id=20] SEVERE hudson.util.BootFailure#publish: Failed to initialize Jenkins
io.jenkins.plugins.casc.ConfiguratorException: Invalid configuration elements for type class jenkins.model.GlobalConfigurationCategory$Unclassified : splunkJenkinsInstallation.
Available attributes : administrativeMonitorsConfiguration, alerter, ansiColorBuildWrapper, artifactManager, bitbucketEndpointConfiguration, buildStepOperation, casCGlobalConfig, cloud, contentFilters, defaultView, descriptionSetterWrapper, dropDownGlobalConfiguration, envInjectNodeProperty, experimentalPlugins, extendedEmailPublisher, findBugsReporter, findbugs, gitHubConfiguration, gitHubPluginConfig, gitSCM, globalConfigFiles, globalConfigurationImpl, globalDefaultFlowDurabilityLevel, globalLibraries, globalSettings, httpRequestGlobalConfig, jiraGlobalConfiguration, location, mailer, masterBuild, mavenModuleSet, metricsAccessKey, myView, naginatorPublisher, nodeProperties, openTasks, operationsCenterRootAction, pipeline-model, plugin, pollSCM, prioritySorterConfiguration, projectNamingStrategy, provisioningLimitConfiguration, quietPeriod, rebuildDescriptor, resourceRoot, scmRetryCount, sdlcBuildNotifierAdv, shell, simple-theme-plugin, singleConditionalBuilder, sonarGlobalConfiguration, tasksReporter, testResultsAnalyzerExtension, throttleJobProperty, timestamperConfig, usageStatistics, viewsTabBar, warningsParsers, warningsPlugin
at io.jenkins.plugins.casc.BaseConfigurator.handleUnknown(BaseConfigurator.java:377)
at io.jenkins.plugins.casc.BaseConfigurator.configure(BaseConfigurator.java:366)
at io.jenkins.plugins.casc.BaseConfigurator.check(BaseConfigurator.java:287)
at io.jenkins.plugins.casc.ConfigurationAsCode.lambda$checkWith$7(ConfigurationAsCode.java:746)
at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:696)
Caused: io.jenkins.plugins.casc.ConfiguratorException: unclassified: error configuring 'unclassified' with class io.jenkins.plugins.casc.impl.configurators.GlobalConfigurationCategoryConfigurator configurator
at io.jenkins.plugins.casc.ConfigurationAsCode.invokeWith(ConfigurationAsCode.java:702)
at io.jenkins.plugins.casc.ConfigurationAsCode.checkWith(ConfigurationAsCode.java:746)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:731)
at io.jenkins.plugins.casc.ConfigurationAsCode.configureWith(ConfigurationAsCode.java:611)
at io.jenkins.plugins.casc.ConfigurationAsCode.configure(ConfigurationAsCode.java:292)
at io.jenkins.plugins.casc.ConfigurationAsCode.init(ConfigurationAsCode.java:284)
Caused: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
Caused: java.lang.Error
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1121)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused: org.jvnet.hudson.reactor.ReactorException
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282)
at jenkins.InitReactorRunner.run(InitReactorRunner.java:48)
at jenkins.model.Jenkins.executeReactor(Jenkins.java:1154)
at jenkins.model.Jenkins.<init>(Jenkins.java:965)
at hudson.model.Hudson.<init>(Hudson.java:85)
at hudson.model.Hudson.<init>(Hudson.java:81)
at hudson.WebAppMain$3.run(WebAppMain.java:233)
Caused: hudson.util.HudsonFailedToLoad
at hudson.WebAppMain$3.run(WebAppMain.java:250)
2020-02-17 06:23:40.038+0000 [id=20] INFO jenkins.model.Jenkins#cleanUp: Stopping Jenkins
2020-02-17 06:23:40.070+0000 [id=20] INFO jenkins.model.Jenkins$18#onAttained: Started termination
2020-02-17 06:23:40.139+0000 [id=20] SEVERE jenkins.model.Jenkins$18#onTaskFailed: Failed TrayGC.releaseMetadata
java.lang.AssertionError: com.cloudbees.opscenter.dumbwaiter.LocalDumbWaiter is not available until after all jobs are loaded
at com.cloudbees.opscenter.dumbwaiter.LocalDumbWaiter.getInstance(LocalDumbWaiter.java:138)
at com.cloudbees.opscenter.dumbwaiter.LocalDumbWaiter$TrayGC.releaseMetadata(LocalDumbWaiter.java:408)
Caused: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
Caused: java.lang.Error
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3434)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3434)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:276)
at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3431)
at jenkins.model.Jenkins.cleanUp(Jenkins.java:3352)
at hudson.WebAppMain$3.run(WebAppMain.java:254)
2020-02-17 06:23:40.140+0000 [id=20] SEVERE jenkins.model.Jenkins$18#onTaskFailed: Failed Messaging.beforeShutdown
java.lang.AssertionError: com.cloudbees.opscenter.context.Messaging is not available until after all jobs are loaded
at com.cloudbees.opscenter.context.Messaging.getInstance(Messaging.java:408)
at com.cloudbees.opscenter.context.Messaging.close(Messaging.java:481)
at com.cloudbees.opscenter.context.Messaging.beforeShutdown(Messaging.java:65)
Caused: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
Caused: java.lang.Error
at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3434)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
at jenkins.model.Jenkins$17.execute(Jenkins.java:3434)
at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:276)
at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3431)
at jenkins.model.Jenkins.cleanUp(Jenkins.java:3352)
at hudson.WebAppMain$3.run(WebAppMain.java:254)
2020-02-17 06:23:40.147+0000 [id=20] INFO c.c.j.p.k.KubernetesCredentialProvider#onClose: Secrets watcher stopped
2020-02-17 06:23:40.152+0000 [id=20] INFO jenkins.model.Jenkins$18#onAttained: Completed termination
2020-02-17 06:23:40.152+0000 [id=20] INFO jenkins.model.Jenkins#_cleanUpDisconnectComputers: Starting node disconnection
2020-02-17 06:23:40.154+0000 [id=20] INFO jenkins.model.Jenkins#_cleanUpShutdownPluginManager: Stopping plugin manager
2020-02-17 06:23:40.238+0000 [id=20] INFO jenkins.model.Jenkins#_cleanUpPersistQueue: Persisting build queue
2020-02-17 06:23:40.244+0000 [id=20] INFO jenkins.model.Jenkins#_cleanUpAwaitDisconnects: Waiting for node disconnection completion
2020-02-17 06:23:40.246+0000 [id=20] INFO jenkins.model.Jenkins#cleanUp: Jenkins stopped
-
Official comment
Hi Harikrishna,
The first error ( jenkins.model.GlobalConfigurationCategory$Unclassified : splunkJenkinsInstallation ) looks to come from Jenkins Config As Code which is failing to configure the instance and thus it dies:
2020-02-17 06:23:39.960+0000 [id=27] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed ConfigurationAsCode.init
io.jenkins.plugins.casc.ConfiguratorException: Invalid configuration elements for type class jenkins.model.GlobalConfigurationCategory$Unclassified : splunkJenkinsInstallation.
Available attributes : administrativeMonitorsConfiguration, alerter, ansiColorBuildWrapper, artifactManager, bitbucketEndpointConfiguration, buildStepOperation, casCGlobalConfig, cloud, contentFilters, defaultView, descriptionSetterWrapper, dropDownGlobalConfiguration, envInjectNodeProperty, experimentalPlugins, extendedEmailPublisher, findBugsReporter, findbugs, gitHubConfiguration, gitHubPluginConfig, gitSCM, globalConfigFiles, globalConfigurationImpl, globalDefaultFlowDurabilityLevel, globalLibraries, globalSettings, httpRequestGlobalConfig, jiraGlobalConfiguration, location, mailer, masterBuild, mavenModuleSet, metricsAccessKey, myView, naginatorPublisher, nodeProperties, openTasks, operationsCenterRootAction, pipeline-model, plugin, pollSCM, prioritySorterConfiguration, projectNamingStrategy, provisioningLimitConfiguration, quietPeriod, rebuildDescriptor, resourceRoot, scmRetryCount, sdlcBuildNotifierAdv, shell, simple-theme-plugin, singleConditionalBuilder, sonarGlobalConfiguration, tasksReporter, testResultsAnalyzerExtension, throttleJobProperty, timestamperConfig, usageStatistics, viewsTabBar, warningsParsers, warningsPluginThe best solution is to open a support case and we will assist you to fix it.
Best regards
-
Further to this, we have two master jenkins on this cloudbees core jenkins (kubernetes) as shown below, the master jenkins "teams-buildeng" dont have any issue when doing upgrade, however other master jenkins "teams-toolseng" failed with above exception stack (except tenant_name, domain_name etc, the config remains same, still it fails). Please let us know what caused this issue ?
1. teams-buildeng
2. teams-toolseng
PS : just upgraded to 2.204.2.x, still same issue persists.
0 -
Hi Arnaud,
To be clear, the config between these two jenkins master - teams-buildeng and teams-toolseng remains same including "splunkJenkinsInstallation", no issue is seen in teams-buildeng while issue is seen in other jenkins master -teams-toolseng. Any clue ?
0 -
it's probably a problem with a plugin installed in a different version or not installed in one server.
0
Please sign in to leave a comment.
Comments
4 comments