Issue
- Best Practices for monitoring Jenkins instance performance.
- Which Application Performance Management (APM) software should I use for monitoring CloudBees CI?
Environment
- CloudBees CI (CloudBees Core) on modern cloud platforms - Managed controller
- CloudBees CI (CloudBees Core) on modern cloud platforms - Operations Center
- CloudBees CI (CloudBees Core) on traditional platforms - Client controller
- CloudBees CI (CloudBees Core) on traditional platforms - Operations Center
- CloudBees Jenkins Enterprise
- CloudBees Jenkins Enterprise - Managed controller
- CloudBees Jenkins Enterprise - Operations Center
- CloudBees Jenkins Platform - Client controller
- CloudBees Jenkins Platform - Operations Center
- Jenkins LTS
Resolution
CloudBees CI monitoring capabilities rely on exploding Jenkins Metrics plugin which exposes a set a metrics including: System and Java Virtual Machine metrics, Web UI metrics and Jenkins specific metrics. Other plugins might add additional metrics like the Metrics Disk Usage plugin.
Depending on where you explode those metrics from, the following options are available:
1. Inside the CloudBees CI
Using CloudBees Monitoring plugin (Proprietary) for alerting and Jenkins Monitoring plugin (Community) for reporting.
Reference to this blog post: Troubleshooting Jenkins Performance: Kubernetes Edition - Part 2 to extend information on this topic (The content of that blog post is also applicable for CloudBees CI Traditional)
2. Outside the CloudBees CI
2.1 CloudBees DevOptics
CloudBees DevOptics allows organizations to measure, analyze, and manage DevOps. CloudBees DevOptics provides a comprehensive view of the status of your CD platform as well as software value streams, correlated from data collected across software delivery pipelines. It acts as a single source of truth to analyze and continually optimize the flow of value from idea to production.
- Built-in Value Streams capabilities make it easy to map then actively measure and manage your software delivery process and activities.
- Industry standard DevOps metrics allow you to measure performance across teams to leverage best practices and enable continuous improvement.
- CD platform monitoring (Run Insights) capabilities allow you to track availability, usage and team engagement of your CD platforms to ensure they are optimally supporting your software delivery activities.
There is a DevOptics free version (check this comparison table) for Jenkins LTS.
2.2 Third-party alternatives
Depending on how you explode those metrics, the following options are available:
2.2.1 Enabling JMX
Java 8 Monitoring and Management Using JMX Technology
See this configuration example Monitoring CloudBees CI on Modern with Datadog | CloudBees Docs.
2.2.2 Tier 3 plugins
For monitoring Jenkins metrics and events:
- Datadog (push to service)
- Prometheus (pull from service). It is used in combination with Grafana for visualization which offers a list of built-in dashboards for Jenkins performance.
For monitoring Jenkins logs and build data:
Alternatively, custom data index can be created and pushed to your favortite APM tool like the following example: How to send build data to an Elasticsearch server.
Check support policies for Tier 3 plugins.
Note about JVM Profiling Agents
From our support experience, we have observed in certain scenarios that some External JVM Profiling Agents (it might depends on a specific versions of a agent) could create undesired side effects like System and network overhead.
On the other hand, the data collected by the Java agent will be stored in the Application heap. It is because this kind of agents run as a part of the JVM. While the actual overhead incurred depends on the specificity of the data gathered and the application itself, less than 5% performance overhead is an optimistic general estimate. This fact, needs to be considered when dimensioning nodes which run CloudBees CI application with JVM Agents.
0 Comments