Weblogic Monitoring with Prometheus and Grafana
Since quite some time Oracle is providing in their GitHub Repository the weblogic-monitoring-exporter https://github.com/oracle/weblogic-monitoring-exporter.
This weblogic-monitoring-exporter is basically a war file, which uses the WebLogic Server RESTful Management API to collect defined metrics of your WebLogic Server and exports them in a Prometheus compatible format.
At first you will need to download the get[Version-Number].sh from the GitHub Repository https://github.com/oracle/weblogic-monitoring-exporter/releases. Next step is to create a YAML file in which you define which metrics you want to collect from your WebLogic Server.
Place both files in one directory and execute the get[Version-Number].sh with your YAML file as input parameter. The get[Version-Number].sh will simple download from GitHub the required war file, so make sure that your system on which you execute the script got internet connection, and merge your YAML file into the war file.
Example YAML file named exporter-config.yml
Now let's merge the exporter-config.yml into the wls-exporter.war
As next we can deploy the generated wls-exporter.war on your WebLogic Server, just connect with a browser to your WebLogic Console (http://<servername>:<AdminServerPort>/console)
Navigate under the Domain Structure to the Option Deployments, click the "Lock & Edit" button in the Change Center, if you are running your WebLogic Domain in Production Mode and click the "Install" button under the Configuration tab in the Deployments.
Now navigate to the location of the wls-exporter.war file and select it and hit the button "Next"
Make sure that the option "Install this deployment as an application" is marked and hit the button "Next"
Target the deployment of the wls-exporter.war to your Servers and hit the button "Next"
Leave the default and hit the button "Next"
Leave the default and hit the button "Next"
Click the button "Save" and finally click the button "Activate Changes" in the Change Center
After the successful activation you should see as follows
Now we are done with the deployment of the wls-exporter.war and we can start the application, just go to the tab "Control" and click under "Start" the option "Servicing all requests"
Confirm the startup of the application with "Yes"
After that you should see under the Control tab that the wls-exporter application is now in state Active
The next step is to configure your Prometheus environment to collect the metrics from the wls-exporter application. Just add under the scrape_configs section in your prometheus.yml configuration file the necessary entries for the WebLogic Servers.
Simply add under the scrape_config section in the prometheus.yml a job_name for each WebLogic Managed Server (including your AdminServer). If you are adding multiple WebLogic Servers, its advisable to use the metric_relabel_configs option, to prefix each metric collected by Prometheus with for example the name of the WebLogic Server:
fter this changes, just restart your Prometheus and check under the Prometheus Console - Targets the status of your job_name.
You can see that we have now 3 Endpoints, one for each WebLogic Server (one AdminServer and two Managed Servers).
Under the Prometheus Console - Graph you can see all your defined metrics for your WebLogic Servers.
As next you can create a Dashboard in your Grafana environment.
So with a little bit of work you can easily monitor your WebLogic Environment with Prometheus and Grafana. In addition you can configure thresholds for your metrics and trigger alerts either via the alertmanager from Prometheus or via Grafana.
One point you should take care of with the weblogic-monitoring-exporter is the usage of the configuration option "domainQualifier: true". As documented this configuration is the metrics with the domain name, but currently the option "domainQualifier: true" will cause a non-Prometheus-conform formatted metric.
As you can see from above listing, the wls-exporter is generating the first line wrong, as the metric value is a string and therefor it can not be parsed by Prometheus.
I have currently opened an issue in the GitHub Repository for the weblogic-monitoring-exporter and I hope to get soon a feedback and/or solution https://github.com/oracle/weblogic-monitoring-exporter/issues/82
But nevertheless, the weblogic-monitoring-exporter is a great add-on for the Oracle WebLogic Server in order to monitor your environment with Prometheus and visualise your metrics with Grafana.
UPDATE 2nd April 2020: the above mentioned problem with the non-conform Prometheus formatted metric is resolved by the Oracle Development Team for the WebLogic-Monitoring-Exporter, just get the release 1.1.2 (https://github.com/oracle/weblogic-monitoring-exporter/releases)
This weblogic-monitoring-exporter is basically a war file, which uses the WebLogic Server RESTful Management API to collect defined metrics of your WebLogic Server and exports them in a Prometheus compatible format.
At first you will need to download the get[Version-Number].sh from the GitHub Repository https://github.com/oracle/weblogic-monitoring-exporter/releases. Next step is to create a YAML file in which you define which metrics you want to collect from your WebLogic Server.
Place both files in one directory and execute the get[Version-Number].sh with your YAML file as input parameter. The get[Version-Number].sh will simple download from GitHub the required war file, so make sure that your system on which you execute the script got internet connection, and merge your YAML file into the war file.
Example YAML file named exporter-config.yml
metricsNameSnakeCase: true
queries:
- key: name
keyName: location
prefix: wls_server_
applicationRuntimes:
key: name
keyName: app
componentRuntimes:
prefix: wls_webapp_config_
type: WebAppComponentRuntime
key: name
values: [deploymentState, contextRoot, sourceInfo, sessionsOpenedTotalCount, openSessionsCurrentCount, openSessionsHighCount]
servlets:
prefix: wls_servlet_
key: servletName
- JVMRuntime:
prefix: wls_jvm_
key: name
- executeQueueRuntimes:
prefix: wls_socketmuxer_
key: name
values: [pendingRequestCurrentCount]
- workManagerRuntimes:
prefix: wls_workmanager_
key: name
values: [stuckThreadCount, pendingRequests, completedRequests]
- threadPoolRuntime:
prefix: wls_threadpool_
key: name
values: [executeThreadTotalCount, queueLength, stuckThreadCount, hoggingThreadCount]
- JMSRuntime:
key: name
keyName: jmsruntime
prefix: wls_jmsruntime_
JMSServers:
prefix: wls_jms_
key: name
keyName: jmsserver
destinations:
prefix: wls_jms_dest_
key: name
keyName: destination
- persistentStoreRuntimes:
prefix: wls_persistentstore_
key: name
- JDBCServiceRuntime:
JDBCDataSourceRuntimeMBeans:
prefix: wls_datasource_
key: name
- JTARuntime:
prefix: wls_jta_
key: name
Now let's merge the exporter-config.yml into the wls-exporter.war
#
./get1.1.1.sh exporter-config.yml
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 607 100 607 0 0 2872 0 --:--:-- --:--:-- --:--:-- 2876
100 2018k 100 2018k 0 0 1165k 0 0:00:01 0:00:01 --:--:-- 2247k
created /tmp/ci-21tVFvZiR4
/tmp/ci-21tVFvZiR4 /u00/app/oracle/install
in temp dir
adding: config.yml (deflated 64%)
/u00/app/oracle/install
As next we can deploy the generated wls-exporter.war on your WebLogic Server, just connect with a browser to your WebLogic Console (http://<servername>:<AdminServerPort>/console)
Navigate under the Domain Structure to the Option Deployments, click the "Lock & Edit" button in the Change Center, if you are running your WebLogic Domain in Production Mode and click the "Install" button under the Configuration tab in the Deployments.
Now navigate to the location of the wls-exporter.war file and select it and hit the button "Next"
Make sure that the option "Install this deployment as an application" is marked and hit the button "Next"
Target the deployment of the wls-exporter.war to your Servers and hit the button "Next"
Leave the default and hit the button "Next"
Leave the default and hit the button "Next"
Click the button "Save" and finally click the button "Activate Changes" in the Change Center
After the successful activation you should see as follows
Now we are done with the deployment of the wls-exporter.war and we can start the application, just go to the tab "Control" and click under "Start" the option "Servicing all requests"
Confirm the startup of the application with "Yes"
After that you should see under the Control tab that the wls-exporter application is now in state Active
The next step is to configure your Prometheus environment to collect the metrics from the wls-exporter application. Just add under the scrape_configs section in your prometheus.yml configuration file the necessary entries for the WebLogic Servers.
Simply add under the scrape_config section in the prometheus.yml a job_name for each WebLogic Managed Server (including your AdminServer). If you are adding multiple WebLogic Servers, its advisable to use the metric_relabel_configs option, to prefix each metric collected by Prometheus with for example the name of the WebLogic Server:
# Align the job_name with your settings
# Align the IP address and Ports for the targets with your settings
# Align the metric_relabel_configs with your settings
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_timeout: 10s
. . .
. . .
scrape_configs:
. . .
. . .
- job_name: 'WebLogicAdminServer'
scrape_interval: 10s
metrics_path: /wls-exporter/metrics
static_configs:
- targets: ['172.17.0.1:7001']
basic_auth:
username: weblogic
password: Oracle12c
metric_relabel_configs:
- source_labels: [__name__]
target_label: __name__
replacement: "DemoAdminServer_${1}"
- job_name: 'WebLogicDEMOMS1'
scrape_interval: 10s
metrics_path: /wls-exporter/metrics
static_configs:
- targets: ['172.17.0.1:7003']
basic_auth:
username: weblogic
password: Oracle12c
metric_relabel_configs:
- source_labels: [__name__]
target_label: __name__
replacement: "DEMOMS1_${1}"
- job_name: 'WebLogicDEMOMS2'
scrape_interval: 10s
metrics_path: /wls-exporter/metrics
static_configs:
- targets: ['172.17.0.1:7004']
basic_auth:
username: weblogic
password: Oracle12c
metric_relabel_configs:
- source_labels: [__name__]
target_label: __name__
replacement: "DEMOMS2_${1}"
. . .
. . .
fter this changes, just restart your Prometheus and check under the Prometheus Console - Targets the status of your job_name.
You can see that we have now 3 Endpoints, one for each WebLogic Server (one AdminServer and two Managed Servers).
Under the Prometheus Console - Graph you can see all your defined metrics for your WebLogic Servers.
As next you can create a Dashboard in your Grafana environment.
So with a little bit of work you can easily monitor your WebLogic Environment with Prometheus and Grafana. In addition you can configure thresholds for your metrics and trigger alerts either via the alertmanager from Prometheus or via Grafana.
One point you should take care of with the weblogic-monitoring-exporter is the usage of the configuration option "domainQualifier: true". As documented this configuration is the metrics with the domain name, but currently the option "domainQualifier: true" will cause a non-Prometheus-conform formatted metric.
name{domain="demo_domain"} demo_domain
DemoAdminServer_wls_server_activation_time{domain="demo_domain",location="DemoAdminServer"} 1581419617765
DemoAdminServer_wls_server_admin_server_listen_port{domain="demo_domain",location="DemoAdminServer"} 7001
DemoAdminServer_wls_server_open_sockets_current_count{domain="demo_domain",location="DemoAdminServer"} 4
DemoAdminServer_wls_server_state_val{domain="demo_domain",location="DemoAdminServer"} 2
As you can see from above listing, the wls-exporter is generating the first line wrong, as the metric value is a string and therefor it can not be parsed by Prometheus.
I have currently opened an issue in the GitHub Repository for the weblogic-monitoring-exporter and I hope to get soon a feedback and/or solution https://github.com/oracle/weblogic-monitoring-exporter/issues/82
But nevertheless, the weblogic-monitoring-exporter is a great add-on for the Oracle WebLogic Server in order to monitor your environment with Prometheus and visualise your metrics with Grafana.
UPDATE 2nd April 2020: the above mentioned problem with the non-conform Prometheus formatted metric is resolved by the Oracle Development Team for the WebLogic-Monitoring-Exporter, just get the release 1.1.2 (https://github.com/oracle/weblogic-monitoring-exporter/releases)
