Skip to main content
Version: Self Hosted Turbo

Monitor Java Application in Kubernetes

Overview

Java applications running in Kubernetes can be monitored in SnappyFlow using two approaches:

Java monitoring with sfKubeAgent

In this option, the Java application should be run with Jolokia agent and sfKubeAgent running as a sidecar container and fetches metrics via Jolokia port. Refer to sfKubeAgent Overview

Prerequisites

  • Copy Jolokia JAR into docker image

  • Run the java application with Jolokia JAR in docker image:

    -javaagent:/<path_jolokia_jar>/jolokia-jvm-<version>-agent.jar  

Configurations

Run sfKubeAgent with JVMJolokia plugin, which is specified using the config map shown below:

apiVersion: v1 
kind: ConfigMap
metadata:
name: jvm-configmap
data:
config.yaml: |-
key: <profile_key>
metrics:
plugins:
name: jvmjolokia
enabled: true
interval: 300 config:
ip: 127.0.0.1
protocol: http
port: <userDefinedJolokiaPort>
context: jolokia
monitorDeadlocks: false
deadLockMonitoringInterval: 300

The example illustrates instantiating sfKubeAgent with jvm-configmap. sfAKubeAgent talks to the Java application via userDefinedJolokiaPort (this example used 8778)

kind: Pod 
apiVersion: v1
metadata:
name: my-first-pod-1
labels:
snappyflow/appname: <app_name>
snappyflow/projectname: <project_name>
spec:
containers:
name: java-container
image: <docker_id>/<docker_image>:<tag> ports:
name: jolokiaport
containerPort: <userDefinedJolokiaPort>
Snappyflow's sfkubeagent container
name: java-sfagent
image: snappyflowml/sfagent:latest
imagePullPolicy: Always command:
/app/sfagent
-enable-console-log
env:
name: APP_NAME
value: <app_name>
name: PROJECT_NAME
value: <project_name>
volumeMounts:
name: configmap-jvm
mountPath: /opt/sfagent/config.yaml
subPath: config.yaml
volumes:
name: configmap-jvm configMap:
name: jvm-configmap

Viewing data and dashboards

  • Data generated by plugin can be viewed in browse data page inside the respective application under plugin=jvm_jolokia and documentType=jvm
  • Dashboard for this data can be instantiated by Importing dashboard template JVM to the application dashboard

Troubleshooting

  1. Check if the Jolokia port is accessible

    From inside the application container, run a curl command to the userDefinedJolokiaPort.

    curl http://localhost:<userDefinedJolokiaPort> 
  2. Check the logs in sfKubeAgent container for any errors

JVM Monitoring with Prometheus exporter

Refer to Prometheus Exporter Overview. Prometheus exporter is deployed as a sidecar container in the application pod and connects to the JMX target exposed by the application to scrape the metrics. sfPod polls Prometheus exporter to scrape the metrics.

Pre-requisites

sfPod can access Prometheus exporter at Service IP: userDefinedPrometheusPort

Configurations

  • Run Java application with JMX options:

    -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port= <userDefinedJMXPort> -Dcom.sun.management.jmxremote.authenticate=false - Dcom.sun.management.jmxremote.ssl=false 
  • Start the Prometheus exporter with

    java -jar jmx_prometheus_httpserver.jar <userDefinedPrometheusPort> <exporterConfigFile>

    Configurations are passed using config map:

    apiVersion: v1 
    kind: ConfigMap
    metadata:
    labels:
    snappyflow/appname: <app_name>
    snappyflow/projectname: <project_name>
    data:
    jmx-config.yaml: |
    ---
    jmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:<userDefinedJMXPort>/jmxrmi
    ssl: false
    rules:
    - pattern: '.*'
  • Prometheus exporter interfaces to JMX via userDefinedJMXPort. Example below uses 555S as the port.

  • Prometheus exporter exposes userDefinedPrometheusPort for scraping. Example uses 5556 as the port

  • Pod definition YAML that illustrates the configuration for Java application and exporter

    kind: Pod 
    apiVersion: v1
    metadata:
    name: my-first-pod
    labels:
    snappyflow/appname: <app_name>
    snappyflow/projectname: <project_name>
    spec:
    containers:
    name: app-container
    image: <docker_id>/<docker_image>:<tag> command:
    sh
    -c
    -x
    java -jar -Dcom.sun.management.jmxremote - Dcom.sun.management.jmxremote.port=<userDefinedJMXPort> - Dcom.sun.management.jmxremote.authenticate=false - Dcom.sun.management.jmxremote.ssl=false <application_jar>
    name: "exporter-container"
    image: "bitnami/jmx-exporter:latest" imagePullPolicy:
    command:
    sh
    -c
    -x
    java -jar jmx_prometheus_httpserver.jar <userDefinedPrometheusPort> /tmp/jmx-config.yaml
    ports:
    name: exporter-port
    containerPort: <userDefinedPrometheusPort>
    volumeMounts:
    name: configmap-jmx
    mountPath: /tmp
    volumes:
    name: configmap-jmx configMap:
    name: jmx-configmap

Viewing data and dashboards

  • Data generated by plugin can be viewed in “browse data” page inside the respective application under plugin=‘kube-prom-jmx‘ and documentType=‘jmxStats’
  • Dashboard for this data can be instantiated by Importing dashboard template “JVM” to the application dashboard.

Troubleshooting

  1. Check if the JMX port is accessible .From inside the application container, run a curl command to the userDefinedJMXPort.

    curl http://localhost:<userDefinedJMXPort> 
  2. Check if metrics are getting scraped. From inside the exporter container, run a curl command to the userDefinedPrometheusPort

    curl  http://localhost:<userDefinedPrometheusPort>/metrics