Stream Monitoring with Prometheus, Wavefront and InfluxDB

This section describes how to monitor the applications that were deployed as part of a Stream data pipeline. The setup for each platform is different, but the general architecture is the same across all the platforms.

The Data Flow metrics architecture is designed around the Micrometer library, to provide a simple facade over the instrumentation clients for the most popular monitoring systems. See the Micrometer documentation for the list of supported monitoring systems. Micrometer powers the delivery of application metrics from Spring Boot. Spring Integration provides additional micrometer integration to expose metrics around message rates and errors, which is critical to the monitoring of deployed streams.

All Spring Cloud Stream App Starters and Stream Applications are pre-configured to support three of the most popular monitoring systems: Prometheus, Wavefront, and InfluxDB. You can declaratively select which monitoring system the deployed applications use.

To enable support for different monitoring systems, you can customize the Stream Applications to use a different monitoring system.

To help you get started monitoring streams, Data Flow provides Grafana dashboards (for Prometheus and InfluxDB) that you can install and customize for your needs. For Wavefront, you can use the Data Flow Integration tile for a rich and comprehensive metrics visualization.

The following image shows the general architecture of how streaming applications are monitored:

Additionally, for streaming data pipelines based on Kafka binder, a dedicated Kafka and Kafka Stream dashboard is provided based on the Apache Kafka metrics:

Data Flow Metric Tags

To allow aggregating metrics per application type and per instance or per stream, the Spring Cloud Stream Application Starters are configured to use the following Micrometer tags:

• stream.name: The name of the stream that contains the applications that send the metrics
• application.name: The name or label of the application that reports the metrics
• application.type: The type (source, processor, or sink) of the application that reports the metrics
• application.guid: The unique instance identifier of the application instance that reports the metrics
• application.index: The application instance ID (when available)

Data Flow Metric Buttons

If the Data Flow server is started with the spring.cloud.dataflow.metrics.dashboard.url property pointing to your Grafana or Wavefront URL, the Dashboard feature is enabled and the Data Flow UI provides you with Grafana or Wavefront buttons that can open a particular dashboard for a given stream, application, or application instance. The following screenshots illustrate these buttons (look for the spiral icon):

Choosing between Wavefront, Prometheus, or InfluxDB and installing the necessary components differs, depending on the platform on which you run. Links to installation instructions are provided in each section below.

Local

This section describes how to view application metrics for streams that use Prometheus or InfluxDB as the metrics store on your local machine. Wavefront is a cloud offering, but you still can deploy Data Flow locally and point it to a cloud-managed Wavefront monitoring system.

dataflow:>app import --uri https://dataflow.spring.io/kafka-maven-latest --force
dataflow:>stream create stream2 --definition "time --fixed-delay=10 --time-unit=MILLISECONDS | filter --expression=payload.contains('3') | log" --deploy

dataflow:>app import --uri https://dataflow.spring.io/kafka-maven-latest --force

dataflow:>stream create stream2 --definition "time --fixed-delay=10 --time-unit=MILLISECONDS | filter --expression=payload.contains('3') | log" --deploy

docker exec -it influxdb /bin/sh
docker exec -it grafana /bin/bash

root:/# influx
> show databases
> use myinfluxdb
> show measurements
> select * from spring_integration_send limit 10

Kubernetes

This section describes how to view application metrics for streams that use Prometheus as the metrics store on Kubernetes.

minikube service --url grafana
http://192.168.99.100:31595

dataflow:>stream create metricstest --definition "time --fixed-delay=10 --time-unit=MILLISECONDS | filter --expression=payload.contains('3') | log" --deploy

management:
  metrics:
    export:
      wavefront:
        enabled: true
        api-token: <YOUR API-KEY>
        uri: <YOUR WAVEFRONT-URI>
        source: demo-scdf-source

Cloud Foundry

This section describes how to view application metrics for streams that use Prometheus as the metrics store on Cloud Foundry.

dataflow:>stream create metricstest --definition "time --fixed-delay=10 --time-unit=MILLISECONDS | filter --expression=payload.contains('3') | log" --deploy