Pre-packaged Applications

This page contains information about the latest 3.x versions (release 2020.0.0 +) of stream-applications, which introduces many new features and some changes that are incompatible with the 2.x (Einstein) release. If you are looking for information about the 2.x release, see pre-packaged applications 2.x.

The Spring team provides and supports a selection of pre-packaged applications that you can use to assemble various data integration and processing pipelines and to support Spring Cloud Data Flow development, learning, and experimentation.

Getting Started

All pre-packaged streaming applications:

  • Are available as Apache Maven artifacts or Docker images.
  • Use RabbitMQ or Apache Kafka.
  • Support monitoring through Prometheus and InfluxDB.
  • Contain metadata for application properties used in the UI and code completion in the shell.

You can register stream and task applications by using the Data Flow UI or the shell.

You can register applications individually by using the app register command or in bulk by using the app import command.

For streams, depending on whether you use Kafka or RabbitMQ, you can register the applications by using their respective URLs:

Kafka

RabbitMQ

When you use the Data Flow UI, the links shown in the following image are included for pre-fill:

Bulk register applications

From the Data Flow Shell, you can bulk import and register the applications, as the following example shows:

dataflow:>app import --uri https://dataflow.spring.io/kafka-maven-milestone

Stream Applications

The Spring team develops and maintains stream applications and publishes these applications to the Spring public Maven repository and to Dockerhub in accordance with a release schedule, normally following significant Spring Boot or Spring Cloud Stream releases. The pre-packaged stream applications are Spring Boot executable jars that are built with a specific binder implementation. For each stream app, we provide separate executable applications for RabbitMQ and Kafka.

The following table shows the currently available stream applications:

Source Processor Sink
cdc-debezium aggregator analytics
file bridge cassandra
ftp filter dataflow-tasklauncher
geode groovy elasticsearch
http header-enricher file
jdbc http-request ftp
jms image-recognition geode
load-generator object-detection jdbc
mail semantic-segmentation log
mongodb script mongodb
mqtt splitter mqtt
rabbit transform pgcopy
s3 twitter-trend rabbit
sftp redis
syslog router
tcp rsocket
time s3
twitter-message sftp
twitter-search tcp
twitter-stream throughput
websocket twitter-message
zeromq twitter-update
wavefront
websocket
zeromq

Bulk Registration of Stream Applications

Spring Cloud Data Flow supports bulk registration of applications, through a standard properties file format. For convenience, we publish static properties files with application URIs (for either maven or docker) for all the out-of-the-box stream and task and batch apps. You can use these files in Spring Cloud Data Flow to register all the application URIs in bulk. Alternately, you can register them individually or provide your own custom property file with only the required application URIs in it. Bulk registration is convenient for getting started with SCDF. To reduce clutter, we recommend maintaining a "focused" list of desired application URIs in a custom property file.

Supported Spring Cloud Stream Applications

Artifact Type Latest Stable Release SNAPSHOT Release
spring-cloud-stream 3.1.x
spring-boot 2.5.x
RabbitMQ + Maven https://dataflow.spring.io/rabbitmq-maven-latest https://dataflow.spring.io/rabbitmq-maven-latest-snapshot
RabbitMQ + Docker (Docker Hub) https://dataflow.spring.io/rabbitmq-docker-latest https://dataflow.spring.io/rabbitmq-docker-latest-snapshot
RabbitMQ + Docker (Harbor Registry) https://dataflow.spring.io/rabbitmq-harbor-latest N/A
Kafka + Maven https://dataflow.spring.io/kafka-maven-latest https://dataflow.spring.io/kafka-maven-latest-snapshot
Kafka + Docker https://dataflow.spring.io/kafka-docker-latest https://dataflow.spring.io/kafka-docker-latest-snapshot
Kafka + Docker (Harbor Registry) https://dataflow.spring.io/kafka-harbor-latest N/A

Supported Spring Cloud Task and Batch Applications

Artifact Type Latest Stable Release SNAPSHOT Release
spring-cloud-stream 2.1.x
spring-boot 2.1.x
Maven https://dataflow.spring.io/task-maven-latest https://dataflow.spring.io/Elston-BUILD-SNAPSHOT-task-applications-maven
Docker https://dataflow.spring.io/task-docker-latest https://dataflow.spring.io/Elston-BUILD-SNAPSHOT-task-applications-docker

The following table lists previous releases for reference only. NOTE: These may no longer be supported (i.e. they depend on an EOL spring-boot release):

Previous Releases of Stream Applications (2020)

Artifact Type Latest Stable Release SNAPSHOT Release
spring-cloud-stream 3.0.x
spring-boot 2.3.x
RabbitMQ + Maven https://dataflow.spring.io/rabbitmq-maven-2020 %stream-app-rabbit-maven-2020-snapshot"%
RabbitMQ + Docker (Docker Hub) https://dataflow.spring.io/rabbitmq-docker-2020 https://dataflow.spring.io/rabbitmq-docker-2020-snapshot
RabbitMQ + Docker (Harbor Registry) https://dataflow.spring.io/rabbitmq-harbor-2020 N/A
Kafka + Maven https://dataflow.spring.io/kafka-maven-2020 https://dataflow.spring.io/kafka-maven-2020-snapshot
Kafka + Docker https://dataflow.spring.io/kafka-docker-2020 https://dataflow.spring.io/kafka-docker-2020-snapshot
Kafka + Docker (Harbor Registry) https://dataflow.spring.io/kafka-harbor-2020 N/A

Previous Releases of Stream Applications (Einstein)

Artifact Type Previous Release
spring-cloud-stream 2.0.x
spring-boot 2.0.x
RabbitMQ + Maven https://dataflow.spring.io/rabbitmq-maven-einstein
RabbitMQ + Docker https://dataflow.spring.io/rabbitmq-docker-einstein
Kafka + Maven https://dataflow.spring.io/kafka-maven-einstein
Kafka + Docker https://dataflow.spring.io/kafka-docker-einstein

Previous Releases of Spring Cloud Task and Batch Applications

Artifact Type Previous Release
spring-cloud-stream 2.0.x
spring-boot 2.0.x
Maven https://dataflow.spring.io/Dearborn-SR1-task-applications-maven
Docker https://dataflow.spring.io/Dearborn-SR1-task-applications-docker