Spring Cloud Data Flow provides a Docker Compose file to let you quickly bring up Spring Cloud Data Flow, Skipper, and MySQL, Apache Kafka. Follow the customizing guide to switch to RabbitMQ or to a different database. Prometheus or InfluxDB, along with Grafana can be configured for monitoring the stream and task applications.
We recommended that you upgrade to the latest version of Docker before running the
docker-compose command. We have tested with Docker Engine version
To download the Spring Cloud Data Flow Server Docker Compose file, run the following command:
If wget is unavailable, you can use curl or another platform-specific utility. Click here to download the docker-compose.yml file.
The Docker Compose file starts instances of the following products:
- Spring Cloud Data Flow Server
- Spring Cloud Skipper Server
- Apache Kafka
In the directory where you downloaded
docker-compose.yml, start the system, by running the following commands:
export DATAFLOW_VERSION=2.3.0.BUILD-SNAPSHOT export SKIPPER_VERSION=2.2.0.BUILD-SNAPSHOT docker-compose up
If you use Windows, you can define environment variables by using the
command. To start the system on Windows, enter the following commands:
C:\ set DATAFLOW_VERSION=2.3.0.BUILD-SNAPSHOT C:\ set SKIPPER_VERSION=2.2.0.BUILD-SNAPSHOT C:\ docker-compose up
Spring Cloud Data Flow is ready for use once the
docker-compose command stops emitting log messages.
docker-compose.yml file defines
SKIPPER_VERSION variables, so that you can easily change those values.
The preceding commands first set the
SKIPPER_VERSION to use and then start
You can also use a shorthand version that exposes only the
SKIPPER_VERSION variables to the
docker-compose process (rather than setting it in the environment), as follows:
DATAFLOW_VERSION=2.3.0.BUILD-SNAPSHOT SKIPPER_VERSION=2.2.0.BUILD-SNAPSHOT docker-compose up
By default, the latest GA releases of Stream and Task applications are imported automatically. Use the
TASK_APPS_URI to alter the default imported Stream and Task applications. For example to override the default maven apps (https://dataflow.spring.io/kafka-maven-latest) with docker based distribution (https://dataflow.spring.io/kafka-docker-latest) run:
STREAM_APPS_URI=https://dataflow.spring.io/kafka-docker-latest docker-compose up
By default, Docker Compose uses locally available images. For example, when using the
latest tag, you can run
docker-compose pull prior to
docker-compose up to ensure the latest image is downloaded.
Some stream applications may open a port (for example,
http --server.port=). By default, a port range of
9000-9010 is exposed from the container to the host. If you need to change this range, you can modify the
ports block of the
dataflow-server service in the
Now that docker compose is up, you can access the Spring Cloud Data Flow Dashboard. In your browser, navigate to the Spring Cloud Data Flow Dashboard URL.
When you want to shut down Spring Cloud Data Flow, you can use the
docker-compose down command. To do so:
Open a new terminal window.
Change directory to the directory in which you started (where the
docker-compose.ymlfile is located).
Run the following command:
DATAFLOW_VERSION=2.3.0.BUILD-SNAPSHOT SKIPPER_VERSION=2.2.0.BUILD-SNAPSHOT docker-compose down
You need to specify the
DATAFLOW_VERSION and the
SKIPPER_VERSION environment variables if you are running the command in a separate terminal window from which you started Docker Compose. The
export commands you used earlier set the variables for only that terminal window, so those values are not found in the new terminal window.
If all else fails, you can shut down Docker Compose by pressing
For convenience and as an alternative to using the Spring Cloud Data Flow Dashboard, you can use the Spring Cloud Data Flow Shell as an alternative to the UI. The shell supports tab completion for commands and stream and task application configuration properties.
If you have started Spring Cloud Data Flow by using Docker Compose, the shell is also included in the
springcloud/spring-cloud-dataflow-server Docker image.
To use it, open another console window and type the following:
docker exec -it dataflow-server java -jar shell.jar
If you have started the Data Flow server with
java -jar, you can download and start the shell.
To download the Spring Cloud Data Flow Shell application, run the following command:
By default, the Data Flow docker-compose does not enable the monitoring functionality for Stream and Task applications. To enable that, follow the Monitoring with Prometheus and Grafana or Monitoring with InfluxDB and Grafana sections for how to configure Prometheus or InfluxDB based monitoring set up for Spring Cloud Data Flow.
To learn more about the monitoring experience in Spring Cloud Data Flow with Prometheus and InfluxDB, see the Stream Monitoring feature guide.