Create and launch a Composed Task using Data Flow
In the previous sections, we used two applications:
billsetuptaskconfigures the database for the
billrunreads usage information from a file and generates a billing statement record for each entry from a usage file.
Previously, we ran the
billsetuptask application and then the
billrun application in sequence manually.
This certainly works for the small case where we want to run two applications back to back and where the first app runs in a few seconds.
But what about the cases where there are five, ten, twenty or even more applications that need to be run in a sequence and the run time of some applications may be hours?
Also, what do we do in cases where one of the apps returns a non-zero exit code (that is, it ends with an error of some kind)? Spring Cloud Data Flow provides a solution for this issue through the use of composed tasks.
A composed task is a directed graph where each node of the graph is a task application. For the two applications (
billrun) that we have used in the previous examples, the graph would look something like this:
In this section, we show how to create a composed task by using Spring Cloud Data Flow's UI and then launch the composed task to Cloud Foundry, Kubernetes, and your local machine.
Before you start on this sample, you need to:
- Install Spring Cloud Data Flow.
- Install the Spring Cloud Task project that we use in this project.
Data Flow Installation
You must you have installed Spring Cloud Data Flow to one of the following platforms:
Registering the Billing Applications
For this guide, you need to have registered the billsetuptask and the billrun applications in Spring Cloud Data Flow.
Creating a Composed Task
To create a composed task:
- Select Tasks from the left navigation bar.
- Select CREATE TASK. The task creation view appears:
This displays a graphical editor that you can use to compose tasks.
The initial canvas contains
ENDnodes. To the left of the canvas, you can see the available task applications, including the
- Drag the
billsetuptaskapplication to the canvas.
- Connect the START node to the
- Drag the
billrunapplication to the canvas.
- Connect the
billsetuptaskapplication to the
- Connect the
billrunapplication to the END node to complete the composed task definition. In this case, the composed task definition consists of two task apps (
billrun). If there are application-defined configuration properties, we would set them here.
- Click CREATE TASK.
This prompts you to name the task definition, which is the logical name for the runtime configuration we want to deploy.
In this case, we use the same name as the task application, which is
ct-statement. The task confirmation view appears:
- Click Create the task. Doing so displays the main Tasks view.
TIP: To see all the things that you can do with the Composed Task Graph, see the Composed Task section of the reference documentation.
Launching the Task
Once we return to the task list (shown in the following image), we can see that we now have three task definitions:
We see that the following three task definitions were created when we created the ct-statement task:
ct-statement: This definition represents the composed task runner that runs the tasks in the composed task graph.
ct-statement-billsetuptask: This definition represents the
billsetuptaskapp that is run.
ct-statement-billrun: This definition represents the
billrunapp that is to be run.
Now we can launch
ct-statement by clicking the drop down on the left side of the
ct-statement row and select the Launch option, as follows:
Doing so takes you to a form where you can add command line arguments and deployment parameters, but we do not need any for this composed task.
Click the LAUNCH THE TASK button.
Doing so runs the composed task runner on the Data Flow server's task platform, which manages the execution of the composed task graph.
TIP: If a composed task needs to be run multiple times, you need to set the
increment-instance-enabled property to
true for each launch.
Verifying the Results
When the execution is complete, the status for each of the task definitions turns to a green color and shows
You can select the Task executions tab (shown in the following image) to view a summary of executions for this task.
Deleting a Composed Task
As shown earlier, a composed task creates a definition for each application in the graph. To delete all the definitions that were created as a part of the composed task, we can delete the composed task runner definition. To do so:
Click the drop down on the left side of the
ct-statementrow and select the Destroy option:
- A dialog asks you to
Confirm Destroy Task. Click the DESTROY THE TASK button. Doing so deletes the definitions that comprised the