Batch Job Scheduling

In Batch Developers Guide we showed you how to build batch applications and launch them in an ad-hoc fashion.
But typically batch jobs are launched at as scheduled time or based on some event. In this section, we will show you how you can schedule the launch of the batch jobs using Spring Cloud Data Flow.

Spring Cloud Data Flow Scheduling Overview

Spring Cloud Data Flow lets a user schedule the launching of tasks via a cron expression. A schedule can be created through the RESTful API or the Spring Cloud Data Flow UI. Spring Cloud Data Flow will schedule the execution of its tasks via a scheduling agent that is available on the cloud platform. Scheduling Architecture When using the Cloud Foundry platform Spring Cloud Data Flow will use the PCF Scheduler. When using Kubernetes, CronJobs will be used.

Scheduling a Batch Job

Make sure you have registered your timestamp application and created a definition for it as discussed in the Getting Started Guide. So using the UI we will schedule the timestamp-task by pressing the drop down button and selecting the "Schedule" option as highlighted below: Create Schedule

So for this scenario we want to launch this application once a minute. To do this fill out the Schedule setup page as follows:

In the sample below we set the name of the schedule to timestamp-task-once-a-minute and the cron expression to */1 * ? * *. Notice that the cron expression is expressed in the format used by Quartz. Also notice we can add command line arguments and deployment parameters for this schedule, but we will not do so for this example. Once you have entered the cron expression, press the CREATE SCHEDULE(S) button. Now PCF Scheduler will handle the scheduled launches for your batch application.

Schedule Batch App Cloud Foundry

In the sample below we set the name of the schedule to timestamp-task-once-a-minute and the cron expression to */1 * * * *. Notice we can add command line arguments and deployment parameters for this schedule, but we will not do so for this example. Now press the CREATE SCHEDULE(S) button. Now a Cron Job has been created to handle the scheduled launches for your batch application.

Schedule Batch App Kubernetes

Spring Cloud Data Flow does not offer an out of the box solution for scheduling task launches on the local platform.
However, there are at least two Spring Boot native solutions that provide a scheduling option, which can be custom implemented to make it work in SCDF running locally.

Spring Boot Implementing a Quartz Scheduler

One option is to create a Spring Boot application that utilizes the Quartz scheduler to execute RESTful API calls to launch tasks on Spring Cloud Data Flow. More information can be read about it here.

Spring Boot Implementing the @Scheduled annotation

Another option is to create a Spring Boot application that utilizes the @Scheduled annotation on a method that executes RESTful API calls to launch tasks on Spring Cloud Data Flow. More information can be read about it here.

When creating a task schedule a user can specify command line arguments and deployment properties for the scheduled application in the same way as they are specified when launching a task. More can be read about specifying deployment properties here.

Monitoring Task Launches

The status of each scheduled launch can be viewed on Spring Cloud Data Flow's Tasks executions tab.

SCDF Scheduled Executions

Deleting a Schedule

From the Schedules tab, press the drop down button next to the schedule you want to remove and select the "Destroy" option as shown below: Delete Schedule

When the confirmation dialog appears, press the DELETE THE SCHEDULE button. SCDF Confirm Schedule Delete