Named Destinations

Instead of referencing a source or sink application, you can use a named destination. A named destination corresponds to a specific destination name in the middleware broker (RabbitMQ, Kafka, and others). When using the | symbol, applications are connected to each other with messaging middleware destination names created by the Data Flow server. In keeping with the Unix analogy, one can redirect standard input and output using the less-than (<) and greater-than (>) characters. To specify the name of the destination, prefix it with a colon (:). For example, the following stream has the destination name in the source position:

stream create --definition ":myDestination > log" --name ingest_from_broker --deploy

This stream receives messages from the destination called myDestination, located at the broker, and connects it to the log app. You can also create additional streams that consume data from the same named destination.

The following stream has the destination name in the sink position:

stream create --definition "http > :myDestination" --name ingest_to_broker --deploy

It is also possible to connect two different destinations (source and sink positions) at the broker in a stream, as shown in the following example:

stream create --definition ":destination1 > :destination2" --name bridge_destinations --deploy

In the preceding stream, both the destinations (destination1 and destination2) are located in the broker. The messages flow from the source destination to the sink destination over a bridge app that connects them.