Flume can modify and drop events in-flight with the help of Interceptors, which can be attached to any Flume source. Flume MorphlineInterceptor runs the transformations of a morphline on intercepted events. For example the morphline can ignore events or alter or insert certain event headers using regular expression-based pattern matching, or it can auto-detect and set a MIME type using Apache Tika on events that are intercepted. This packet sniffing can be used for content-based routing in a Flume topology.

Flume supports multiplexing the event flow to destinations by defining a flow multiplexer that can replicate or selectively route an event to channels. This example shows a source from agent “foo” fanning out the flow to three different channels. This fan out can be replicating or multiplexing. In replicating, each event is sent to all three channels. In multiplexing, an event is delivered to a subset of available channels when that event's attribute matches a preconfigured value. For example, if an event attribute called stream.type is set to application/pdf, it goes to channel1 and channel3. If the attribute is set to avro/binary, it goes to channel2. If that channel is unavailable then an exception is thrown and the event is replayed later when the channel becomes available again. You can set the mapping in the flume.conf file.

Flume MorphlineInterceptor provides the following configuration options in the flume.conf file:

org.apache.flume.sink.solr.
morphline.MorphlineInterceptor$Builder

agent.sources.avroSrc.interceptors = morphlineinterceptor
agent.sources.avroSrc.interceptors.morphlineinterceptor.type = org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder
agent.sources.avroSrc.interceptors.morphlineinterceptor.morphlineFile = /etc/flume-ng/conf/morphline.conf
agent.sources.avroSrc.interceptors.morphlineinterceptor.morphlineId = morphline1

  Note: A morphline interceptor cannot generate more than one output record for each input event.