Flume can modify or drop events in-flight with the help of Interceptors, which can be attached to any Flume Source. Flume Solr UUIDInterceptor sets a universally unique identifier on all intercepted events. For example, UUID b5755073-77a9-43c1-8fad-b7a586fc1b97 represents a 128-bit value.

You can use UUIDInterceptor to automatically assign a UUID to a document event if no application-level unique key for the event is available. Assign UUIDs to events as soon as they enter the Flume network—that is, in the first Flume source of the flow. This enables deduplicating documents that may be accidentally duplicated as a result of replication and redelivery in a Flume network that is designed for high availability and high performance. If available, an application-level key is preferable to an auto-generated UUID because it enables subsequent updates and deletion of the document in Solr using that key.

Flume Solr UUIDInterceptor provides the following configuration options in the flume.conf file:

org.apache.flume.sink.solr.
morphline.UUIDInterceptor$Builder

For examples, see the BlobHandler and BlobDeserializer.