org.apache.kudu.flume.sink

Class AvroKuduOperationsProducer

java.lang.Object

org.apache.kudu.flume.sink.AvroKuduOperationsProducer

All Implemented Interfaces:

AutoCloseable, org.apache.flume.conf.Configurable, KuduOperationsProducer

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public class AvroKuduOperationsProducer
extends Object
implements KuduOperationsProducer

An Avro serializer that generates one operation per event by deserializing the event body as an Avro record and mapping its fields to columns in a Kudu table.

Avro Kudu Operations Producer configuration parameters

Property Name	Default	Required?	Description
producer.operation	upsert	No	The operation used to write events to Kudu. Supported operations are 'insert' and 'upsert'
producer.schemaPath		No	The location of the Avro schema file used to deserialize the Avro-encoded event bodies. It's used whenever an event does not include its own schema. If not specified, the schema must be specified on a per-event basis, either by url or as a literal. Schemas must be record type.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_OPERATION
static String	OPERATION_PROP
static String	SCHEMA_LITERAL_HEADER
static String	SCHEMA_PROP
static String	SCHEMA_URL_HEADER

Constructor Summary

Constructors

Constructor and Description
AvroKuduOperationsProducer()

Method Summary

Modifier and Type	Method and Description
void	close() Cleans up any state.
void	configure(org.apache.flume.Context context)
List<Operation>	getOperations(org.apache.flume.Event event) Returns the operations that should be written to Kudu as a result of this event.
void	initialize(KuduTable table) Initializes the operations producer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OPERATION_PROP

public static final String OPERATION_PROP

See Also:

Constant Field Values

SCHEMA_PROP

public static final String SCHEMA_PROP

See Also:

Constant Field Values

DEFAULT_OPERATION

public static final String DEFAULT_OPERATION

See Also:

Constant Field Values

SCHEMA_URL_HEADER

public static final String SCHEMA_URL_HEADER

See Also:

Constant Field Values

SCHEMA_LITERAL_HEADER

public static final String SCHEMA_LITERAL_HEADER

See Also:

Constant Field Values

Constructor Detail

AvroKuduOperationsProducer

public AvroKuduOperationsProducer()

Method Detail

configure

public void configure(org.apache.flume.Context context)

Specified by:

configure in interface org.apache.flume.conf.Configurable

initialize

public void initialize(KuduTable table)

Description copied from interface: KuduOperationsProducer

Initializes the operations producer. Called between configure and getOperations.

Specified by:

initialize in interface KuduOperationsProducer

Parameters:

table - the KuduTable used to create Kudu Operation objects

getOperations

public List<Operation> getOperations(org.apache.flume.Event event)
                              throws org.apache.flume.FlumeException

Description copied from interface: KuduOperationsProducer

Returns the operations that should be written to Kudu as a result of this event.

Specified by:

getOperations in interface KuduOperationsProducer

Parameters:

event - Event to convert to one or more Operations

Returns:

List of Operations that should be written to Kudu

Throws:

org.apache.flume.FlumeException

close

public void close()

Description copied from interface: KuduOperationsProducer

Cleans up any state. Called when the sink is stopped.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface KuduOperationsProducer