org.apache.kudu.flume.sink

Class SimpleKeyedKuduOperationsProducer

java.lang.Object

org.apache.kudu.flume.sink.SimpleKeyedKuduOperationsProducer

All Implemented Interfaces:

AutoCloseable, org.apache.flume.conf.Configurable, KuduOperationsProducer

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public class SimpleKeyedKuduOperationsProducer
extends Object
implements KuduOperationsProducer

A simple serializer that generates one Insert or Upsert per Event by writing the event body into a BINARY column. The pair (key column name, key column value) should be a header in the Event; the column name is configurable but the column type must be STRING. Multiple key columns are not supported.

Simple Keyed Kudu Operations Producer configuration parameters

Property Name	Default	Required?	Description
producer.payloadColumn	payload	No	The name of the BINARY column to write the Flume event body to.
producer.keyColumn	key	No	The name of the STRING key column of the target Kudu table.
producer.operation	upsert	No	The operation used to write events to Kudu. Supported operations are 'insert' and 'upsert'

Field Summary

Fields

Modifier and Type	Field and Description
static String	KEY_COLUMN_DEFAULT
static String	KEY_COLUMN_PROP
static String	OPERATION_DEFAULT
static String	OPERATION_PROP
static String	PAYLOAD_COLUMN_DEFAULT
static String	PAYLOAD_COLUMN_PROP

Constructor Summary

Constructors

Constructor and Description
SimpleKeyedKuduOperationsProducer()

Method Summary

Modifier and Type	Method and Description
void	close() Cleans up any state.
void	configure(org.apache.flume.Context context)
List<Operation>	getOperations(org.apache.flume.Event event) Returns the operations that should be written to Kudu as a result of this event.
void	initialize(KuduTable table) Initializes the operations producer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PAYLOAD_COLUMN_PROP

public static final String PAYLOAD_COLUMN_PROP

See Also:

Constant Field Values

PAYLOAD_COLUMN_DEFAULT

public static final String PAYLOAD_COLUMN_DEFAULT

See Also:

Constant Field Values

KEY_COLUMN_PROP

public static final String KEY_COLUMN_PROP

See Also:

Constant Field Values

KEY_COLUMN_DEFAULT

public static final String KEY_COLUMN_DEFAULT

See Also:

Constant Field Values

OPERATION_PROP

public static final String OPERATION_PROP

See Also:

Constant Field Values

OPERATION_DEFAULT

public static final String OPERATION_DEFAULT

See Also:

Constant Field Values

Constructor Detail

SimpleKeyedKuduOperationsProducer

public SimpleKeyedKuduOperationsProducer()

Method Detail

configure

public void configure(org.apache.flume.Context context)

Specified by:

configure in interface org.apache.flume.conf.Configurable

initialize

public void initialize(KuduTable table)

Description copied from interface: KuduOperationsProducer

Initializes the operations producer. Called between configure and getOperations.

Specified by:

initialize in interface KuduOperationsProducer

Parameters:

table - the KuduTable used to create Kudu Operation objects

getOperations

public List<Operation> getOperations(org.apache.flume.Event event)
                              throws org.apache.flume.FlumeException

Description copied from interface: KuduOperationsProducer

Returns the operations that should be written to Kudu as a result of this event.

Specified by:

getOperations in interface KuduOperationsProducer

Parameters:

event - Event to convert to one or more Operations

Returns:

List of Operations that should be written to Kudu

Throws:

org.apache.flume.FlumeException

close

public void close()

Description copied from interface: KuduOperationsProducer

Cleans up any state. Called when the sink is stopped.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface KuduOperationsProducer