org.kududb.flume.sink

Class KuduSink

java.lang.Object

org.apache.flume.sink.AbstractSink

org.kududb.flume.sink.KuduSink

All Implemented Interfaces:

org.apache.flume.conf.Configurable, org.apache.flume.lifecycle.LifecycleAware, org.apache.flume.NamedComponent, org.apache.flume.Sink

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public class KuduSink
extends org.apache.flume.sink.AbstractSink
implements org.apache.flume.conf.Configurable

A Flume sink that reads events from a channel and writes them to a Kudu table.

Flume Kudu Sink configuration parameters

Property Name	Default	Required?	Description
channel		Yes	The name of the Flume channel to read from.
type		Yes	Component name. Must be org.kududb.flume.sink.KuduSink
masterAddresses		Yes	Comma-separated list of "host:port" pairs of the Kudu master servers. The port is optional.
tableName		Yes	The name of the Kudu table to write to.
batchSize	100	No	The maximum number of events the sink will attempt to take from the channel per transaction.
ignoreDuplicateRows	true	No	Whether to ignore errors indicating that we attempted to insert duplicate rows into Kudu.
timeoutMillis	10000	No	Timeout period for Kudu write operations, in milliseconds.
producer	SimpleKuduEventProducer	No	The fully qualified class name of the KuduEventProducer the sink should use.
producer.*		(Varies by event producer)	Configuration properties to pass to the event producer implementation.

Installation

After building the sink, in order to use it with Flume, place the file named kudu-flume-sink-VERSION-jar-with-dependencies.jar in the Flume plugins.d directory under kudu-flume-sink/lib/.

For detailed instructions on using Flume's plugins.d mechanism, please see the plugins.d section of the Flume User Guide.

Constructor Summary

Constructors

Constructor and Description
KuduSink()
KuduSink(KuduClient kuduClient)

Method Summary

Modifier and Type	Method and Description
void	configure(org.apache.flume.Context context)
KuduClient	getClient()
org.apache.flume.Sink.Status	process()
void	start()
void	stop()

Methods inherited from class org.apache.flume.sink.AbstractSink

getChannel, getLifecycleState, getName, setChannel, setName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

KuduSink

public KuduSink()

KuduSink

@InterfaceAudience.Private
public KuduSink(KuduClient kuduClient)

Method Detail

start

public void start()

Specified by:

start in interface org.apache.flume.lifecycle.LifecycleAware

Overrides:

start in class org.apache.flume.sink.AbstractSink

stop

public void stop()

Specified by:

stop in interface org.apache.flume.lifecycle.LifecycleAware

Overrides:

stop in class org.apache.flume.sink.AbstractSink

configure

public void configure(org.apache.flume.Context context)

Specified by:

configure in interface org.apache.flume.conf.Configurable

getClient

public KuduClient getClient()

process

public org.apache.flume.Sink.Status process()
                                     throws org.apache.flume.EventDeliveryException

Specified by:

process in interface org.apache.flume.Sink

Throws:

org.apache.flume.EventDeliveryException