org.apache.kudu.mapreduce

Class KuduTableOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Operation>

org.apache.kudu.mapreduce.KuduTableOutputFormat

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public class KuduTableOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Operation>
implements org.apache.hadoop.conf.Configurable

Use KuduTableMapReduceUtil.TableOutputFormatConfigurator to correctly setup this output format, then KuduTableMapReduceUtil.getTableFromContext(org.apache.hadoop.mapreduce.TaskInputOutputContext) to get a KuduTable.

Hadoop doesn't have the concept of "closing" the output format so in order to release the resources we assume that once either checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext) or KuduTableOutputFormat.TableRecordWriter.close(org.apache.hadoop.mapreduce.TaskAttemptContext) have been called that the object won't be used again and the KuduClient is shut down.

Constructor Summary

Constructors

Constructor and Description
KuduTableOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext)
org.apache.hadoop.conf.Configuration	getConf()
static KuduTable	getKuduTable(String multitonKey)
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.NullWritable,Operation>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
void	setConf(org.apache.hadoop.conf.Configuration entries)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KuduTableOutputFormat

public KuduTableOutputFormat()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration entries)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getKuduTable

public static KuduTable getKuduTable(String multitonKey)

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.NullWritable,Operation> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                                                                                                      throws IOException,
                                                                                                             InterruptedException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Operation>

Throws:

IOException

InterruptedException

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext jobContext)
                      throws IOException,
                             InterruptedException

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Operation>

Throws:

IOException

InterruptedException

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                                                               throws IOException,
                                                                      InterruptedException

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.NullWritable,Operation>

Throws:

IOException

InterruptedException