org.kududb.mapreduce

Class KuduTableMapReduceUtil

java.lang.Object

org.kududb.mapreduce.KuduTableMapReduceUtil

@InterfaceAudience.Public
 @InterfaceStability.Evolving
public class KuduTableMapReduceUtil
extends Object

Utility class to setup MR jobs that use Kudu as an input and/or output.

Method Summary

Modifier and Type	Method and Description
static void	addDependencyJars(org.apache.hadoop.conf.Configuration conf, Class<?>... classes) Add the jars containing the given classes to the job's configuration such that JobClient will ship them to the cluster and add them to the DistributedCache.
static void	addDependencyJars(org.apache.hadoop.mapreduce.Job job) Add the Kudu dependency jars as well as jars for any of the configured job classes to the job configuration, so that JobClient will ship them to the cluster and add them to the DistributedCache.
static void	addKuduDependencyJars(org.apache.hadoop.conf.Configuration conf) Add Kudu and its dependencies (only) to the job configuration.
static KuduTable	getTableFromContext(org.apache.hadoop.mapreduce.TaskInputOutputContext context) Use this method when setting up a task to get access to the KuduTable in order to create Inserts, Updates, and Deletes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getTableFromContext

public static KuduTable getTableFromContext(org.apache.hadoop.mapreduce.TaskInputOutputContext context)

Use this method when setting up a task to get access to the KuduTable in order to create Inserts, Updates, and Deletes.

Parameters:

context - Map context

Returns:

The kudu table object as setup by the output format

addDependencyJars

public static void addDependencyJars(org.apache.hadoop.mapreduce.Job job)
                              throws IOException

Add the Kudu dependency jars as well as jars for any of the configured job classes to the job configuration, so that JobClient will ship them to the cluster and add them to the DistributedCache.

Throws:

IOException

addDependencyJars

public static void addDependencyJars(org.apache.hadoop.conf.Configuration conf,
                                     Class<?>... classes)
                              throws IOException

Add the jars containing the given classes to the job's configuration such that JobClient will ship them to the cluster and add them to the DistributedCache.

Throws:

IOException

addKuduDependencyJars

public static void addKuduDependencyJars(org.apache.hadoop.conf.Configuration conf)
                                  throws IOException

Add Kudu and its dependencies (only) to the job configuration.

This is intended as a low-level API, facilitating code reuse between this class and its mapred counterpart. It also of use to external tools that need to build a MapReduce job that interacts with Kudu but want fine-grained control over the jars shipped to the cluster.

Parameters:

conf - The Configuration object to extend with dependencies.

Throws:

IOException

See Also:

KuduTableMapReduceUtil, PIG-3285