@InterfaceAudience.Public @InterfaceStability.Evolving public class KuduTableMapReduceUtil extends Object
Modifier and Type | Method and Description |
---|---|
static void |
addCredentialsToJob(KuduClient client,
org.apache.hadoop.mapreduce.Job job)
Export the credentials from a
KuduClient and store them in the given MapReduce
Job so that KuduClient s created from within tasks of that job can
authenticate to Kudu. |
static void |
addDependencyJars(org.apache.hadoop.conf.Configuration conf,
Class<?>... classes)
Add the jars containing the given classes to the job's configuration
such that JobClient will ship them to the cluster and add them to
the DistributedCache.
|
static void |
addDependencyJars(org.apache.hadoop.mapreduce.Job job)
Add the Kudu dependency jars as well as jars for any of the configured
job classes to the job configuration, so that JobClient will ship them
to the cluster and add them to the DistributedCache.
|
static void |
addKuduDependencyJars(org.apache.hadoop.conf.Configuration conf)
Add Kudu and its dependencies (only) to the job configuration.
|
static KuduTable |
getTableFromContext(org.apache.hadoop.mapreduce.TaskInputOutputContext context)
Use this method when setting up a task to get access to the KuduTable in order to create
Inserts, Updates, and Deletes.
|
static void |
importCredentialsFromCurrentSubject(KuduClient client)
Import credentials from the current thread's JAAS
Subject into the provided
KuduClient . |
public static KuduTable getTableFromContext(org.apache.hadoop.mapreduce.TaskInputOutputContext context)
context
- Map contextpublic static void addCredentialsToJob(KuduClient client, org.apache.hadoop.mapreduce.Job job) throws KuduException
KuduClient
and store them in the given MapReduce
Job
so that KuduClient
s created from within tasks of that job can
authenticate to Kudu.
This must be used before submitting a job when running against a Kudu cluster
configured to require authentication. If using KuduTableMapReduceUtil.TableInputFormatConfigurator
,
KuduTableMapReduceUtil.TableOutputFormatConfigurator
or another such utility class, this is called
automatically and does not need to be called.client
- the client whose credentials to exportjob
- the job to configureKuduException
- if credentials cannot be exportedpublic static void importCredentialsFromCurrentSubject(KuduClient client)
Subject
into the provided
KuduClient
.
This must be called for any clients created within a MapReduce job in order to
adopt the credentials added by addCredentialsToJob(KuduClient, Job)
.
When using KuduTableInputFormat
or KuduTableOutputFormat
, the
implementation automatically handles creating the client and importing necessary
credentials. As such, this is only necessary in jobs that explicitly create a
KuduClient
.
If no appropriate credentials are found, does nothing.public static void addDependencyJars(org.apache.hadoop.mapreduce.Job job) throws IOException
IOException
public static void addDependencyJars(org.apache.hadoop.conf.Configuration conf, Class<?>... classes) throws IOException
IOException
public static void addKuduDependencyJars(org.apache.hadoop.conf.Configuration conf) throws IOException
This is intended as a low-level API, facilitating code reuse between this class and its mapred counterpart. It also of use to external tools that need to build a MapReduce job that interacts with Kudu but want fine-grained control over the jars shipped to the cluster.
conf
- The Configuration object to extend with dependencies.IOException
KuduTableMapReduceUtil
,
PIG-3285