@InterfaceAudience.Public @InterfaceStability.Unstable public class AsyncKuduClient extends Object implements AutoCloseable
A single Kudu client instance corresponds to a single remote Kudu cluster,
and can be used to read or write any number of tables within that cluster.
An application should use exactly one Kudu client instance per distinct Kudu
cluster it connects to.
In rare cases where a single application needs multiple instances connected
to the same cluster, or when many applications each using one or more Kudu
client instances are running on the same machine, it may be necessary to
adjust the instances to use less resources. See the options in
AsyncKuduClient.AsyncKuduClientBuilder
.
AsyncKuduClient
instance may
be created using the AsyncKuduClient.AsyncKuduClientBuilder
class. If
a synchronous API is preferred, KuduClient.KuduClientBuilder
may be
used instead. See the documentation on these classes for more details on
client configuration options.
Subject
instance, and associate the Subject with the current thread of execution.
The Kudu client then accesses the Kerberos credentials in the
Subject
and uses them to authenticate to the
remote cluster as necessary.
Kerberos credentials are typically obtained in one of two ways:
In the case of the Kudu client, Kudu will automatically look for credentials in the standard system-configured ticket cache location. No additional code needs to be written to enable this behavior.
Kudu will automatically detect if the ticket it has obtained from the ticket cache is about to expire. When that is the case, it will attempt to re-read the ticket cache to obtain a new ticket with a later expiration time. So, if an application needs to run for longer than the lifetime of a single ticket, the user must ensure that the ticket cache is periodically refreshed, for example by re-running 'kinit' once each day.
The Kudu client does not provide any utility code to facilitate logging in
from a keytab. Instead, applications should invoke the JAAS APIs directly,
and then ensure that the resulting Subject
instance is associated with the current thread's
AccessControlContext
when instantiating the Kudu client
instance for the first time. The Subject
instance
will be stored and used whenever Kerberos authentication is required.
Note: if the Kudu client is instantiated with a
Subject
as described above, it will not
make any attempt to re-login from the keytab. Instead, the application should
arrange to periodically re-initiate the login process and update the
credentials stored in the same Subject instance as was provided when the
client was instantiated.
The easiest way to authenticate using a keytab is by creating a JAAS config file such as this:
ExampleLoginContextName { com.sun.security.auth.module.Krb5LoginModule required useKeyTab = true keyTab = "/path/to/app.keytab" principal = "appuser"; };This can then be passed to the application by adding
-Djava.security.auth.login.config=/path/to/jaas.conf
to the command when
starting it.
This authentication method needs to be set in the code as well by wrapping
the code interacting with Kudu with a Subject.doAs(javax.security.auth.Subject, java.security.PrivilegedAction<T>)
after creating a login context using the
JAAS config, logging in, and passing the Subject
to the doAs:
LoginContext login = new LoginContext("ExampleLoginContextName"); login.login(); KuduClient c = Subject.doAs(login.getSubject(), (PrivilegedAction<KuduClient>) () -> { return myClientBuilder.build(); });In this case it's necessary to periodically re-login as needed and run doAs using the new subject.
In the context of the Hadoop ecosystem, the org.apache.hadoop.security.UserGroupInformation
class provides utility
methods to login from a keytab and then run code as the resulting Subject
:
UserGroupInformation.loginUserFromKeytab("appuser", "/path/to/app.keytab"); KuduClient c = UserGroupInformation.getLoginUser().doAs( new PrivilegedExceptionActionThe() { @Override public KuduClient run() throws Exception { return myClientBuilder.build(); } } );
UserGroupInformation
class will also automatically
start a thread to periodically re-login from the keytab. It's not necessary
to pass a JAAS config.
org.apache.kudu.client.SecurityContext
slf4j category. Enabling DEBUG
logging for this class may help you understand which credentials are being
obtained by the Kudu client when it is instantiated. Additionally, if the
Java system property kudu.jaas.debug
is set to true
, Kudu
will enable the debug
option when configuring Krb5LoginModule
when it attempts to log in from a ticket cache. JDK-specific system properties
such as sun.security.krb5.debug
may also be useful in troubleshooting
Kerberos authentication failures.
An authentication token is a time-limited credential which can be obtained by an application which has already authenticated via Kerberos. The token is represented by an opaque byte string, and it can be passed from one client to another to transfer credentials.
A token may be generated using the
exportAuthenticationCredentials()
API, and then
imported to another client using
importAuthenticationCredentials(byte[])
.
public
visibility, the
method is not part of the public API and there is no guarantee that its
existence or behavior will be maintained in subsequent versions of the Kudu
client library.
Other APIs are annotated with the InterfaceStability.Unstable annotation.
These APIs are meant for public consumption but may change between minor releases.
Note that the asynchronous client is currently considered unstable.
AsyncKuduSession
nor its synchronous wrapper KuduSession
is
thread-safe. Refer to the documentation for each individual class for more
details.
Deferred
instance to which you can attach a Callback
chain
that will execute when the asynchronous operation completes.
The asynchronous calls themselves typically do not throw exceptions. Instead,
an errback
should be attached which will be called with the Exception
that occurred.
Modifier and Type | Class and Description |
---|---|
static class |
AsyncKuduClient.AsyncKuduClientBuilder
Builder class to use in order to connect to Kudu.
|
Modifier and Type | Field and Description |
---|---|
static long |
DEFAULT_KEEP_ALIVE_PERIOD_MS |
static long |
DEFAULT_NEGOTIATION_TIMEOUT_MS |
static long |
DEFAULT_OPERATION_TIMEOUT_MS |
static byte[] |
EMPTY_ARRAY |
static long |
INVALID_TXN_ID |
static org.slf4j.Logger |
LOG |
static int |
NO_SOFT_DELETED_STATE_RESERVED_SECONDS |
static long |
NO_TIMESTAMP |
static int |
SLEEP_TIME |
Modifier and Type | Method and Description |
---|---|
com.stumbleupon.async.Deferred<AlterTableResponse> |
alterTable(String name,
AlterTableOptions ato)
Alter a table on the cluster as specified by the builder.
|
void |
close()
Invokes
shutdown() and waits. |
com.stumbleupon.async.Deferred<KuduTable> |
createTable(String name,
Schema schema,
CreateTableOptions builder)
Create a table on the cluster with the specified name, schema, and table configurations.
|
com.stumbleupon.async.Deferred<DeleteTableResponse> |
deleteTable(String name)
Delete a table with the specified name.
|
com.stumbleupon.async.Deferred<DeleteTableResponse> |
deleteTable(String name,
int reserveSeconds)
Delete a table with the specified name.
|
com.stumbleupon.async.Deferred<byte[]> |
exportAuthenticationCredentials()
Export serialized authentication data that may be passed to a different
client instance and imported to provide that client the ability to connect
to the cluster.
|
String |
getClusterId()
Returns the ID of the cluster that this client is connected to.
|
long |
getDefaultAdminOperationTimeoutMs()
Get the timeout used for admin operations.
|
long |
getDefaultOperationTimeoutMs()
Get the timeout used for operations on sessions and scanners.
|
long |
getDefaultSocketReadTimeoutMs()
Deprecated.
socket read timeouts are no longer used
|
long |
getLastPropagatedTimestamp()
Returns the last timestamp received from a server.
|
String |
getLocationString()
Returns a string representation of this client's location.
|
String |
getMasterAddressesAsString() |
com.stumbleupon.async.Deferred<ListTablesResponse> |
getSoftDeletedTablesList()
Get the list of all the soft deleted tables.
|
Statistics |
getStatistics()
Get the statistics object of this client.
|
com.stumbleupon.async.Deferred<ListTablesResponse> |
getTablesList()
Get the list of all the regular (i.e.
|
com.stumbleupon.async.Deferred<ListTablesResponse> |
getTablesList(String nameFilter)
Get a list of regular table names.
|
com.stumbleupon.async.Deferred<ListTablesResponse> |
getTablesList(String nameFilter,
boolean showSoftDeleted)
Get a list of table names.
|
com.stumbleupon.async.Deferred<KuduTableStatistics> |
getTableStatistics(String name)
Get table's statistics from master.
|
boolean |
hasLastPropagatedTimestamp()
Checks if the client received any timestamps from a server.
|
void |
importAuthenticationCredentials(byte[] authnData)
Import data allowing this client to authenticate to the cluster.
|
com.stumbleupon.async.Deferred<IsAlterTableDoneResponse> |
isAlterTableDone(String name)
Check whether a previously issued alterTable() is done.
|
com.stumbleupon.async.Deferred<IsCreateTableDoneResponse> |
isCreateTableDone(String name)
Check whether a previously issued createTable() is done.
|
boolean |
isStatisticsEnabled()
Check if statistics collection is enabled for this client.
|
void |
jwt(String jwt)
Set JWT (JSON Web Token) to authenticate the client to a server.
|
com.stumbleupon.async.Deferred<ListTabletServersResponse> |
listTabletServers()
Get the list of running tablet servers.
|
AsyncKuduScanner.AsyncKuduScannerBuilder |
newScannerBuilder(KuduTable table)
Creates a new
AsyncKuduScanner.AsyncKuduScannerBuilder for a particular table. |
AsyncKuduSession |
newSession()
Create a new session for interacting with the cluster.
|
com.stumbleupon.async.Deferred<KuduTable> |
openTable(String name)
Open the table with the given name.
|
com.stumbleupon.async.Deferred<RecallDeletedTableResponse> |
recallDeletedTable(String id)
Recall a soft-deleted table on the cluster with the specified id
|
com.stumbleupon.async.Deferred<RecallDeletedTableResponse> |
recallDeletedTable(String id,
String newTableName)
Recall a soft-deleted table on the cluster with the specified id
|
com.stumbleupon.async.Deferred<ArrayList<Void>> |
shutdown()
Performs a graceful shutdown of this instance.
|
KuduClient |
syncClient()
Returns a synchronous
KuduClient which wraps this asynchronous client. |
com.stumbleupon.async.Deferred<Boolean> |
tableExists(String name)
Test if a table exists.
|
void |
trustedCertificates(List<com.google.protobuf.ByteString> certificates)
Mark the given CA certificates (in DER format) as the trusted ones for the
client.
|
void |
updateLastPropagatedTimestamp(long lastPropagatedTimestamp)
Updates the last timestamp received from a server.
|
public static final org.slf4j.Logger LOG
public static final int SLEEP_TIME
public static final byte[] EMPTY_ARRAY
public static final long NO_TIMESTAMP
public static final long INVALID_TXN_ID
public static final long DEFAULT_OPERATION_TIMEOUT_MS
public static final int NO_SOFT_DELETED_STATE_RESERVED_SECONDS
public static final long DEFAULT_KEEP_ALIVE_PERIOD_MS
public static final long DEFAULT_NEGOTIATION_TIMEOUT_MS
public void updateLastPropagatedTimestamp(long lastPropagatedTimestamp)
lastPropagatedTimestamp
- the last timestamp received from a serverpublic long getLastPropagatedTimestamp()
public boolean hasLastPropagatedTimestamp()
public String getLocationString()
public String getClusterId()
public KuduClient syncClient()
KuduClient
which wraps this asynchronous client.
Calling KuduClient.close()
on the returned client will close this client.
If this asynchronous client should outlive the returned synchronous client,
then do not close the synchronous client.KuduClient
public com.stumbleupon.async.Deferred<KuduTable> createTable(String name, Schema schema, CreateTableOptions builder)
NonRecoverableException
name
- the table's nameschema
- the table's schemabuilder
- a builder containing the table's configurationspublic com.stumbleupon.async.Deferred<IsCreateTableDoneResponse> isCreateTableDone(String name)
name
- table's namepublic com.stumbleupon.async.Deferred<DeleteTableResponse> deleteTable(String name, int reserveSeconds)
name
- the table's namereserveSeconds
- the soft deleted table to be alive timepublic com.stumbleupon.async.Deferred<DeleteTableResponse> deleteTable(String name)
name
- the table's namepublic com.stumbleupon.async.Deferred<RecallDeletedTableResponse> recallDeletedTable(String id)
id
- the table's idpublic com.stumbleupon.async.Deferred<RecallDeletedTableResponse> recallDeletedTable(String id, String newTableName)
id
- the table's idnewTableName
- the table's new name after recallpublic com.stumbleupon.async.Deferred<AlterTableResponse> alterTable(String name, AlterTableOptions ato)
name
- the table's name (old name if the table is being renamed)ato
- the alter table optionspublic com.stumbleupon.async.Deferred<IsAlterTableDoneResponse> isAlterTableDone(String name)
name
- table namepublic com.stumbleupon.async.Deferred<ListTabletServersResponse> listTabletServers()
public com.stumbleupon.async.Deferred<ListTablesResponse> getTablesList()
public com.stumbleupon.async.Deferred<ListTablesResponse> getTablesList(String nameFilter)
nameFilter
- an optional table name filterpublic com.stumbleupon.async.Deferred<ListTablesResponse> getTablesList(String nameFilter, boolean showSoftDeleted)
nameFilter
- an optional table name filtershowSoftDeleted
- whether to display only regular (i.e. not soft deleted)
tables or all tables (i.e. soft deleted tables and regular tables)public com.stumbleupon.async.Deferred<ListTablesResponse> getSoftDeletedTablesList()
public com.stumbleupon.async.Deferred<KuduTableStatistics> getTableStatistics(String name)
name
- the table's namepublic com.stumbleupon.async.Deferred<Boolean> tableExists(String name)
name
- a non-null table namepublic com.stumbleupon.async.Deferred<KuduTable> openTable(String name)
name
- table to open@InterfaceStability.Unstable public com.stumbleupon.async.Deferred<byte[]> exportAuthenticationCredentials()
@InterfaceStability.Unstable public void importAuthenticationCredentials(byte[] authnData)
authnData
- then authentication data provided by a prior call to
exportAuthenticationCredentials()
@InterfaceStability.Unstable public void trustedCertificates(List<com.google.protobuf.ByteString> certificates) throws CertificateException
certificates
- list of certificates to trust (in DER format)CertificateException
- if any of the specified certificates were invalid@InterfaceStability.Unstable public void jwt(String jwt)
jwt
- The JSON web token to set.public long getDefaultOperationTimeoutMs()
public long getDefaultAdminOperationTimeoutMs()
@Deprecated public long getDefaultSocketReadTimeoutMs()
public String getMasterAddressesAsString()
public boolean isStatisticsEnabled()
public Statistics getStatistics()
IllegalStateException
- thrown if statistics collection has been disabledpublic AsyncKuduScanner.AsyncKuduScannerBuilder newScannerBuilder(KuduTable table)
AsyncKuduScanner.AsyncKuduScannerBuilder
for a particular table.table
- the name of the table you intend to scan.
The string is assumed to use the platform's default charset.public AsyncKuduSession newSession()
public void close() throws Exception
shutdown()
and waits. This method returns void, so consider invoking
shutdown()
directly if there's a need to handle dangling RPCs.close
in interface AutoCloseable
Exception
- if an error happens while closing the connectionspublic com.stumbleupon.async.Deferred<ArrayList<Void>> shutdown()
Flushes
all buffered edits.Deferred
, whose callback chain will be invoked once all
of the above have been done. If this callback chain doesn't fail, then
the clean shutdown will be successful, and all the data will be safe on
the Kudu side. In case of a failure (the "errback" is invoked) you will have
to open a new AsyncKuduClient if you want to retry those operations.
The Deferred doesn't actually hold any content.