Support for Python 2.x and Python 3.4 and earlier is deprecated and may be removed in the next minor release.

Clients can now require authentication and encryption instead of depending on server-side settings (KUDU-1921).

Kudu Masters now automatically attempt to add themselves to an existing cluster if there is a healthy Raft quorum among Kudu Masters.

A new tool kudu master unsafe_rebuild is added to reconstruct the master catalog from tablet metadata collected from tablet servers. This can be used in emergencies to restore access to tables when all masters are unavailable.

A new tool kudu table set_replication_factor is added to alter the replication factor of a table. The tool immediately updates table metadata in the master, and the master will asynchronously effect the new replication factor. Progress can be monitored by running ksck.

It’s now possible to require a minimum replication factor for a Kudu table. This can be achieved by customizing the setting for the newly introduced --min_num_replicas kudu-master’s flag. For example, setting --min_num_replicas=3 enforces every newly created table to have at least 3 replicas for each of its tablets, so there cannot be a data loss when just a single tablet server in the cluster fails irrecoverably. For the sake of backward compatibility, --min_num_replicas is set to 1 by default.

It’s now possible to track startup progress on the /startup page on the web UI. There are also metrics added to track the overall server startup progress as well as the processing of the log block containers and starting of the tablets (KUDU-1959).

A new tool kudu table add_column is added to add columns to existing tables using the CLI (KUDU-3339).

A new tool kudu tserver unregister is added to remove a dead tablet server from the cluster without restarting the masters (KUDU-2915).

Kudu will now more aggressively fsync consensus-related metadata when metadata is configured to be on an XFS mount. This may lead to increased contention on the device that backs metadata, but will prevent corruption in the event of an outage (KUDU-2195).

A clearer message is logged when the Ranger subprocess crashes, to specify a problem with the Ranger client.

Two new flags have been introduced for the kudu table scan and kudu perf table_scan CLI tools: --row_count_only and --report_scanner_stats. With these new flags, the above mentioned CLI tools allow to issue scan requests equivalent to running “SELECT COUNT(1) FROM <table_name>” from impala-shell. These new provisions are useful in detecting and troubleshooting scan performance issues.

Added replica selection configuration knob for the kudu table scan and kudu perf table_scan CLI tools: it’s controlled by the --replica_selection flag.

To improve security, the following flags are now marked as sensitive and will be redacted in the logs and WebUI when the redaction is enabled:

The logic to select the effective time source when running with --time_source=auto has been updated. The builtin time source would be auto-selected if a Kudu server runs with --time_source=auto in an environment where the instance detector isn’t aware of dedicated NTP servers AND the --builtin_ntp_servers flag is set to a valid value. Otherwise, if --builtin_ntp_servers flag is set to an empty or invalid value, the effective time source becomes system for platforms supporting the get_ntptime() API, otherwise the catch-all case selects the system_unsync as the time effective source.

It is now possible to print or edit PBC files in batch using the kudu pbc CLI tool, and also to format its JSON input/output as “pretty”.

Client connection timeout is now configurable in the Java client (KUDU-3240).

A new /healthz endpoint is now available on the kudu-master and tablet-server embedded web servers for liveness checks (KUDU-3308).

Hive Metastore URI is now logged to the console when connecting via kudu hms CLI tool (KUDU-3189).

It is now possible to start up a master when there is an additional master address present in the master addresses flag (KUDU-3311).

Table entity is now accessible in KuduWriteOperation in the C++ client, making understanding errors on the client side easier (KUDU-2623).

The rebalancer tool now doesn’t move replicas to tablet servers in maintenance mode (KUDU-3328).

Improved the performance of the run length encoding (RLE).

Log4J used in Ranger subprocess was upgraded to 2.17.1 which contains patches go several security vulnerabilities (CVE-2021-44832, CVE-2021-45105, CVE-2021-45046, and CVE-2021-44228).

Kudu servers previously crashed if hostnames became unresolvable via DNS (e.g. if the container hosting a server were destroyed). Such errors are now treated as transient and the lookups are retried periodically. See KUDU-75, KUDU-1620, and KUDU-1885 for more details.

Fixed an issue in Kudu Java client where concurrent flushing of data buffers could lead to errors reported as 'java.lang.AssertionError: This Deferred was already called' (KUDU-3277).

Fixed Kudu RPC negotiation issue when running with cyrus-sasl-gssapi-2.1.27-5 and newer versions of the RPM package. A failed RPC connection negotiation attempt would result in an error logged along with the full connection negotiation trace: Runtime error: SASL(-15): mechanism too weak for this user: Unable to find a callback: 32775 (KUDU-3297).

Fixed crash in kudu-master and kudu-tserver when running with kernel where the getrandom(2) API is not available (versions of Linux kernel prior to 3.17).

Fixed bug which could lead to exhaustion of the address space for the outgoing connections on a busy Kudu cluster (KUDU-3352).

Fixed a bug in the Java client where a malformed tablet server ID in the scan token causes connection failures and timeouts in some cases (KUDU-3349).

Fixed a bug where the rebalancer failed with -ignored_tservers flag (KUDU-3346).

Kudu 1.16 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.15 and Kudu 1.16 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.16 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.16 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.16 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Riza Suminto

Zoltan Chovan

kedeng

khazarmammadli

yejiabao

The kudu-mapreduce integration has been removed in the 1.15.0 release. Similar functionality and capabilities now exist via the Apache Spark, Apache Hive, Apache Impala, and Apache NiFi integrations. See KUDU-3142 for details.

Support for Python 2.x and Python 3.4 and earlier is deprecated and may be removed in the next minor release.

Kudu now experimentally supports multi-row transactions. Currently only INSERT and INSERT_IGNORE operations are supported. See here for a design overview of this feature.

Kudu now supports Raft configuration change for Kudu masters and CLI tools for orchestrating addition and removal of masters in a Kudu cluster. These tools substantially simplify the process of migrating to multiple masters, recovering a dead master and removing masters from a Kudu cluster. For detailed steps, see the latest administration documentation. This feature is evolving and the steps to add, remove and recover masters may change in the future. See KUDU-2181 for details.

Kudu now supports table comments directly on Kudu tables which are automatically synchronized when the Hive Metastore integration is enabled. These comments can be added at table creation time and changed via table alteration.

Kudu now experimentally supports per-table size limits based on leader disk space usage or number of rows. When generating new authorization tokens, Masters will now consider the size limits and strip tokens of INSERT and UPDATE privileges if either limit is reached. To enable this feature, set the --enable_table_write_limit master flag; adjust the --table_disk_size_limit and --table_row_count_limit flags as desired or use the kudu table set_limit tool to set limits per table.

It is now possible to change the Kerberos Service Principal Name using the --principal flag. The default SPN is still kudu/_HOST. Clients connecting to a cluster using a non-default SPN must set the sasl_protocol_name or saslProtocolName to match the SPN base (i.e. “kudu” if the SPN is “kudu/_HOST”) in the client builder or the Kudu CLI. See KUDU-1884 for details.

Kudu RPC now supports TLSv1.3. Kudu servers and clients automatically negotiate TLSv1.3 for Kudu RPC if OpenSSL (or Java runtime correspondingly) on each side supports TLSv1.3. If necessary, use the newly introduced flag --rpc_tls_ciphersuites to customize TLSv1.3-specific cipher suites at the server side. See KUDU-2871 for details.

TLS ciphers renegotiation for TLSv1.2 and prior protocol versions is now explicitly disabled. See KUDU-1926 for details.

The location assignment for Kudu clients is now disabled by default since it doesn’t bring a lot of benefits, but rather puts an extra load to Kudu masters. This change reduces the load on Kudu masters which is essential if too many clients run in a cluster. To enable the location assignment for clients, override the default by setting --master_client_location_assignment_enabled=true for Kudu masters.

The behavior of the C++ client replica selection for closest replica, the default, was updated to match the behavior of the Java client. Instead of picking a random replica each time, a static value is used for each process ensuring that the selection remains deterministic and can benefit from better caching. See KUDU-3248 for details.

The Web UI /rpcz endpoint now displays information on whether an RPC connection is protected by TLS, and if so, provides information on the negotiated TLS cipher suite.

Tooling requests and C++ client requests bound for leader masters will now be retried in the event the masters cannot be reached.

Cluster tooling will now validate that the master argument contains no duplicate values. See KUDU-3226 for details.

The error message output by Kudu Java client in an attempt to write into a non-existent table partition now contains the table’s name. See KUDU-3267 for details.

Fixed a bug in the Kudu tablet servers that could result in a crash when performing an incremental backup of rows that had many batches of updates. See KUDU-3291 for more details.

The Kudu Java client will now retry scans bound for tablets hosted on quiescing tablet servers at replicas on other tablet servers. See KUDU-3213 for more details.

Fixed a race between the scheduling of a maintenance op and the destruction of a tablet. This could previously lead to a crash. See KUDU-3268 for more details.

Fixed crash in Kudu C++ client introduced with KUDU-1802. See KUDU-3254 for details.

Fixed bug in Kudu Java client which manifested in AUTO_FLUSH_BACKGROUND sessions hung in a call to KuduSession.flush() method. Another sign of the bug were stuck data ingest workloads based on Java client (e.g., kudu-spark applications) with "java.lang.AssertionError: This Deferred was already called!" message in the logs. See KUDU-3277 for details.

Fixed crash in Kudu server due to lack of getrandom(2) system call in Linux kernel version earlier than 3.17 by instead using /dev/random for uuid generation in the Boost library. Crash includes the following message in the logs "terminate called after throwing an instance of 'boost::wrapexcept<boost::uuids::entropy_error>'". See the fix for a sample stack trace.

Kudu 1.15 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.14 and Kudu 1.15 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.15 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.15 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.15 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Updated hash computation for empty strings in the FastHash implementation to conform with the handling in Apache Impala. For Bloom filter predicate pushdown feature that uses FastHash, this makes the Kudu client older than version 1.15.0 incompatible with Kudu server version 1.15.0 and Kudu client version at or newer than 1.15.0 incompatible with Kudu server version earlier than 1.15.0. Both client library and Kudu server need to be updated to version 1.15.0 or above if using the Bloom filter predicate feature. One manifestation of this incompatibility is following message in the logs, "Not implemented: call requires unsupported application feature flags: 4". See KUDU-3286 for details.

Abhishek Chennaka

shenxingwuying

Support for CentOS 6/RHEL 6, Ubuntu 14, Ubuntu 16, and Debian 8 platforms has been dropped given they are at or near end-of-life. We will no longer validate these platforms as a part of the release process, though patches will still be accepted going forward.

Developer support for OS X 10.10 Yosemite, OS X 10.11 El Capitan, and OS X 10.12 Sierra has been dropped. We will no longer validate these versions as a part of the release process, though patches will still be accepted going forward.

Support for Python 2.x and Python 3.4 and earlier is deprecated and may be removed in the next minor release.

The kudu-mapreduce integration has been deprecated and may be removed in the next minor release. Similar functionality and capabilities now exist via the Apache Spark, Apache Hive, Apache Impala, and Apache NiFi integrations.

Full support for INSERT_IGNORE, UPDATE_IGNORE, and DELETE_IGNORE operations was added. The INSERT_IGNORE operation will insert a row if one matching the key does not exist and ignore the operation if one already exists. The UPDATE_IGNORE operation will update the row if one matching the key exists and ignore the operation if one does not exist. The DELETE_IGNORE operation will delete the row if one matching the key exists and ignore the operation if one does not exist. These operations are particularly useful in situations where retries or duplicate operations could occur and you do not want to handle the errors that could result manually or you do not want to cause unnecessary writes and compaction work as a result of using the UPSERT operation. The Java client can check if the cluster it is communicating with supports these operations by calling the supportsIgnoreOperations() method on the KuduClient. See KUDU-1563 for more details.

Spark 3 compatible JARs compiled for Scala 2.12 are now published for the Kudu Spark integration. See KUDU-3202 for more details.

Every Kudu cluster now has an automatically generated cluster Id that can be used to uniquely identify a cluster. The cluster Id is shown in the masters web-UI, the kudu master list tool, and in master server logs. See KUDU-2574 for more details.

It is now possible to enforce that OpenSSL is initialized in FIPS approved mode in the servers and the C++ client by setting the KUDU_REQUIRE_FIPS_MODE environment variable to “1”, “yes” or “true”. See KUDU-3210 for more details.

Downloading the WAL data and data blocks when copying tablets to another tablet server is now parallelized, resulting in much faster tablet copy operations. These operations occur when recovering from a down tablet server or when running the cluster rebalancer. See KUDU-1728 and KUDU-3214 for more details.

The HMS integration now supports multiple Kudu clusters associated with a single HMS including Kudu clusters that do not have HMS synchronization enabled. This is possible, because the Kudu master will now leverage the cluster Id to ignore notifications from tables in a different cluster. Additionally, the HMS plugin will check if the Kudu cluster associated with a table has HMS synchronization enabled. See KUDU-3192 and KUDU-3187 for more details.

The HMS integration now supports gzipped HMS notifications. This is important in order to support Hive 4 where the default encoder was changed to be the GzipJSONMessageEncoder. See KUDU-3201 for more details.

Kudu will now fail tablet replicas that have been corrupted due to KUDU-2233 instead of crashing the tablet server. If a healthy majority still exists, a new replica will be created and the failed replica will be evicted and deleted. See KUDU-3191 and KUDU-2233 for more details.

DeltaMemStores will now be flushed as long as any DMS in a tablet is older than the point defined by --flush_threshold_secs, rather than flushing once every --flush_threshold_secs period. This can reduce memory pressure under update- or delete-heavy workloads, and lower tablet server restart times following such workloads. See KUDU-3195 for more details.

The kudu perf loadgen CLI tool now supports UPSERT for storing the generated data into the table. To switch to UPSERT for row operations (instead of default INSERT), add the --use_upsert command-line flag.

Users can now specify the level of parallelization when copying a tablet using the kudu local_replica copy_from_remote CLI tool by passing the --tablet_copy_download_threads_nums_per_session argument.

The Kudu Masters now discriminate between overlapped and exact duplicate key ranges when adding new partitions, returning Status::AlreadyPresent() for exact range duplicates and Status::InvalidArgument() for otherwise overlapped ones. In prior releases, the master returned Status::InvalidArgument() both in case of duplicate and otherwise overlapped ranges.

The handling of an empty list of master addresses in Kudu C++ client has improved. In prior releases, KuduClientBuilder::Build() would hang in ConnectToCluster() if no master addresses were provided. Now, KuduClientBuilder::Build() instantly returns Status::InvalidArgument() in such a case.

The connection negotiation timeout for Kudu C client is now programmatically configurable. To customize the connection negotiation timeout, use the newly introduced `KuduClientBuilder::connection_negotiation_timeout()` method in the Kudu C client API.

All RPC-related kudu CLI tools now have --negotiation_timeout_ms command line flag to control the client-side connection negotiation timeout. The default value for the new flag is set to 3000 milliseconds for backward compatibility. Keep in mind that the total RPC timeout includes the connection negotiation time, so in general it makes sense to bump --timeout_ms along with --negotiation_timeout_ms by the same delta.

Kudu now reports on slow SASL calls (i.e. calls taking more than 250 milliseconds to complete) when connecting to a server. This is to help diagnose issues like described in KUDU-3217.

MaintenanceManager now has a new histogram-based maintenance_op_find_best_candidate_duration metric to capture the stats on how long it takes (in microseconds) to find the best maintenance operation among available candidates. The newly introduced metric can help in diagnosing conditions where MaintenanceManager seems lagging behind the rate of write operations in a busy Kudu cluster with many replicas per tablet server.

The KuduScanToken Java API has been extended with a deserializeIntoScannerBuilder() method that can be used to further customize generated tokens.

Logging of the error message produced when applying an op while a Java KuduSession is closed has been throttled. See KUDU-3012 for more details.

Added a new uptime metric for a Kudu server. The metric’s value is reported as the length of the time interval passed from the start of the server, in microseconds. Knowing the server’s uptime, it’s easier to interpret and compare metrics reported by different Kudu servers.

Documentation for Kudu’s metrics are now automatically generated for each release and can be seen here.

Fixed lock contention between MaintenanceManager op registration and the scheduling of new maintenance ops. On particularly dense tablet servers, this contention was previously shown to significantly slow down startup times. See KUDU-3149 for more details.

Fixed lock contention between MaintenanceManager’s threads performing already scheduled operations and the scheduler thread itself. This benefits clusters with heavy ingest/update workloads that have many replicas per tablet server. See KUDU-1954 for more details.

Fixed a bug in the merge iterator that could result in a crash. This could surface as a crash when performing ordered or differential scans, particularly when the underlying data contained deletes and reinserts. See KUDU-3108 for more details.

Fixed a heap-use-after-free bug in Kudu C++ client that might manifest itself when altering a table to update the partitioning schema. See KUDU-3238 for more details.

Fixed a bug where building scan tokens would result in a NullPointerException if a tablet not found error occurred before generating the token. See KUDU-3205 for more details.

Fixed a bug where a delete operation would fail if the row being deleted contained exactly 64 columns and all values were set on the row. See KUDU-3198 for more details.

Fixed a bug where Slf4j classes were shaded into the Spark integration JARs. See KUDU-3157 for more details.

Fixed a bug where the 'kudu hms fix' tool mistakenly reports non-matching master addresses when the addresses are in-fact canonically the same. See KUDU-2884 for more details.

Kudu 1.14 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.13 and Kudu 1.14 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.14 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.14 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.14 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

liguohao

The Sentry integration has been removed and the Ranger integration should now be used in its place for fine-grained authorization.

Support for Python 2.x and Python 3.4 and earlier is deprecated and may be removed in the next minor release.

The kudu-mapreduce integration has been deprecated and may be removed in the next minor release. Similar functionality and capabilities now exist via the Apache Spark, Apache Hive, Apache Impala, and Apache NiFi integrations.

Added table ownership support. All newly created tables are automatically owned by the user creating them. It is also possible to change the owner by altering the table. You can also assign privileges to table owners via Apache Ranger (see KUDU-3090).

An experimental feature is added to Kudu that allows it to automatically rebalance tablet replicas among tablet servers. The background task can be enabled by setting the --auto_rebalancing_enabled flag on the Kudu masters. Before starting auto-rebalancing on an existing cluster, the CLI rebalancer tool should be run first (see KUDU-2780).

Bloom filter column predicate pushdown has been added to allow optimized execution of filters which match on a set of column values with a false-positive rate. Support for Impala queries utilizing Bloom filter predicate is available yielding performance improvements of 19% to 30% in TPC-H benchmarks and around 41% improvement for distributed joins across large tables. Support for Spark is not yet available. (see KUDU-2483).

AArch64-based (ARM) architectures are now supported including published Docker images.

The Java client now supports the columnar row format returned from the server transparently. Using this format can reduce the server CPU and size of the request over the network for scans. The columnar format can be enabled via the setRowDataFormat() method on the KuduScanner.

An experimental feature that can be enabled by setting the --enable_workload_score_for_perf_improvement_ops prioritizes flushing and compacting hot tablets.

Hive metastore synchronization now supports Hive 3 and later.

The Spark KuduContext accumulator metrics now track operation counts per table instead of cumulatively for all tables.

The kudu local_replica delete CLI tool now accepts multiple tablet identifiers. Along with the newly added --ignore_nonexistent flag, this helps with scripting scenarios when removing multiple tablet replicas from a particular Tablet Server.

Both Master’s and Tablet Server’s web UI now displays the name for a service thread pool group at the /threadz page

Introduced queue_overflow_rejections_ metrics for both Masters and Tablet Servers: number of RPC requests of a particular type dropped due to RPC service queue overflow.

Introduced a CoDel-like queue control mechanism for the apply queue. This helps to avoid accumulating too many write requests and timing them out in case of seek-bound workloads (e.g., uniform random inserts). The newly introduced queue control mechanism is disabled by default. To enable it, set the --tablet_apply_pool_overload_threshold_ms Tablet Server’s flag to appropriate value, e.g. 250 (see KUDU-1587).

Java client’s error collector can be resized (see KUDU-1422).

Calls to the Kudu master server are now drastically reduced when using scan tokens. Previously deserializing a scan token would result in a GetTableSchema request and potentially a GetTableLocations request. Now the table schema and location information is serialized into the scan token itself avoiding the need for any requests to the master when processing them.

The default size of Master’s RPC queue is now 100 (it was 50 in earlier releases). This is to optimize for use cases where a Kudu cluster has many clients working concurrently.

Masters now have an option to cache table location responses. This is targeted for Kudu clusters which have many clients working concurrently. By default, the caching of table location responses is disabled. To enable table location caching, set the proper capacity of the table location cache using Master’s --table_locations_cache_capacity_mb flag (setting to 0 disables the caching). Up to 17% of improvement is observed in GetTableLocations request rate when enabling the caching.

Removed lock contention on Raft consensus lock in Tablet Servers while processing a write request. This helps to avoid RPC queue overflows when handling concurrent write requests to the same tablet from multiple clients (see KUDU-2727).

Master’s performance for handling concurrent GetTableSchema requests has been improved. End-to-end tests indicated up to 15% improvement in sustained request rate for high concurrency scenarios.

Kudu servers now use protobuf Arena objects to perform all RPC request/response-related memory allocations. This gives a boost for overall RPC performance, and with further optimization the result request rate was increased significantly for certain methods. For example, the result request rate increased up to 25% for Master’s GetTabletLocations() RPC in case of highly concurrent scenarios (see KUDU-636).

Tablet Servers now use protobuf Arena for allocating Raft-related runtime structures. This results in substantial reduction of CPU cycles used and increases write throughput (see KUDU-636).

Tablet Servers now use protobuf Arena for allocating EncodedKeys to reduce allocator contention and improve memory locality (see KUDU-636).

Bloom filter predicate evaluation for scans can be computationally expensive. A heuristic has been added that verifies rejection rate of the supplied Bloom filter predicate below which the Bloom filter predicate is automatically disabled. This helped reduce regression observed with Bloom filter predicate in TPC-H benchmark query #9 (see KUDU-3140).

Improved scan performance of dictionary and plain-encoded string columns by avoiding copying them (see KUDU-2844).

Improved maintenance manager’s heuristics to prioritize larger memstores (see KUDU-3180).

Spark client’s KuduReadOptions now supports setting a snapshot timestamp for repeatable reads with READ_AT_SNAPSHOT consistency mode (see KUDU-3177).

Kudu scans now honor location assignments when multiple tablet servers are co-located with the client.

Fixed a bug that caused IllegalArgumentException to be thrown when trying to create a predicate for a DATE column in Kudu Java client (see KUDU-3152).

Fixed a potential race when multiple RPCs work on the same scanner object.

Kudu 1.13 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.12 and Kudu 1.13 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.13 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.13 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.13 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Jim Apple

Kevin J McCarthy

Li Zhiming

Mahesh Reddy

Romain Rigaux

RuiChen

Shuping Zhou

ningw

wenjie

The Flume sink has been migrated to the Apache Flume project and removed from Kudu. Users depending on the Flume integration can use the old kudu-flume jars or migrate to the Flume jars containing the Kudu sink.

Support for Apache Sentry authorization has been deprecated and may be removed in the next minor release. Users depending on the Sentry integration should migrate to the Apache Ranger integration for fine-grained authorization.

Support for Python 2 has been deprecated and may be removed in the next minor release.

Support for CentOS/RHEL 6, Debian 8, Ubuntu 14 has been deprecated and may be removed in the next minor release.

Kudu now supports native fine-grained authorization via integration with Apache Ranger. Kudu may now enforce access control policies defined for Kudu tables and columns stored in Ranger. See the authorization documentation for more details.

Kudu’s web UI now supports proxying via Apache Knox. Kudu may be deployed in a firewalled state behind a Knox Gateway which will forward HTTP requests and responses between clients and the Kudu web UI.

Kudu’s web UI now supports HTTP keep-alive. Operations that access multiple URLs will now reuse a single HTTP connection, improving their performance.

The kudu tserver quiesce tool is added to quiesce tablet servers. While a tablet server is quiescing, it will stop hosting tablet leaders and stop serving new scan requests. This can be used to orchestrate a rolling restart without stopping on-going Kudu workloads.

Introduced auto time source for HybridClock timestamps. With --time_source=auto in AWS and GCE cloud environments, Kudu masters and tablet servers use the built-in NTP client synchronized with dedicated NTP servers available via host-only networks. With --time_source=auto in environments other than AWS/GCE, Kudu masters and tablet servers rely on their local machine’s clock synchronized by NTP. The default setting for the HybridClock time source (--time_source=system) is backward-compatible, requiring the local machine’s clock to be synchronized by the kernel’s NTP discipline.

The kudu cluster rebalance tool now supports moving replicas away from specific tablet servers by supplying the --ignored_tservers and --move_replicas_from_ignored_tservers arguments (see KUDU-2914 for more details).

The kudu table create tool is added to allow users to specify table creation options using JSON.

Kudu now supports DATE and VARCHAR data types. See the schema design documentation for more details.

Write Ahead Log file segments and index chunks are now managed by Kudu’s file cache. With that, all long-lived file descriptors used by Kudu are managed by the file cache, and there’s no longer a need for capacity planning of file descriptor usage.

Kudu no longer requires the running of kudu fs update_dirs to change a directory configuration or recover from a disk failure (see KUDU-2993).

Kudu tablet servers and masters now expose a tablet-level metric num_raft_leaders for the number of Raft leaders hosted on the server.

Kudu’s maintenance operation scheduling has been updated to prioritize reducing WAL retention under memory pressure. Kudu would previously prioritize operations that yielded high-memory reduction, which could result in high WAL disk usage in workloads that contained updates (see KUDU-3002).

A new maintenance operation is introduced to remove rowsets that have had all of their rows deleted and whose newest delete operations are considered ancient (see KUDU-1625).

The built-in NTP client is now fully supported as the time source for Kudu’s HybridTime clock, i.e. it’s no longer marked as experimental. To switch the time source from the existing system time source (which is the default) to the built-in NTP client, use --time_source=builtin.

Introduced additional metrics for the built-in NTP client (see KUDU-3048).

Updated /config page of masters' and tablet servers' web UI to display configured and effective time source. In addition, the effective list of reference servers for the built-in NTP client is shown there as well, if applicable.

chronyd (version 3.4 and newer) is now supported as NTP server for synchronizing the local machine’s clock in a Kudu cluster. It’s important to have the rtcsync option enabled in the configuration of the chronyd NTP daemon (see KUDU-2573).

Kudu now supports building and running on RHEL/CentOS 8. This has been tested with CentOS 8.1.

The processing of Raft consensus vote requests has been improved to be more robust during high contention scenarios like election storms.

Added a validator to enforce consistency between the maximum size of an RPC and the maximum size of tablet transaction memory, controlled by --rpc_max_message_size and --tablet_transaction_memory flags correspondingly. In prior releases, if the limit on the size of RPC requests is increased and the limit on tablet transaction memory size is kept with the default setting, certain Raft transactions could be committed but not applied (see KUDU-3023).

The metrics endpoint now supports filtering metrics by a metric severity level. See the documentation for more details.

Many kudu local_replica tools are updated to not open the block manager, which significantly reduces the amount of IO done when running them (see KUDU-3070 for more details).

The Kudu Java client now exposes a way to get the resource metrics associated with a given scanner (see KUDU-2162 for more details).

Scan predicates are pushed down to RLE decoders, improving predicate-evaluation-efficiency in some workloads (see KUDU-2852 for more details).

The log block manager will now attempt to use multiple threads to open blocks in each data directory, in some tests reducing startup time by up to 20% (see KUDU-2977 and KUDU-3001 for more details).

Kudu’s tablet server web UI scans page is updated to show the number of round trips per scanner.

Kudu’s master and tablet server web UIs are updated to show critical partition information, including tablet count and on-disk size.

Kudu servers now expose the last_read_elapsed_seconds and last_write_elapsed_seconds tablet-level metrics that indicate how long ago the most recent read and write operations to a given tablet were.

Kudu servers now expose the transaction_memory_limit_rejections tablet-level metric that tracks the number of transactions rejected because a given tablet’s transactional memory limit was reached (see KUDU-3021 for more details).

Fixed a bug in which Kudu would not schedule compactions if a server were under memory pressure (see KUDU-2929).

Fixed a bug where DDL operations like ALTER TABLE on tables with huge number of partitions might result in a DoS situation for Kudu masters (see KUDU-3036).

Fixed a bug where Kudu Java client cannot negotiate a secure connection with Kudu masters and tablet servers if using BouncyCastle JCE provider (see KUDU-3106).

Kudu masters will now crash immediately upon hitting a disk failure (see KUDU-2904 for more details).

Fixed an issue in the Kudu master in which delays in receiving tablet server heartbeats could result in an excess amount of RPC traffic between the masters and tablet servers (see KUDU-2992 for more details).

Fixed an issue with Kudu’s location placement policy that would place all replicas in one location when two locations were available (see KUDU-3008 for more details).

The Java client will now correctly propagate timestamps when sending write batches (see KUDU-3035 for more detail).

Fixed an issue with the Kudu backup Spark jobs in which Kudu would return with a non-zero exit if the job succeeded but backed up no rows (see KUDU-3099 for more details).

The raft_term and time_since_last_leader_heartbeat aggregated table metrics will now return the maximum metric reported instead of the sum.

Kudu 1.12 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.11 and Kudu 1.12 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.12 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.12 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.12 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Andy Singer

Michele Milesi

Ning Wang

Renhai Zhao

Sheng Liu

Thomas D’Silva

Tianhua Huang

Waleed Fateem

Fixed an issue with distributing libnuma dynamic library with kudu-binary JAR artifact. Also, fixed the issue of statically compiling in libnuma.a into kudu-master and kudu-tserver binaries when building Kudu from source in release mode. The fix removes both numactl and memkind projects from Kudu’s thirdparty dependencies and makes the dependency on the libmemkind library optional, opening the library using dlopen() and resolving required symbols via dlsym() (see KUDU-2990).

Fixed an issue with kudu cluster rebalancer CLI tool crashing when running against a location-aware cluster if a tablet server in one location doesn’t contain a single tablet replica (see KUDU-2987).

Fixed an issue with connection negotiation using SASL mechanism when server FQDN is longer than 64 characters (see KUDU-2989).

Fixed an issue in the test harness of the kudu-binary JAR artifact. With this fix, kudu-master and kudu-tserver processes of the mini-cluster’s test harness no longer rely on the test NTP server to synchronize their built-in NTP client. Instead, the test harness relies on the local machine clock synchronized by the system NTP daemon (see KUDU-2994).

Since KUDU-2625 is addressed, tablet servers now reject individual write operations which violate schema constraints in a batch of write operations. In prior versions the behavior was to reject the whole batch of write operations if a violation of the schema constraints is detected even for a single row. It’s recommended to revise applications which relied on the behavior mentioned above upon upgrading to Kudu 1.11.0.

The Kudu Flume integration is deprecated and may be removed in the next minor release. The integration will be moved to the Apache Flume project going forward (see FLUME-3345).

Kudu now supports putting tablet servers into maintenance. While in this mode, the tablet server’s replicas will not be re-replicated if it fails. Only upon exiting maintenance will re-replication be triggered for any remaining under-replicated tablets. The kudu tserver state enter_maintenance and kudu tserver state exit_maintenance tools are added to orchestrate tablet server maintenance, and the kudu tserver list tool is amended with a "state" column option to display current state of each tablet server (see KUDU-2069).

Kudu now has a built-in NTP client which maintains the internal wallclock time used for generation of HybridTime timestamps. When enabled, system clock synchronization for nodes running Kudu is no longer necessary. This is useful for containerized deployments and in other cases when it’s troublesome to maintain properly configured system NTP service at each node of a Kudu cluster. The list of NTP servers to synchronize against is specified with the --builtin_ntp_servers flag. By default, Kudu masters and tablet servers use public servers hosted by the NTP Pool project. To use the built-in NTP client, set --time_source=builtin and reconfigure --builtin_ntp_servers if necessary (see KUDU-2935).

Aggregated table statistics are now available to Kudu clients via KuduClient.getTableStatistics() and KuduTable.getTableStatistics() methods in the Kudu Java client and KuduClient.GetTableStatistics() in the Kudu C++ client. This allows for various query optimizations. For example, Spark now uses it to perform join optimizations. The statistics are available via the API of both C++ and Java Kudu clients. In addition, per-table statistics are available via kudu table statistics CLI tool. The statistics are also available via master’s Web UI at master:8051/metrics and master:8051/table?id=<uuid> URIs (see KUDU-2797 and KUDU-2921).

The kudu CLI tool now supports altering table columns. Use the newly introduced sub-commands such as kudu table column_set_default, kudu table column_remove_default, kudu table column_set_compression, kudu table column_set_encoding, and kudu table column_set_block_size to alter a column of the specified table.

The kudu CLI tool now supports dropping table columns. Use the newly introduced kudu table delete_column sub-command to drop a column of the specified table.

The kudu CLI tool now supports getting and setting extra configuration properties for a table. Use kudu table get_extra_configs and kudu table set_extra_config sub-commands to perform the corresponding operations (see KUDU-2514).

The kudu CLI tool now supports creating and dropping range partitions for a table. Use kudu table add_range_partition and kudu table drop_range_partition sub-commands to perform the corresponding operations (see KUDU-2881).

The kudu fs dump uuid CLI tool is now significantly faster and consumes significantly less IO.

The memory consumed by CFileReaders and BloomFileReaders is factored out and accounted separately by the tablet server memory tracking. The stats are available via Web UI as "CFileReaders" and "BloomFileReaders" entries.

KuduScanBatch::const_iterator in Kudu C++ client now supports operator→() (see KUDU-1561).

Master server Web UI now supports sorting the list of tables by the columns of "Table Name", "Create Time", and "Last Alter Time".

Tablet servers now expand a tablet’s data directory group with available healthy directories when all directories of the group are full (see KUDU-2907).

For scan operations run with CLOSEST_REPLICA selection mode, the Kudu Java client now picks a random available replica in case no replica is located at the same node with the client that initiated the scan operation. This helps to spread the load generated by multiple scan requests to the same tablet among all available replicas. In prior releases, all such scan requests might end up fetching data from the same tablet replica (see KUDU-2348).

The serialization of in-memory rows to Kudu’s wire format has been optimized to be more CPU efficient (see KUDU-2847).

Tablet servers and masters can now aggregate metrics by the same attribute. For example, it’s now possible to fetch aggregated metrics from a tablet server by retrieving data from URLs of form http://<host>:<port>/metrics?merge_rules=tablet|table|table_name

Introduced Docker image for Python Kudu client (see KUDU-2849).

Tablet servers now consider available disk space when choosing a set of data directories for a tablet’s data directory group, and when deciding in which data directory a new block should be written (see KUDU-2901).

Added a quick-start example of using Apache Spark to load, query, and modify a real data set stored in Kudu.

Added a quick-start example of using Apache Nifi to ingest data into Kudu.

Tablet servers now reject individual write operations which violate schema constraints in a batch of write operations received from a client. The previous behavior was to reject the whole batch of write operations if a violation of the schema constraints is detected even for a single row (see KUDU-2625).

Tablet replicas can now be optionally placed in accordance with a dimension-based placement policy. To specify a dimension label for a table, use the KuduTableCreator::dimension_label() and CreateTableOptions.setDimensionLabel() methods of the C++ and Java Kudu clients. To add a partition with a dimension label, use the KuduTableAlterer::AddRangePartitionWithDimension() and AlterTableOptions.addRangePartition() methods of the C++ and Java Kudu clients (see KUDU-2823).

Kudu RPC now enables TCP keepalive for all outbound connections for faster detection of no-longer-reachable nodes (see KUDU-2192).

The kudu table scan and kudu table copy CLI tools now fail gracefully rather than crashing upon hitting an error (see KUDU-2851).

Optimized decoding of deltas' timestamps (see KUDU-2867).

Optimized the initialization of DeltaMemStore for the case when no matching deltas are present (see KUDU-2381).

Improved the rehydration of scan tokens. Now a scan token created before renaming a column can be used even after the column has been renamed.

The memory reserved by tcmalloc is now released to OS periodically to avoid potential OOM issues in the case of read-only workloads (see KUDU-2836).

Optimized evaluation of predicates on columns of primitive types and NULL/NOT NULL predicates to leverage SIMD instructions (see KUDU-2846).

Fixed an issue of fault-tolerant scan operation failing for a projection with key columns specified in other than the table schema’s order (see KUDU-2980).

Fixed an issue that would cause frequent leader elections in case when persisting Raft transactions to the WAL took longer than the leader election timeout. The issue was contributing to election storms (see KUDU-2947).

Fixed a tablet server crash in cases where blocks were not removed due to IO error. This issue may have surfaced after recovering from a disk failure (see KUDU-2635).

Fixed a crash in master and tablet server by validating the size of default values when de-serializing ColumnSchemaPB (see KUDU-2622).

Fixed RPC negotiation failure in the case when TLS v1.3 is supported at both the client and the server side. This is a temporary workaround before the connection negotiation code is properly updated to support 1.5-RTT handshake used in TLS v1.3. The issue affected Linux distributions shipped or updated with OpenSSL version 1.0.2 and newer (see KUDU-2871).

Fixed a race between GetTabletLocations() and tablet report processing. The race could crash the Kudu master (see KUDU-2842).

Fixed a bug in AlterSchemaTransactionState::ToString() that led to a crash of tablet server when removing a tablet replica with a pending AlterSchema transaction.

Kudu 1.11 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.10 and Kudu 1.11 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.11 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.11 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.11 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Hannah Nguyen

lingbin

Ritwik Yadav

Scott Reynolds

Volodymyr Verovkin

Xiaokai Wang

Xin He

Yao Wang

Fixed an issue with distributing libnuma dynamic library with kudu-binary JAR artifact. Also, fixed the issue of statically compiling in libnuma.a into kudu-master and kudu-tserver binaries when building Kudu from source in release mode. The fix removes both numactl and memkind projects from Kudu’s thirdparty dependencies and makes the dependency on the libmemkind library optional, opening the library using dlopen() and resolving required symbols via dlsym() (see KUDU-2990).

The default tablet history retention time has been raised from 15 minutes to 7 days to better support touchless incremental backups (see KUDU-2677).

Kudu now supports both full and incremental table backups via a job implemented using Apache Spark. Additionally it supports restoring tables from full and incremental backups via a restore job implemented using Apache Spark. See the backup documentation for more details.

Kudu can now synchronize its internal catalog with the Apache Hive Metastore, automatically updating Hive Metastore table entries upon table creation, deletion, and alterations in Kudu. See the HMS synchronization documentation for more details.

Kudu now supports native fine-grained authorization via integration with Apache Sentry. Kudu may now enforce access control policies defined for Kudu tables and columns, as well as policies defined on Hive servers and databases that may store Kudu tables. See the authorization documentation for more details.

Kudu’s web UI now supports SPNEGO, a protocol for securing HTTP requests with Kerberos by passing negotiation through HTTP headers. To enable, set the --webserver_require_spnego command line flag.

Column comments can now be stored in Kudu tables, and can be updated using the AlterTable API (see KUDU-1711).

The Java scan token builder can now create multiple tokens per tablet. To use this functionality, call setSplitSizeBytes() to specify how many bytes of data each token should scan. The same API is also available in Kudu’s Spark integration, where it can be used to spawn multiple Spark tasks per scanned tablet (see KUDU-2670).

Experimental Kudu Docker images are now published on Docker Hub.

Kudu now has an experimental Kubernetes StatefulSet manifest and Helm chart, which can be used to define and provision Kudu clusters using Kubernetes (see KUDU-2398).

The Kudu CLI now has rudimentary YAML-based configuration file support, which can be used to provide cluster connection information via cluster name instead of keying in comma-separated lists of master addresses. See the cluster name documentation for more details.

kudu perf table_scan scans a table and displays a table’s row count as well as the time it took to run the scan.

kudu table copy copies data from one table to another, within the same cluster or across clusters. Note, this implementation leverages a single client, therefore it may not be suitable for large tables.

Tablet history retention time can now be configured on a table-by-table basis. (see KUDU-2514).

The performance of mutations (i.e. UPDATE, DELETE, and re-INSERT) to not-yet-flushed Kudu data has been significantly optimized (see KUDU-2826 and f9f9526d3).

Predicate performance for primitive columns has been optimized (see KUDU-2846).

IS NULL and IS NOT NULL predicate performance has been optimized (see KUDU-2846).

Optimized the performance of fetching tablet locations from the master for tables with large numbers of partitions. This can improve the performance of short-running Spark or Impala queries as well as user applications which make use of short-lived client instances (see KUDU-2711).

The tableExists() (Java) and TableExists() (C++) APIs are now more performant (see KUDU-2802).

Fault tolerant scans are now much more performant and consume far less memory (see KUDU-2466).

kudu cluster ksck now sends more requests in parallel, which should result in a speed-up when running against clusters with many tables or when there’s high latency between the node running the CLI and the cluster nodes.

Kudu’s block manager now deletes spent block containers when needed instead of just at server startup. This should reduce server startup times somewhat (see KUDU-2636).

DNS resolutions are now cached by Kudu masters, tablet servers, and Kudu C++ clients. The TTL for a resolved DNS entry in the cache is 15 seconds by default (see KUDU-2791).

Tables created in Kudu 1.10.0 or later will show their creation time as well as their last alteration time in the web UI (see KUDU-2750).

The Kudu CLI and C++ client now support overriding the local username using the ‘KUDU_USER_NAME’ environment variable. This allows operating against a Kudu cluster using an identity which differs from the local Unix user on the client. Note that this has no effect on secure clusters, where client identity is determined by Kerberos authentication (see KUDU-2717).

Kudu C++ client now performs stricter verification on the input data of INSERT and UPSERT operations w.r.t. table schema constraints. This helps spotting schema violations before sending the data to a tablet server.

The KuduScanner in the Java client is now iterable. Additionally the KuduScannerIterator will automatically make scanner keep alive calls to ensure scanners do not time out while iterating.

A KuduPartitioner API was added to the Java client. The KuduPartitioner API allows a client to determine which partition a row falls into without actually writing that row. For example, the KuduPartitioner is used in the Spark integration to optionally repartition and pre-sort the data before writing to Kudu (see KUDU-2674 and KUDU-2672).

The PartialRow and RowResult Java API have new methods that accept and return Java Objects. These methods are useful when you don’t care about autoboxing and your existing type handling logic is based on Java types. See the javadoc for more details.

The Kudu Java client now logs RPC trace summaries instead of full RPC traces when the log level is INFO or higher. This reduces log noise and makes RPC issues more visible in a more compact format (see KUDU-2830).

Kudu servers now display the time at which they were started in their web UIs.

Kudu tablet servers now display a table’s total column count in the web UI.

The /metrics web UI endpoint now supports filtering on entity types, entity IDs, entity attributes, and metric names. This can be used to more efficiently collect important metrics when there is a large number of tablets on a tablet server.

The Kudu rebalancer now accepts the --ignored_tservers command line argument, which can be used to ignore the health status of specific tablet servers (i.e. if they are down) when deciding whether or not it’s safe to rebalance the cluster.

kudu master list now displays the Raft consensus role of each master in the cluster (i.e. LEADER or FOLLOWER) (see KUDU-2825).

kudu table scan no longer interleaves its output, and now projects all columns without having to manually list the column names.

kudu perf loadgen now supports creating empty tables. The semantics of the special value of 0 for --num_rows_per_thread flag has changed. A value of 0 now indicates that no rows should be generated, and a value of -1 indicates there should be no limit to the number of rows generated.

Running make install after building Kudu from source will now install the Kudu binaries into appropriate locations. (see KUDU-1344).

Fixed an issue where the Java client would fail scans that took a very long time to return a single block of rows, such as highly selective scans over a large amount of data (see KUDU-1868).

Fixed the handling of SERVICE_UNAVAILABLE errors that caused the Java client to do unnecessary master lookups.

Kudu scan tokens now work correctly when the target table is renamed between when the scan token is created and when it is rehydrated into a scanner.

Kudu’s “NTP synchronization wait” behavior at startup now works properly when Kudu is run in a containerized environment.

Fixed a crash when a flush or compaction overlapped with another compaction (see KUDU-2807).

Fixed a rare race at startup where the leader master would fruitlessly try to tablet copy to a healthy follower master, causing the cluster to operate as if it had two masters until master leadership changed (see KUDU-2748).

Under rare circumstances, it was possible for Kudu to crash in libkrb5 when negotiating multiple TLS connections concurrently. This crash has been fixed (see KUDU-2706).

Kudu no longer crashes at startup on machines with disabled CPUs (see KUDU-2721).

Kudu 1.10 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.9 and Kudu 1.10 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.10 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.10 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.10 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Support for building and running with Java 7 has been dropped in this release. It had been deprecated since Kudu 1.5.0. (see KUDU-2099).

Csaba Fulop

Florentino Sainz

Guangchao Deng

Jia Hongchao

Ye Yuqiang

Yifan Zhang

Flume 1.8+ requires Java 8 at runtime even though the Kudu Flume integration is Java 7 compatible. Flume 1.9 is the default dependency version as of Kudu 1.9.0.

Hadoop 3.0+ requires Java 8 at runtime even though the Kudu Hadoop integration is Java 7 compatible. Hadoop 3.2 is the default dependency version as of Kudu 1.9.0.

Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in the next major release.

Kudu now supports location awareness. When configured, Kudu will make a best effort to avoid placing a majority of replicas for a given tablet at the same location. The kudu cluster rebalance tool has been updated to act in accordance with the placement policy of a location-aware Kudu. The administrative documentation has been updated to detail the usage of this feature.

Docker scripts have been introduced to build and run Kudu on various operating systems. See the /docker subdirectory of the source repository for more details. An official repository has been created for Apache Kudu Docker artifacts.

Developers integrating with Kudu can now write Java tests that start a Kudu mini cluster without having to first locally build and install Kudu. This is made possible by the Kudu team providing platform-specific binaries available to Gradle or Maven for download and install at test time. More information on this feature can be found here. This binary test artifact is currently considered to be experimental.

When creating a table, the master now enforces a restriction on the total number of replicas rather than the total number of partitions. If manually overriding --max_create_tablets_per_ts, the maximum size of a new table has effectively been cut by a factor of its replication factor. Note that partitions can still be added after table creation.

The compaction policy has been updated to favor reducing the number of rowsets. This can lead to faster scans and lower bootup times, particularly in the face of a “trickling inserts” workload, where rows are inserted slowly in primary key order (see KUDU-1400).

A tablet-level metric average_diskrowset_height has been added to indicate how much a replica needs to be compacted, as indicated by the average number of rowsets per unit of keyspace.

Scans which read multiple columns of tables undergoing a heavy UPDATE workload are now more CPU efficient. In some cases, scan performance of such tables may be several times faster upon upgrading to this release.

Kudu-Spark users can now provide the short “kudu” format alias to Spark. This enables using .format(“kudu”) in places where you would have needed to provide the fully qualified name like .format(“org.apache.kudu.spark.kudu") or imported org.apache.kudu.spark.kudu._ and used the implicit .kudu functions. The Spark integration documentation has been updated to reflect this improvement.

The KuduSink class has been added to the Spark integration as a StreamSinkProvider, allowing structured streaming writes into Kudu (see KUDU-2640).

The amount of server-side logging has been greatly reduced for Kudu’s consensus implementation and background processes. This logging was determined to be not useful and unnecessarily verbose.

The web UI now more obviously depicts which columns are a part of the primary key (see KUDU-2477).

The kudu table describe tool has been added to support describing table attributes, including schema, partitioning, replication factor, column encodings, compressions, and default values.

The kudu table scan tool has been added to scan rows from a table, supporting comparison, in-list, and is-null predicates.

The kudu locate_row tool has been added to allow users to determine what tablet a given primary key belongs to, and whether a row exists for that primary key.

The kudu diagnose dump_mem_trackers tool is added to allow users to output the contents of the /mem-trackers web UI page in a CSV format.

To avoid glitches and undefined behavior, the Kudu Python client now detects and reports on conflicting/incorrect initialization of the OpenSSL library.

Fixed a crash caused by a race between altering tablet schemas and deleting tablet replicas (see KUDU-1678).

Fixed an issue that would prevent the kudu fs update_dirs tool from removing directories in the presence of tablet tombstones (see KUDU-2680).

The --cmeta_force_fsync flag may be used to fsync Kudu’s consensus metadata more aggressively. Setting this to true may decrease Kudu’s performance, but improve its durability in the face of power failures and forced shutdowns (see KUDU-2195).

Fixed an issue that would cause an excessive amount of RPC traffic from Kudu masters if the tablet servers were configured with duplicated master addresses (see KUDU-2684).

Fixed an issue that would cause the kudu cluster rebalance tool to run indefinitely in the case of tables with a replication factor of 2 (see KUDU-2688).

Fixed an issue that could lead to a failure to bootstrap tablet replicas that were a part of workloads with many alter table operations (see KUDU-2690).

Fixed an issue with the Java scanner’s keepAlive that could lead to a permanent hang in the scanner (see KUDU-2710).

Fixed an issue that would cause undefined behavior upon connecting to a secure cluster concurrently from multiple C++ clients (see KUDU-2706).

Kudu 1.9 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.8 and Kudu 1.9 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.9 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.9 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.9 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Bankim Bhavsar

Mike Parker

Mitch Barnett

Tim Armstrong

Yingchun Lai

Upgrading directly from Kudu 1.7.0 is supported and no special upgrade steps are required. A rolling upgrade may work, however it has not been tested. When upgrading Kudu, it is recommended to first shut down all Kudu processes across the cluster, then upgrade the software on all servers, then restart the Kudu processes on all servers in the cluster.

Kudu Flume Sink released with Kudu 1.8.0 is compiled against Apache Flume 1.8 and might not function with earlier versions of Flume. Note that Flume 1.8 requires Java 1.8 or higher.

Hadoop 3.0+ requires Java 8 at runtime even though the Kudu Hadoop integration is Java 7 compatible. Hadoop 3.1 is the default dependency version as of Kudu 1.8.0, used by certain features in the Java client.

The -table_num_buckets configuration option of the kudu perf loadgen tool is now removed in favor of -table_num_hash_partitions and -table_num_range_partitions (see KUDU-1861).

Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in the next major release.

The producer.skipMissingColumn, producer.skipBadColumnValue, and producer.warnUnmatchedRows Kudu Flume sink configuration parameters have been deprecated in favor of producer.missingColumnPolicy, producer.badColumnValuePolicy, and producer.unmatchedRowPolicy respectively (see KUDU-1882).

Examples showcasing functionality in C++, Java, and Python, previously hosted in a separate repository have been added. They can be found in the examples/ top-level subdirectory.

Added kudu diagnose parse_stacks, a tool to parse sampled stack traces out of a diagnostics log (see KUDU-2353).

Added support for IS NULL and IS NOT NULL predicates to the Kudu Python client (see KUDU-2399).

Introduced manual data rebalancer into the kudu CLI tool. The rebalancer can be used to redistribute table replicas among tablet servers. The rebalancer can be run via kudu cluster rebalance sub-command. Using the new tool, it’s possible to rebalance Kudu clusters of version 1.4.0 and newer.

Added kudu tserver get_flags and kudu master get_flags, two tools that allow superusers to retrieve all the values of command line flags from remote Kudu processes. The get_flags tools support filtering the returned flags by tag, and by default will return only flags that were explicitly set.

Added kudu tablet unsafe_replace_tablet, a tool to replace a tablet with a new one. This tool is meant to be used to recover a table when one of its tablets has permanently lost all replicas. The data in the tablet that is replaced is lost, so this tool should only be used as a last resort (see KUDU-2290).

There is a new metric for each tablet replica tracking the number of election failures since the last successful election attempt and the time since the last heartbeat from the leader (see KUDU-2287).

Kudu now supports building and running on Ubuntu 18.04 (“Bionic Beaver”) (see KUDU-2427).

Kudu now supports building and running against OpenSSL 1.1 (see KUDU-1889).

Added Kerberos support to the Kudu Flume sink (see KUDU-2012).

The Kudu Spark connector now supports Spark Streaming DataFrames (see KUDU-2539).

Added -tables filtering argument to kudu table list (see KUDU-2529).

Clients now support setting a limit on the number of returned rows in scans (see KUDU-16).

Added Pandas support to the Python client (see KUDU-1276).

Enabled configuration of mutation buffer in the Python client (see KUDU-2441).

Added a keepAlive API call to the KuduScanner and AsyncKuduScanner in the Java client. This API can be used to keep the scanners alive on the server when processing of messages will take longer than the scanner TTL (see KUDU-2095).

The Kudu Spark integration now uses the keepAlive API when reading data. By default it will call keepAlive on a scanner with a period of 15 seconds. This will ensure that Spark jobs with large batch sizes or slow processing times do not fail with scanner not found errors (see KUDU-2563).

Number of reactor threads in the C++ client is now configurable (see KUDU-2368).

Added an optimization to reduce CPU consumption when performing hot metadata lookups in the C++ client (see KUDU-1977).

Added an optimization to avoid bottlenecks on getpwuid_r() in libnss during a Raft leader election storm (see KUDU-2395).

Improved rowset tree pruning making scans with open-ended intervals on primary key (see KUDU-2566).

The kudu perf loadgen tool now supports generating range-partitioned tables. The -table_num_buckets configuration is now removed in favor of -table_num_hash_partitions and -table_num_range_partitions (see KUDU-1861).

CFile checksum failures will now cause the affected tablet replicas to be failed and re-replicated elsewhere (see KUDU-2469).

Servers are now able to start up with data directories missing on disk (see KUDU-2359).

The kudu perf loadgen tool now creates tables with a period-separated database name, for example default.loadgen_auto_abc123. This new behavior does not take effect if the --table flag is provided. The database of the table can be changed using a new --auto_database flag. This change is made in anticipation of an eventual Kudu/HMS integration (see KUDU-2191).

Introduced FAILED_UNRECOVERABLE replica health status. This is to mark replicas which are not able to catch up with the leader due to GC-collected segments of WAL and other unrecoverable cases like disk failure. With that, the replica management scheme becomes hybrid: the system evicts replicas with FAILED_UNRECOVERABLE health status before adding a replacement if it anticipates that it can commit the transaction, while in other cases it first adds a non-voter replica and removes the failed one only after promoting a newly added replica to voter role.

Two additional configuration parameters, socketReadTimeoutMs and scanRequestTimeout have been added to the Spark connector to allow better tuning to avoid scan timeouts under high load.

The kudu table tool now supports two new options to rename tables and columns, rename_table and rename_column respectively.

Kudu will now wait for the clock to become synchronized at startup, controlled by a new flag -ntp_initial_sync_wait_secs (see KUDU-2242).

Tablet deletions are now throttled, which will help Kudu clusters remain stable even when many tablets are deleted at once. The number of tablets that a tablet server will delete at once is controlled by the new flag -num_tablets_to_delete_simultaneously (see KUDU-2289).

The kudu cluster ksck tool has been significantly enhanced. It now checks master health and consensus status, displays any unsafe or hidden flags set in the cluster, and produces a summary of the Kudu versions running on the master and tablet servers. In addition, it now supports JSON output, both in pretty-printed and compact form. The output format is controlled by the -ksck_format flag.

When a tablet server was wiped and recreated with the same RPC address, ksck listed it twice, both as healthy, even though only one of them was there. This bug is now fixed by verifying the UUID of the server (see KUDU-2364).

Fixed an issue preventing Kudu from starting when using Vormetric’s encrypted filesystem (secfs2) on ext4 (see KUDU-2406).

Fixed an issue where Kudu’s block cache memory tracking (as seen on the /mem-trackers web UI page) wasn’t accounting for all of the overhead of the cache itself (see KUDU-972).

Fixed an issue where the C++ client would fail to reopen an expired scanner; instead, the client would retry in a tight loop and eventually timeout (see KUDU-2414).

When a tablet is deleted, its write-ahead log recovery directory is also deleted, if it exists (see KUDU-1038).

Fixed a tablet server crash when a tablet is scanned with two predicates on its primary key and the predicates do not overlap (see KUDU-2447).

Fixed an issue where the Kudu MapReduce connector’s KuduTableInputFormat may exhaust its scan too early (see KUDU-2525).

Fixed an issue with failed tablet copies that would cause subsequent tablet copies to crash the tablet server (see KUDU-2293).

Fixed a bug in which incorrect results would be returned in scans following a server restart (see KUDU-2463).

Fixed a bug causing a tablet server crash when a write batch request from a client failed coarse-grained authorization (see KUDU-2540).

Fixed use-after-free in case of WAL replay error (see KUDU-2509).

Fixed authentication token reacquisition in the C++ client (see KUDU-2580).

Fixed a bug where leader logged excessively when the followers fell behind (see KUDU-2322).

Fixed reporting of leader health during lifecycle transitions (see KUDU-2335).

Fixed moving single-replica tablets (see KUDU-2443).

Fixed an error that would cause the kudu CLI tool to unexpectedly exit when the connection to the master or tserver was abruptly closed.

Fixed a rare issue where system failure could leave unexpected null bytes at the end of metadata files, causing Kudu to be unable to restart (see KUDU-2260).

Fixed an issue where kudu cluster ksck running a snapshot checksum scan would use a single snapshot timestamp for all tablets. This caused the checksum process to fail if the checksum process took a long time and the number of tablets was sufficiently large. The tool should now be able to checksum tables even if the process takes many hours. (see KUDU-2179).

Kudu 1.8 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Kudu 1.0 clients may connect to servers running Kudu 1.8 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.8 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.8 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Anupama Gupta

Attila Piros

Brian McDevitt

Fengling Wang

Ferenc Szabó

Greg Solovyev

Kiyoshi Mizumaru

Shriya Gupta

Thomas Tauber-Marshall

Tigerquoll

Yao Xu

ZhangYao

helifu

jinxing64

qqchang2nd

Fixed and issue where a leader replica could report a follower’s health status as FAILED instead of FAILED_UNRECOVERABLE. In configurations where the tablet replication factor equals to the total number of tablet servers in the cluster, that lead to situations where the tablet could not be automatically recovered until a new leader was elected or corresponding tablet servers were restarted. (see KUDU-2367).

Fixed an issue where Kudu would fail to start if RLIMIT_NPROC was set to -1. (see KUDU-2377).

Fixed an issue where kudu-spark was unable to connect to secure clusters. (see KUDU-2379).

Fixed an issue where the kudu-python client would not compile in environments where __int128 is not supported. This was most commonly el6 environments. (see KUDU-2412).

Fixed an issue where unaligned loads of __int128 integers could result in a crash. (see KUDU-2378).

Fixed a bug in PartialRow.setMin that could lead to incorrect partition pruning when a decimal column is part of the tables range partition but not a part of the query predicate. (see KUDU-2416).

Fixed an equality check on decimal column predicates that could result in pruning that is too conservative.

Upgrading directly from Kudu 1.6.0 is supported and no special upgrade steps are required. A rolling upgrade of the server side will not work because the default replica management scheme changed, and running masters and tablet servers with different replica management schemes is not supported, see Incompatible Changes in Kudu 1.7.0 for details. However, mixing client and server sides of different versions is not a problem. You can still update your clients before your servers or vice versa. When upgrading to Kudu 1.7, it is required to first shut down all Kudu processes across the cluster, then upgrade the software on all servers, then restart the Kudu processes on all servers in the cluster.

The tcmalloc_contention_time metric, which previously tracked the amount of time spent in memory allocator lock contention, has been removed.

Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in the next major release.

Kudu now supports the decimal column type. The decimal type is a numeric data type with fixed scale and precision suitable for financial and other arithmetic calculations where the imprecise representation and rounding behavior of float and double make those types impractical. The decimal type is also useful for integers larger than int64 and cases with fractional values in a primary key. See Decimal Type for more details.

The strategy Kudu uses for automatically healing tablets which have lost a replica due to server or disk failures has been improved. The new re-replication strategy, or replica management scheme, first adds a replacement tablet replica before evicting the failed one. With the previous replica management scheme, the system first evicts the failed replica and then adds a replacement. The new replica management scheme allows for much faster recovery of tablets in scenarios where one tablet server goes down and then returns back shortly after 5 minutes or so. The new scheme also provides substantially better overall stability on clusters with frequent server failures. (see KUDU-1097).

The kudu fs update_dirs tool now supports removing directories. Unless the --force flag is specified, Kudu will not allow the removal of a directory across which tablets are configured to spread data. If specified, all tablet replicas configured to use that directory will fail upon starting up and be replicated elsewhere, provided a majority exists elsewhere.

Users can use the new --fs_metadata_dir to specify the directory in which to place tablet-specific metadata. It is recommended, although not necessary, that this be placed on a high-performance drive with high bandwidth and low latency, e.g. a solid-state drive. If not specified, metadata will be placed in the directory specified by --fs_wal_dir, or in the directory specified by the first entry of --fs_data_dirs if metadata already exists there from a pre-Kudu 1.7 deployment. Kudu will not automatically move existing metadata based on this configuration.

Kudu 1.7 introduces a new scan read mode READ_YOUR_WRITES. Users can specify READ_YOUR_WRITES when creating a new scanner in C++, Java and Python clients. If this mode is used, the client will perform a read such that it follows all previously known writes and reads from this client. Reads in this mode ensure read-your-writes and read-your-reads session guarantees, while minimizing latency caused by waiting for outstanding write transactions to complete. Note that this is still an experimental feature which may be stabilized in future releases.

The tablet server web UI scans dashboard (/scans) has been improved with several new features, including: showing the most recently completed scans, a pseudo-SQL scan descriptor that concisely shows the selected columns and applied predicates, and more complete and better documented scan statistics.

Kudu daemons now expose a web page /stacks which dumps the current stack trace of every thread running in the server. This information can be helpful when diagnosing performance issues.

By default, each tablet replica will now stripe data blocks across 3 data directories instead of all data directories. This decreases the likelihood that any given tablet will be affected in the event of a single disk failure. No substantial performance impact is expected due to this feature based on performance testing. This change only affects new replicas created after upgrading to Kudu 1.7.

Kudu servers previously offered the ability to enable a separate metrics log which stores periodic snapshots of all metrics available on a server. This functionality is now available as part of a more general “diagnostics log” which is enabled by default. The diagnostics log includes periodic dumps of server metrics as well as collections of thread stack traces. The default configuration ensures that no more than 640MB of diagnostics logs are retained, and typically the space consumption is significantly less due to compression. The format and contents of this log file are documented in the Administration guide.

The handling of errors in the synchronous Java client has been improved so that, when an exception is thrown, the stack trace indicates the correct location where the client function was invoked rather than a call stack of an internal worker thread. The original call stack from the worker thread is available as a “suppressed exception”.

The logging of errors in the Java client has been improved to exclude exception stack traces for expected scenarios such as failure to connect to a server in a cluster. Instead, only a single line informational message will be logged in such cases to aid in debugging.

The Java client now uses a predefined prioritized list of TLS ciphers when establishing an encrypted connection to Kudu servers. This cipher list matches the list of ciphers preferred for server-to-server communication and ensures that the most efficient and secure ciphers are preferred. When the Kudu client is running on Java 8 or newer, this provides a substantial speed-up to read and write performance.

Reporting for the kudu cluster ksck tool has been updated so tablets and tables with on-going tablet copies are shown as "recovering". Additional reporting changes have been made to make various common scenarios, particularly tablet copies, less alarming.

The performance of inserting rows containing many string or binary columns has been improved, especially in the case of highly concurrent write workloads.

By default, Spark tasks that scan Kudu will now be able to scan non-leader replicas. This allows Spark to more easily schedule kudu-spark tasks local to the data. Users can disable this behavior by passing 'leader_only' to the 'kudu.scanLocality' option."

The number of OS threads used in the steady state and during bursts of activity (such as in Raft leader elections triggered by a node failure) has been drastically reduced and should no longer exceed the value of ulimit -u. As such, it should no longer be necessary to increase the value of ulimit -u (or of /proc/sys/kernel/threads-max) in order to run a Kudu tablet server in most cases. (see KUDU-1913).

An issue where sparse column predicates could cause excessive data-block reads has been fixed. Previously in certain scans with sparsely matching predicates on multiple columns, Kudu would read and decode the same data blocks many times. The improvement typically results in a 5-10x performance increase for the affected scans. (see KUDU-2231).

The efficiency and on-disk size of large updated values has been improved. This will improve update-heavy workloads which overwrite large (1KiB+) values. (see KUDU-2253).

Fixed a scenario where the on-disk data of a tablet server was completely erased and and a new tablet server was started on the same host. This issue could prevent tablet replicas previously hosted on the server from being evicted and re-replicated. Tablets now immediately evict replicas that respond with a different server UUID than expected. (see KUDU-1613).

Fixed a rare race condition when connecting to masters during their startup which might cause a client to get a response without a CA certificate and/or authentication token. This would cause the client to fail to authenticate with other servers in the cluster. The leader master now always sends a CA certificate and an authentication token (when applicable) to a Kudu client with a successful ConnectToMaster response. (see KUDU-1927).

The Kudu Java client now will retry a connection if no master is discovered as a leader, and the user has a valid authentication token. This avoids failure in recoverable cases when masters are in the process of the very first leader election after starting up. (see KUDU-2262).

The Java client will now automatically attempt to re-acquire Kerberos credentials from the ticket cache when the prior credentials are about to expire. This allows client instances to persist longer than the expiration time of a single Kerberos ticket so long as some other process renews the credentials in the ticket cache. Documentation on interacting with Kerberos authentication has been added to the Javadoc for the AsyncKuduClient class. (see KUDU-2264).

Follower masters are now able to verify authentication tokens even if they have never been a leader. Prior to this fix, if a follower master had never been a leader, clients would be unable to authenticate to that master, resulting in spurious error messages being logged. (see KUDU-2265).

Fixed a tablet server crash when a tablet replica is deleted during a scan. (see KUDU-2295).

The evaluation order of predicates in scans with multiple predicates has been made deterministic. Due to a bug, this was not necessarily the case previously. Predicates are applied in most to least selective order, with ties broken by column index. The evaluation order may change in the future, particularly when better column statistics are made available internally. (see KUDU-2312).

Previously, the kudu tablet change_config move_replica tool required all tablet servers in the cluster to be available when performing a move. This restriction has been relaxed: only the tablet server that will receive a replica of the tablet being moved and the hosts of the tablet’s existing replicas need to be available for the move to occur. (see KUDU-2331).

Fixed a bug in the Java client which prevented the client from locating the new leader master after a leader failover in the case that the previous leader either remained online or restarted quickly. This bug resulted in the client timing out operations with errors indicating that there was no leader master. (see KUDU-2343).

The Unix process username of the client is now included inside the exported security credentials, so that the effective username of clients who import credentials and subsequently use unauthenticated (SASL PLAIN) connections matches the client who exported the security credentials. For example, this is useful to let the Spark executors know which username to use if the Spark driver has no authentication token. This change only affects clusters with encryption disabled using --rpc_encryption=disabled. (see KUDU-2259).

Kudu 1.7 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.6 and Kudu 1.7 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.7 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.7 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.7 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

The newly introduced replica management scheme is not compatible with the old scheme, so it’s not possible to run pre-1.7 Kudu masters with 1.7 Kudu tablet servers or vice versa. This is a server-side incompatibility only and it does not affect client compatibility. In other words, Kudu clients of prior versions are compatible with upgraded Kudu clusters.

The format of the previously-optional metrics log has changed to include a human-readable timestamp on each line. The path of the log file has also changed with the word “diagnostics” replacing the word “metrics” in the file name. The metrics log has been optimized to only include those metrics which have changed in between successive samples, and to not include entity attributes such as tablet partition information in the log. (see KUDU-2297).

Upgrading directly from Kudu 1.5.0 is supported and no special upgrade steps are required. A rolling upgrade may work, however it has not been tested. When upgrading Kudu, it is recommended to first shut down all Kudu processes across the cluster, then upgrade the software on all servers, then restart the Kudu processes on all servers in the cluster.

Support for Spark 1 (kudu-spark_2.10) has been removed in Kudu 1.6.0 and now only Spark 2 is supported. Spark 1 support was deprecated in Kudu 1.5.0.

Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in the next major release.

Tablet servers' tolerance of disk failures is now enabled by default and has been extended to handle data directory failures at runtime. In the event of a disk failure at runtime, any tablets with data on a failed disk will be shut down and restarted on another tablet server. There is a configurable tradeoff between a newly added tablet’s tolerance to disk failures and its ability to parallelize reads via the experimental --fs_target_data_dirs_per_tablet flag. Tablets that are spread across fewer disks are less likely to be affected by a disk failure, at the cost of reduced parallelism. By default, tablets are striped across all available disks. Note that the first configured data directory and the WAL directory cannot currently tolerate disk failures. This will be further improved in future Kudu releases.

Kudu servers can now adopt new data directories via the new kudu fs update_dirs tool. The new directory will be used by new tablet replicas only. Note that removing directories is not yet supported (see KUDU-2202).

Kudu servers have two new flags to control webui TLS/HTTPS settings: --webserver_tls_ciphers and --webserver_tls_min_protocol. These flags allow the advertised TLS ciphers and TLS protocol versions to be configured. Additionally, the webserver now excludes insecure legacy ciphers by default (see KUDU-2190).

Kudu servers can now tolerate short interruptions in NTP clock synchronization. NTP synchronization is still required when any Kudu daemon starts up. If NTP synchronization is not available, diagnostic information is now logged to help pinpoint the issue (see KUDU-1578).

Tablet server startup time has been improved significantly on servers containing large numbers of blocks.

The log block manager now performs disk data deletion in batches. This optimization can significantly reduce the time taken to delete data on a tablet.

The usage of sensitive data redaction flag has been slightly changed. By setting --redact=log flag, redaction will be disabled in the web UI but retained for server logs. Alternatively, --redact=none can be used to disable redaction completely.

The Spark DataSource integration now can take advantage of scan locality for better scan performance, the scan will take place at the closest replica instead of going to the leader.

Various optimizations were made to reduce the 99th percentile latency of writes on the tablet server. This can also improve throughput on certain write workloads, particularly on larger clusters.

Kudu may now be configured to ignore system-wide auth_to_local mappings configured in /etc/krb5.conf by setting the configuration flag --use_system_auth_to_local=false (see KUDU-2198).

The performance of the compaction scheduler has been improved. In previous versions, certain types of time series workloads were found to cause compaction scheduling to take tens of seconds. These workloads now schedule compactions an order of magnitude more efficiently.

The compaction scheduler has been improved to avoid running a compaction when the benefit of that compaction is extremely small.

Tablet servers now consider the health of all replicas of a tablet before deciding to evict one. This can improve stability of the Kudu cluster after experiencing multiple simultaneous daemon failures (see KUDU-2048).

Several performance improvements have been made to the Kudu master, particularly in concurrency of clients opening tables. This should improve performance in highly concurrent workloads.

The on-disk size metric for a tablet now includes all data and metadata. Previously, it excluded WAL segments and consensus metadata (see KUDU-1755).

Added verbose mode for the 'kudu cluster ksck' command to enable output of detailed information on the cluster’s metadata, even when no errors are detected.

HybridTime timestamp propagation now works in the Java client when using scan tokens (see KUDU-1411).

Fixed an error message commonly found in tablet server logs indicating that operations were being read "from the future" (see KUDU-1078).

Tombstoned tablets no longer report metrics (see KUDU-2044).

Fixed a bug in the C++ client which could cause tablets to be erroneously pruned, or skipped, during certain scans, resulting in fewer results than expected being returned from queries. The bug only affected tables whose range partition columns are a proper prefix of the primary key (see KUDU-2173).

Published Kudu Java artifacts are now fully compatible with JRE 7 and JRE 8. There was previously a bug in the release process which made them compatible only with JRE 8 (see KUDU-2188).

Fixed a typo in the list of default TLS ciphers used by Kudu servers. As a result, two additional cipher suites are now available:

Kudu 1.6 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.5 and Kudu 1.6 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.6 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.6 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.6 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Kudu 1.5 now enables the optional ability to compute, store, and verify checksums on all pieces of data stored on a server by default. Due to storage format changes, downgrading to versions 1.3 or earlier is not supported and will result in an error.

Spark 2.2+ requires Java 8 at runtime even though Kudu Spark 2.x integration is Java 7 compatible. Spark 2.2 is the default dependency version as of Kudu 1.5.0.

The kudu-spark-tools module has been renamed to kudu-spark2-tools_2.11 in order to include the Spark and Scala base versions. This matches the pattern used in the kudu-spark module and artifacts.

To improve security, world-readable Kerberos keytab files are no longer accepted by default. Set --allow_world_readable_credentials=true to override this behavior. See KUDU-1955 for additional details.

Support for Java 7 is deprecated as of Kudu 1.5.0 and may be removed in the next major release.

Support for Spark 1 (kudu-spark_2.10) is deprecated as of Kudu 1.5.0 and may be removed in the next minor release.

Tablet servers are now optionally able to tolerate disk failures at startup. This feature is experimental; by default, Kudu will crash if it experiences a disk failure. When enabled, tablets with any data on the failed disk will not be opened and will be replicated as needed. To enable this, set the --crash_on_eio flag to false. Additionally, there is a configurable tradeoff between a newly added tablet’s tolerance to disk failures and its parallelization of I/O via the --fs_target_data_dirs_per_tablet flag. Tablets that are spread across fewer disks are less likely to be affected by a disk failure, at the cost of reduced parallelism. Note that the first configured data directory and the WAL directory cannot currently tolerate disk failures, and disk failures during run-time are still fatal.

Kudu server web UIs have a new configuration dashboard (/config) which provides a high level summary of important security configuration values, such as whether RPC authentication is required, or web server HTTPS encryption is enabled. Other types of configuration will be added in future releases.

The kudu command line tool has two new features: kudu tablet change_config move_replica and kudu local_replica data_size. The 'tablet change_config move_replica' tool moves a tablet replica from one tablet server to another, under the condition that the tablet is healthy. An operator can use this tool to rebalance tablet replicas between tablet servers. The 'local_replica data size' tool summarizes the space usage of a tablet, breaking it down by type of file, column, and rowset.

kudu-client-tools now supports exporting CSV files and importing Apache Parquet files. This feature is unstable and may change APIs and functionality in future releases.

kudu-spark-tools now supports importing and exporting CSV, Apache Avro and Apache Parquet files. This feature is unstable and may change APIs and functionality in future releases.

The log block manager now performs disk synchronization in batches. This optimization can significantly reduce the time taken to copy tablet data from one server to another; in one case tablet copy time is reduced by 35%. It also improves the general performance of flushes and compactions.

A new feature referred to as "tombstoned voting" is added to the Raft consensus subsystem to allow tablet replicas in the TABLET_DATA_TOMBSTONED state to vote in tablet leader elections. This feature increases Kudu’s stability and availability by improving the likelihood that Kudu will be able to self-heal in more edge-case scenarios, such as when tablet copy operations fail. See KUDU-871 for details.

The tablet on-disk size metric has been made more accurate. Previously, the metric included only REDO deltas; it now counts all deltas. Additionally, the metric includes the size of bloomfiles, ad hoc indexes, and the tablet superblock. WAL segments and consensus metadata are still not counted. The latter is very small compared to the size of data, but the former may be significant depending on the workload (this will be resolved in a future release).

The number of threads used by the Kudu tablet server has been further reduced. Previously, each follower tablet replica used a dedicated thread to detect leader tablet replica failures, and each leader replica used one dedicated thread per follower to send Raft heartbeats to that follower. The work performed by these dedicated threads has been reassigned to other threads. Other improvements were made to facilitate better thread sharing by tablets. For the purpose of capacity planning, expect the Kudu tablet server to create one thread for every five "cold" (i.e. those not servicing writes) tablets, and an additional three threads for every "hot" tablet. This will be further improved upon in future Kudu releases.

The Java Kudu client now automatically requests new authentication tokens after expiration. As a result, long-lived Java clients are now supported. See KUDU-2013 for more details.

Multiple Kerberos compatibility bugs have been fixed, including support for environments with disabled reverse DNS, FreeIPA compatibility, principal names including uppercase characters, and hosts without a FQDN.

A bug in the binary prefix decoder which could cause a tablet server 'check' assertion crash has been fixed. The crash could only be triggered in very specific scenarios; see KUDU-2085 for additional details.

Kudu 1.5 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Rolling upgrade between Kudu 1.4 and Kudu 1.5 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

Kudu 1.0 clients may connect to servers running Kudu 1.5 with the exception of the below-mentioned restrictions regarding secure clusters.

If a Kudu 1.5 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.5 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

The Maintenance Manager now fully uses the threads it’s given (see the improvements described further below), so it’s now able to generate a lot more IO by flushing and compacting more often. Generally, the recommended ratio of MM threads to data directories is 1:3; operators of clusters above that ratio should be mindful of this when upgrading.

The C++ and Java client libraries now support the ability to alter the storage attributes (e.g. encoding and compression) and default value of existing columns. Additionally, it is now possible to rename a column which is part of a table’s primary key.

The C++ client library now includes an experimental KuduPartitioner API which may be used to efficiently map rows to their associated partitions and hosts. This may be used to achieve better locality or distribution of writes in client applications.

The Java client library now supports enabling fault tolerance on scanners. Fault tolerant scanners are able to transparently recover from concurrent server crashes at the cost of some performance overhead. See the Java API documentation for more details on usage.

The kudu command line tool now includes a new advanced administrative command kudu remote_replica unsafe_change_config. This command may be used to force a tablet to perform an unsafe change of its Raft replication configuration. This can be used to recover from scenarios such as a loss of a majority of replicas, at the risk of losing edits.

The kudu command line tool now includes the kudu fs check command which performs various offline consistency checks on the local on-disk storage of a Kudu Tablet Server or Master. In addition to detecting various inconsistencies or corruptions, it can also detect and remove data blocks that are no longer referenced by any tablet but were not fully removed from disk due to a crash or a bug in prior versions of Kudu.

The kudu command line tool can now be used to list the addresses and identifiers of the servers in the cluster using either kudu master list or kudu tserver list.

Kudu 1.4 now includes the optional ability to compute, store, and verify checksums on all pieces of data stored on a server. Prior versions only performed checksums on certain portions of the stored data. This feature is not enabled by default since it makes a backward-incompatible change to the on-disk formats and thus prevent downgrades. Kudu 1.5 will enable the feature by default.

kudu cluster ksck now detects and reports new classes of inconsistencies and issues. In particular, it is better able to detect cases where a configuration change such as a replica eviction or addition is pending but is unable to be committed. It also now properly detects and reports cases where a tablet has no elected leader.

The default size for Write Ahead Log (WAL) segments has been reduced from 64MB to 8MB. Additionally, in the case that all replicas of a tablet are fully up to date and data has been flushed from memory, servers will now retain only a single WAL segment rather than two. These changes are expected to reduce the average consumption of disk space on the configured WAL disk by 16x, as well as improve the startup speed of tablet servers by reducing the number and size of WAL segments that need to be re-read.

The default on-disk storage system used by Kudu servers (Log Block Manager) has been improved to compact its metadata and remove dead containers. This compaction and garbage collection occurs only at startup. Thus, the first startup after upgrade is expected to be longer than usual, and subsequent restarts should be shorter.

The usability of the Kudu web interfaces has been improved, particularly for the case where a server hosts many tablets or a table has many partitions. Pages that list tablets now include a top-level summary of tablet status and show the complete list under a toggleable section.

The Maintenance Manager has been improved to improve utilization of the configured maintenance threads. Previously, maintenance work would only be scheduled a maximum of 4 times per second, but now maintenance work will be scheduled immediately whenever any configured thread is available. This can improve the throughput of write-heavy workloads.

The Maintenance Manager will now aggressively schedule flushes of in-memory data when memory consumption crosses 60% of the configured process-wide memory limit. The backpressure mechanism which begins to throttle client writes has been accordingly adjusted to not begin throttling until reaching 80% of the configured limit. These two changes together result in improved write throughput, more consistent latency, and fewer timeouts due to memory exhaustion.

Many performance improvements were made to write performance. Applications which send large batches of writes to Kudu should see substantially improved throughput in Kudu 1.4.

Several improvements were made to reduce the memory consumption of Kudu Tablet Servers which hold large volumes of data. The specific amount of memory saved varies depending on workload, but the expectation is that approximately 350MB of excess peak memory usage has been eliminated per TB of data stored.

The number of threads used by the Kudu Tablet Server has been reduced. Previously, each tablet used a dedicated thread to append to its WAL. Those threads now automatically stop running if there is no activity on a given tablet for a short period of time.

KUDU-2020 Fixed an issue where re-replication after a failure would proceed significantly slower than expected. This bug caused many tablets to be unnecessarily copied multiple times before successfully being considered re-replicated, resulting in significantly more network and IO bandwidth usage than expected. Mean time to recovery on clusters with large amounts of data is improved by up to 10x by this fix.

KUDU-1982 Fixed an issue where the Java client would call NetworkInterface.getByInetAddress very often, causing performance problems particularly on Windows where this function can be quite slow.

KUDU-1755 Improved the accuracy of the on_disk_size replica metrics to include the size consumed by bloom filters, primary key indexes, and superblock metadata, and delta files. Note that, because the size metric is now more accurate, the reported values are expected to increase after upgrading to Kudu 1.4. This does not indicate that replicas are using more space after the upgrade; rather, it is now accurately reporting the amount of space that has always been used.

KUDU-1192 Kudu servers will now periodically flush their log messages to disk even if no WARNING-level messages have been logged. This makes it easier to tail the logs to see progress output during normal startup.

KUDU-1999 Fixed the ability to run Spark jobs in "cluster" mode against Kudu clusters secured by Kerberos.

Kudu 1.4 clients may connect to servers running Kudu 1.0 or later. If the client uses features that are not available on the target server, an error will be returned.

Kudu 1.0 clients may connect to servers running Kudu 1.4 with the exception of the below-mentioned restrictions regarding secure clusters.

Rolling upgrade between Kudu 1.3 and Kudu 1.4 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

If a Kudu 1.4 cluster is configured with authentication or encryption set to "required", clients older than Kudu 1.3 will be unable to connect.

If a Kudu 1.4 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Kudu servers, by default, will now only allow unencrypted or unauthenticated connections from trusted subnets, which are private networks (127.0.0.0/8,10.0.0.0/8,172.16.0.0/12, 192.168.0.0/16,169.254.0.0/16) and local subnets of all local network interfaces. Unencrypted or unauthenticated connections from publicly routable IPs will be rejected, even if encryption and authentication are not configured.

Kudu 1.3 adds support for strong authentication based on Kerberos. This optional feature allows users to authenticate themselves using Kerberos tickets, and also provides mutual authentication of servers using Kerberos credentials stored in keytabs. This feature is optional, but recommended for deployments requiring security.

Kudu 1.3 adds support for encryption of data on the network using Transport Layer Security (TLS). Kudu will now use TLS to encrypt all network traffic between clients and servers as well as any internal traffic among servers, with the exception of traffic determined to be within a localhost network connection. Encryption is enabled by default whenever it can be determined that both the client and server support the feature.

Kudu 1.3 adds coarse-grained service-level authorization of access to the cluster. The operator may set up lists of permitted users who may act as administrators and as clients of the cluster. Combined with the strong authentication feature described above, this can enable a secure environment for some use cases. Note that fine-grained access control (e.g. table-level or column-level) is not yet supported.

Kudu 1.3 adds a background task to tablet servers which removes historical versions of data which have fallen behind the configured data retention time. This reduces disk space usage in all workloads, but particularly in those with a higher volume of updates or upserts.

Kudu now incorporates Google Breakpad, a library which writes crash reports in the case of a server crash. These reports can be found within the configured log directory, and can be useful during bug diagnosis.

Kudu servers will now change the file permissions of data directories and contained data files based on a new configuration flag --umask. As a result, after upgrading, permissions on disk may be more restrictive than in previous versions. The new default configuration improves data security.

Kudu’s web UI will now redact strings which may include sensitive user data. For example, the monitoring page which shows in-progress scans no longer includes the scanner predicate values. The tracing and RPC diagnostics endpoints no longer include contents of RPCs which may include table data.

By default, Kudu now reserves 1% of each configured data volume as free space. If a volume is seen to have less than 1% of disk space free, Kudu will stop writing to that volume to avoid completely filling up the disk.

The default encoding for numeric columns (int, float, and double) has been changed to BIT_SHUFFLE. The default encoding for binary and string columns has been changed to DICT_ENCODING. Dictionary encoding automatically falls back to the old default (PLAIN) when cardinality is too high to be effectively encoded.

Kudu now uses LZ4 compression when writing its Write Ahead Log (WAL). This improves write performance and stability for many use cases.

Kudu now uses LZ4 compression when writing delta files. This can improve both read and write performance as well as save substantial disk usage, especially for workloads involving a high number of updates or upserts containing compressible data.

The Kudu API now supports the ability to express IS NULL and IS NOT NULL predicates on scanners. The Spark DataSource integration will take advantage of these new predicates when possible.

Both C++ and Java clients have been optimized to prune partitions more effectively when performing scans using the IN (…​) predicate.

The exception messages produced by the Java client are now truncated to a maximum length of 32KB.

KUDU-1893 Fixed a critical bug in which wrong results would be returned when evaluating predicates applied to columns added using the ALTER TABLE operation.

KUDU-1905 Fixed a crash after inserting a row sharing a primary key with a recently-deleted row in tables where the primary key is comprised of all of the columns.

KUDU-1899 Fixed a crash after inserting a row with an empty string as the single-column primary key.

KUDU-1904 Fixed a potential crash when performing random reads against a column using RLE encoding and containing long runs of NULL values.

KUDU-1853 Fixed an issue where disk space could be leaked on servers which experienced an error during the process of copying tablet data from another server.

KUDU-1856 Fixed an issue in which disk space could be leaked by Kudu servers storing data on partitions using the XFS file system. Any leaked disk space will be automatically recovered upon upgrade.

KUDU-1888, KUDU-1906 Fixed multiple issues in the Java client where operation callbacks would never be triggered, causing the client to hang.

Kudu 1.3 clients may connect to servers running Kudu 1.0. If the client uses features that are not available on the target server, an error will be returned.

Kudu 1.0 clients may connect to servers running Kudu 1.3 with the exception of the below-mentioned restrictions regarding secure clusters.

Rolling upgrade between Kudu 1.2 and Kudu 1.3 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

If a Kudu 1.3 cluster is configured with authentication or encryption set to "required", older clients will be unable to connect.

If a Kudu 1.3 cluster is configured with authentication and encryption set to "optional" or "disabled", older clients will still be able to connect.

Due to storage format changes in Kudu 1.3, downgrade from Kudu 1.3 to earlier versions is not supported. After upgrading to Kudu 1.3, attempting to restart with an earlier version will result in an error.

In order to support running MapReduce and Spark jobs on secure clusters, these frameworks now connect to the cluster at job submission time to retrieve authentication credentials which can later be used by the tasks to be spawned. This means that the process submitting jobs to Kudu clusters must have direct access to that cluster.

The embedded web servers in Kudu processes now specify the X-Frame-Options: DENY HTTP header which prevents embedding Kudu web pages in HTML iframe elements.

Kudu clients and servers now redact user data such as cell values from log messages, Java exception messages, and Status strings. User metadata such as table names, column names, and partition bounds are not redacted.

Kudu’s ability to provide consistency guarantees has been substantially improved:

Kudu servers now automatically limit the number of log files. The number of log files retained can be configured using the max_log_files flag. By default, 10 log files will be retained at each severity level.

The logging in the Java and C++ clients has been substantially quieted. Clients no longer log messages in normal operation unless there is some kind of error.

The C++ client now includes a KuduSession::SetErrorBufferSpace API which can limit the amount of memory used to buffer errors from asynchronous operations.

The Java client now fetches tablet locations from the Kudu Master in batches of 1000, increased from batches of 10 in prior versions. This can substantially improve the performance of Spark and Impala queries running against Kudu tables with large numbers of tablets.

Table metadata lock contention in the Kudu Master was substantially reduced. This improves the performance of tablet location lookups on large clusters with a high degree of concurrency.

Lock contention in the Kudu Tablet Server during high-concurrency write workloads was also reduced. This can reduce CPU consumption and improve performance when a large number of concurrent clients are writing to a smaller number of a servers.

Lock contention when writing log messages has been substantially reduced. This source of contention could cause high tail latencies on requests, and when under high load could contribute to cluster instability such as election storms and request timeouts.

The BITSHUFFLE column encoding has been optimized to use the AVX2 instruction set present on processors including Intel® Sandy Bridge and later. Scans on BITSHUFFLE-encoded columns are now up to 30% faster.

The kudu tool now accepts hyphens as an alternative to underscores when specifying actions. For example, kudu local-replica copy-from-remote may be used as an alternative to kudu local_replica copy_from_remote.

KUDU-1508 Fixed a long-standing issue in which running Kudu on ext4 file systems could cause file system corruption.

KUDU-1399 Implemented an LRU cache for open files, which prevents running out of file descriptors on long-lived Kudu clusters. By default, Kudu will limit its file descriptor usage to half of its configured ulimit.

Gerrit #5192 Fixed an issue which caused data corruption and crashes in the case that a table had a non-composite (single-column) primary key, and that column was specified to use DICT_ENCODING or BITSHUFFLE encodings. If a table with an affected schema was written in previous versions of Kudu, the corruption will not be automatically repaired; users are encouraged to re-insert such tables after upgrading to Kudu 1.2 or later.

Gerrit #5541 Fixed a bug in the Spark KuduRDD implementation which could cause rows in the result set to be silently skipped in some cases.

KUDU-1551 Fixed an issue in which the tablet server would crash on restart in the case that it had previously crashed during the process of allocating a new WAL segment.

KUDU-1764 Fixed an issue where Kudu servers would leak approximately 16-32MB of disk space for every 10GB of data written to disk. After upgrading to Kudu 1.2 or later, any disk space leaked in previous versions will be automatically recovered on startup.

KUDU-1750 Fixed an issue where the API to drop a range partition would drop any partition with a matching lower or upper bound, rather than any partition with matching lower and upper bound.

KUDU-1766 Fixed an issue in the Java client where equality predicates which compared an integer column to its maximum possible value (e.g. Integer.MAX_VALUE) would return incorrect results.

KUDU-1780 Fixed the kudu-client Java artifact to properly shade classes in the com.google.thirdparty namespace. The lack of proper shading in prior releases could cause conflicts with certain versions of Google Guava.

Gerrit #5327 Fixed shading issues in the kudu-flume-sink Java artifact. The sink now expects that Hadoop dependencies are provided by Flume, and properly shades the Kudu client’s dependencies.

Fixed a few issues using the Python client library from Python 3.

Kudu 1.2 clients may connect to servers running Kudu 1.0. If the client uses features that are not available on the target server, an error will be returned.

Kudu 1.0 clients may connect to servers running Kudu 1.2 without limitations.

Rolling upgrade between Kudu 1.1 and Kudu 1.2 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

The replication factor of tables is now limited to a maximum of 7. In addition, it is no longer allowed to create a table with an even replication factor.

The GROUP_VARINT encoding is now deprecated. Kudu servers have never supported this encoding, and now the client-side constant has been deprecated to match the server’s capabilities.

The Python client has been brought up to feature parity with the Java and C++ clients and as such the package version will be brought to 1.1 with this release (from 0.3). A list of the highlights can be found below.

IN LIST predicate pushdown support was added to allow optimized execution of filters which match on a set of column values. Support for Spark, Map Reduce and Impala queries utilizing IN LIST pushdown is not yet complete.

The Java client now features client-side request tracing in order to help troubleshoot timeouts. Error messages are now augmented with traces that show which servers were contacted before the timeout occurred instead of just the last error. The traces also contain RPCs that were required to fulfill the client’s request, such as contacting the master to discover a tablet’s location. Note that the traces are not available for successful requests and are not programmatically queryable.

Kudu now publishes JAR files for Spark 2.0 compiled with Scala 2.11 along with the existing Spark 1.6 JAR compiled with Scala 2.10.

The Java client now allows configuring scanners to read from the closest replica instead of the known leader replica. The default remains the latter. Use the relevant ReplicaSelection enum with the scanner’s builder to change this behavior.

Tablet servers use a new policy for retaining write-ahead log (WAL) segments. Previously, servers used the 'log_min_segments_to_retain' flag to prioritize any flushes which were retaining log segments past the configured value (default 2). This policy caused servers to flush in-memory data more frequently than necessary, limiting write performance.

Kudu’s implementation of the Raft consensus algorithm has been improved to include a "pre-election" phase. This can improve the stability of tablet leader election in high-load scenarios, especially if each server hosts a high number of tablets.

Tablet server start-up time has been substantially improved in the case that the server contains a high number of tombstoned tablet replicas.

Kudu 1.1 clients may connect to servers running Kudu 1.0. If the client uses the new 'IN LIST' predicate type, an error will be returned.

Kudu 1.0 clients may connect to servers running Kudu 1.1 without limitations.

Rolling upgrade between Kudu 1.0 and Kudu 1.1 servers is believed to be possible though has not been sufficiently tested. Users are encouraged to shut down all nodes in the cluster, upgrade the software, and then restart the daemons on the new version.

The Java client’s CreateTableBuilder and AlterTableBuilder classes have been renamed to CreateTableOptions and AlterTableOptions. Their methods now also return this objects, allowing them to be used as builders.

The Java client’s AbstractKuduScannerBuilder#maxNumBytes() setter is now called batchSizeBytes as is the corresponding property in AsyncKuduScanner. This makes it consistent with the C++ client.

The "kudu-admin" tool can now list and delete tables via its new subcommands "list_tables" and "delete_table <table_name>".

OSX is now supported for single-host development. Please consult its specific installation instructions in OS X.