Configuring Apache Kudu

To configure the behavior of each Kudu process, you can pass command-line flags when you start it, or read those options from configuration files by passing them using one or more --flagfile=<file> options. You can even include the --flagfile option within your configuration file to include other files. Learn more about gflags by reading its documentation.

You can place options for masters and tablet servers into the same configuration file, and each will ignore options that do not apply.

Flags can be prefixed with either one or two - characters. This documentation standardizes on two: --example_flag.

Only the most common configuration options are documented here. For a more exhaustive list of configuration options, see the Configuration Reference.

To see all configuration flags for a given executable, run it with the --help option. Take care when configuring undocumented flags, as not every possible configuration has been tested, and undocumented options are not guaranteed to be maintained in future releases.

Kudu relies on timestamps generated by its clock implementation for the MVCC and for providing consistency guarantees when processing write and read requests. Aside from the test-only mock clock, Kudu has two different clock implementations: one is based on logical time and the other is based on so-called hybrid time. The former is a plain Lamport clock, the latter is a combination of the node’s system clock and a Lamport clock. Below, the former is referred to as LogicalClock and the latter as HybridClock.

Using the HybridClock implementation is a must for any production-grade, POC, and other regular Kudu deployments: that’s why --use_hybrid_clock is set true by default. Setting the flag to false makes Kudu servers use the LogicalClock implementation: running with such a clock implementation is acceptable only in the context of running specifically crafted test scenarios in Kudu development environment.

To provide better accuracy for multi-node cluster deployments where each node maintains its own system clock, the HybridClock implementation requires each node’s system clock to be synchronized by NTP.

For Kudu masters and tablet servers, there are two options to make the HybridClock implementation use a clock synchronized by NTP:

The latter option is provided as a last resort for deployments where properly configuring NTP daemons at every node of a Kudu cluster is not feasible for some reason and to simplify Kudu deployments in public cloud environments such as EC2 and GCP. For on-prem deployments, it’s still recommended to use the former option since the current implementation of the Kudu built-in NTP client might not be as robust as the battle-tested ntpd and chronyd system NTP daemons.

To switch between these two options above, use the --time_source flag:

If deploying a Kudu cluster in AWS/EC2 or GCE/GCP public clouds, it might make sense to set --time_source=auto for all Kudu masters and tablet servers in the cluster. In this context, setting --time_source=auto leads to the following:

The kudu cluster ksck CLI utility reports the configured and the effective time source for every Kudu master and tablet server in a cluster. The list of the NTP servers for the built-in client is reported as well when the effective time source is builtin. The utility is also able to show the difference in settings of the related time source flags and warn operators if a discrepancy is detected. In addition, the information on the configured and effective time source is reported by the embedded Web server in the Time Source panel at the /config page.

Every Kudu node requires the specification of directory flags. The --fs_wal_dir configuration indicates where Kudu will place its write-ahead logs. The --fs_metadata_dir configuration indicates where Kudu will place metadata for each tablet. It is recommended, although not necessary, that these directories be placed on a high-performance drives with high bandwidth and low latency, e.g. solid-state drives. If --fs_metadata_dir is not specified, metadata will be placed in the directory specified by --fs_wal_dir. Since a Kudu node cannot tolerate the loss of its WAL or metadata directories, it may be wise to mirror the drives containing these directories in order to make recovering from a drive failure easier; however, mirroring may increase the latency of Kudu writes.

The --fs_data_dirs configuration indicates where Kudu will write its data blocks. This is a comma-separated list of directories; if multiple values are specified, data will be striped across the directories. If not specified, data blocks will be placed in the directory specified by --fs_wal_dir. Note that while a single data directory backed by a RAID-0 array will outperform a single data directory backed by a single storage device, it is better to let Kudu manage its own striping over multiple devices rather than delegating the striping to a RAID-0 array.

Additionally, --fs_wal_dir and --fs_metadata_dir may be the same as one of the directories listed in --fs_data_dirs, but must not be sub-directories of any of them.

To see all available configuration options for the kudu-master executable, run it with the --help option:

For the full list of flags for masters, see the Kudu Master Configuration Reference.

To see all available configuration options for the kudu-tserver executable, run it with the --help option:

For the full list of flags for tablet servers, see the Kudu Tablet Server Configuration Reference.