@InterfaceAudience.Public @InterfaceStability.Evolving public class RegexpKuduOperationsProducer extends Object implements KuduOperationsProducer
Insert
or
Upsert
per Event
by parsing the payload into values using a
regular expression. Values are coerced to the proper column types.
Example: if the Kudu table has the schema
key INT32
name STRING
and producer.pattern is '(?<key>\\d+),(?<name>\w+)', then the
RegexpKuduOperationsProducer will parse the string
|12345,Mike||54321,Todd|
into the rows (key=12345, name=Mike) and (key=54321, name=Todd).
Note: this class relies on JDK7 named capturing groups, which are documented
in Pattern
.
Regular Expression Kudu Operations Producer configuration parameters
Property Name | Default | Required? | Description |
---|---|---|---|
producer.pattern | Yes | The regular expression used to parse the event body. | |
producer.charset | utf-8 | No | The charset of the event body. |
producer.operation | upsert | No | Operation type used to write the event to Kudu. Must be 'insert' or 'upsert'. |
producer.skipMissingColumn | false | No | Whether to ignore a column if it has no corresponding capture group, or instead completely abandon the attempt to parse and insert/upsert the row. |
producer.skipBadColumnValue | false | No | Whether to omit a column value from the row if its raw value cannot be coerced to the right type, or instead complete abandon the attempt to parse and insert/operation the row. |
producer.warnUnmatchedRows | true | No | Whether to warn about payloads that do not match the pattern. If this option is not set, event bodies with no matches will be silently dropped. |
Pattern
Modifier and Type | Field and Description |
---|---|
static String |
DEFAULT_ENCODING |
static String |
DEFAULT_OPERATION |
static boolean |
DEFAULT_SKIP_BAD_COLUMN_VALUE |
static boolean |
DEFAULT_SKIP_MISSING_COLUMN |
static boolean |
DEFAULT_WARN_UNMATCHED_ROWS |
static String |
ENCODING_PROP |
static String |
OPERATION_PROP |
static String |
PATTERN_PROP |
static String |
SKIP_BAD_COLUMN_VALUE_PROP |
static String |
SKIP_MISSING_COLUMN_PROP |
static String |
WARN_UNMATCHED_ROWS_PROP |
Constructor and Description |
---|
RegexpKuduOperationsProducer() |
Modifier and Type | Method and Description |
---|---|
void |
close()
Cleans up any state.
|
void |
configure(org.apache.flume.Context context) |
List<Operation> |
getOperations(org.apache.flume.Event event)
Returns the operations that should be written to Kudu as a result of this event.
|
void |
initialize(KuduTable table)
Initializes the operations producer.
|
public static final String PATTERN_PROP
public static final String ENCODING_PROP
public static final String DEFAULT_ENCODING
public static final String OPERATION_PROP
public static final String DEFAULT_OPERATION
public static final String SKIP_MISSING_COLUMN_PROP
public static final boolean DEFAULT_SKIP_MISSING_COLUMN
public static final String SKIP_BAD_COLUMN_VALUE_PROP
public static final boolean DEFAULT_SKIP_BAD_COLUMN_VALUE
public static final String WARN_UNMATCHED_ROWS_PROP
public static final boolean DEFAULT_WARN_UNMATCHED_ROWS
public void configure(org.apache.flume.Context context)
configure
in interface org.apache.flume.conf.Configurable
public void initialize(KuduTable table)
KuduOperationsProducer
initialize
in interface KuduOperationsProducer
table
- the KuduTable used to create Kudu Operation objectspublic List<Operation> getOperations(org.apache.flume.Event event) throws org.apache.flume.FlumeException
KuduOperationsProducer
getOperations
in interface KuduOperationsProducer
Operation
that
should be written to Kuduorg.apache.flume.FlumeException
public void close()
KuduOperationsProducer
close
in interface AutoCloseable
close
in interface KuduOperationsProducer
Copyright © 2017 The Apache Software Foundation. All rights reserved.