Wrangler command-line directives

This page describes the directives you can use in Wrangler.

In Wrangler Power mode (the Wrangler CLI), you can add directives and functions to a recipe. For more information, see Wrangler CLI Directives and Wrangler Functions .

Wrangler supports the following directives:

Column operations

Directive	Description
Change Column Case	Changes column names to either lowercase or uppercase.
Changing Case	Changes the case of column values.
Cleanse Column Names	Sanitizes column names, following specific rules.
Columns Replace	Alters column names in bulk.
Copy	Copies values from a source column into a destination column.
Drop Column	Drops a column in a record.
Fill Null or Empty	If the value is null or empty, enters a fixed column value.
Keep	Keeps specified columns from the record.
Merge	Merges two columns by inserting a third column.
Rename	Renames an existing column in the record.
Set Headers	Sets the names of columns, in the order they're specified.
Split to Columns	Splits a column based on a separator into multiple columns.
Swap	Swaps column names of two columns.
Set Type	Converts the data type of a column.

Date transformations

Directive	Description
Diff Date	Calculates the difference between two dates.
Format Date	Custom patterns for date-time formatting.
Format UNIX Timestamp	Formats a UNIX timestamp as a date.

Datetime transformations

Directive	Description
Current Datetime	Generates current datetime with a given time zone.
Datetime to Timestamp	Creates a timestamp from datetime and given time zone.
Format Datetime	Formats datetime to a string of a given format.
Timestamp to Datetime	Converts a timestamp to datetime.

Encoders and decoders

Directive	Description
Decode	Decodes a column value as one of base32 , base64 , or hex.
Encode	Encodes a column value as one of base32 , base64 , or hex.

Hashing and masking

Directive	Description
Hash	Generates a message digest.
Mask Number	Applies substitution masking on the column values.
Mask Shuffle	Applies shuffle masking on the column values.

Lookups

Directive	Description
Catalog Lookup	Static catalog lookup of ICD-9, ICD-10-2016, and ICD-10-2017 codes.
Table Lookup	Performs lookups into Table datasets.

Natural language processing

Directive	Description
Stemming Tokenized Words	Applies the Porter stemming algorithm for English words.

Output Formatters

Directive	Description
Write as CSV	Converts a record into CSV format.
Write as JSON Map	Converts the record into a JSON map.
Write JSON Object	Composes a JSON object based on the fields specified.
Format as Currency	Formats a number as currency as specified by locale.

Parsers

Directive	Description
JSON Path	Uses a DSL (a JSON path expression) for parsing JSON records.
Parse as AVRO File	Parses an AVRO data file.
Parse as CSV	Parses an input record as comma-separated values
Parse as Currency	Parses a currency value that is a string representation of local currency into a number.
Parse as Datetime	Parses string as datetime data type with a given format.
Parse as Excel	Parses into a Microsoft Excel file.
Parse as Fixed Length	Parses as a fixed length record with specified widths.
Parse as HL7	Parses Health Level 7 Version 2 (HL7 V2) messages.
Parse as JSON	Parses a JSON object.
Parse as Log	Parses access log files, such as Apache httpd and NGINX servers.
Parse as Simple Date	Parses date strings.
Parse as Timestamp	Parses column values representing unix timestamp as date.
Parse XML To JSON	Parses an XML document into a JSON structure.

Row operations

Directive	Description
Filter Rows	Filters records based on a condition.
Filter Row if Matched	Filters rows that match a pattern for a column.
Flatten	Separates the elements in a repeated field.
Fail on condition	Fails processing when the condition is evaluated to true.
Send to Error	Filters records to an error collector.
Send to Error and Continue Processing	Filters records to an error collector and continues processing.
Split to Rows	Splits based on a separator into multiple records.

Transformations

Directive	Description
Changing Case	Changes the case of column values to uppercase or lowercase characters.
Create Record	Creates a record column with nested values by copying values from source columns into a destination column.
Cut Character	Selects parts of a string value.
Set Column	Sets the column value to the result of an expression execution.
Find and Replace	Transforms string column values using a "sed"-like expression.
Quantization	Applies quantization to column values.
Extract Regex Groups	Extracts the data from a group of regular expressions into its own column.
Set Charset	Sets the encoding and then converts the data to a UTF-8 String.
Set Record Delim	Sets the record delimiter.
Split Email	Splits an email ID into an account and its domain.
Split URL	Splits a URL into its constituents.
Text Distance (Fuzzy String Match)	Measures the difference between two sequences of characters.
Text Metric (Fuzzy String Match)	Measures the difference between two sequences of characters.
URL Decode	Decodes from the application/x-www-form-urlencoded MIME format.
URL Encode	Encodes to the application/x-www-form-urlencoded MIME format.
Trimming Spaces	Functions for trimming white spaces around string data.

Transient aggregators and setters

Directive	Description
Increment Variable	Increments a transient variable with a record of processing.
Set Variable	Sets a transient variable with a record of processing.

Unique ID

Directive	Description
Generate UUID	Generates a universally unique identifier (UUID).

What's next

• Learn more about Wrangler functions .