Whistle basics
This page describes the basics of writing a mapping in Whistle to transform data from one schema to another.
For a list of built-in Whistle functions, see the Whistle built-in function reference , and for a list of MDE-specific functions, see the Whistle MDE function reference .
Simple mapping
The following section outlines some array outputs.
Add array outputs
If you use the following Whistle code sample:
Planet
[
0
]:
"Earth"
;
Planet
[
1
]:
"Mars"
;
Planet
[
2
]:
"Jupiter"
;
Moon
[
0
]:
"Luna"
;
The output would be the following:
{
"Moon"
:
[
"Luna"
],
"Planet"
:
[
"Earth"
,
"Mars"
,
"Jupiter"
]
}
To add another moon, Io
, before Luna
, edit your Whistle code as follows:
Planet
[
0
]:
"Earth"
;
Planet
[
1
]:
"Mars"
;
Planet
[
2
]:
"Jupiter"
;
Moon
[
0
]:
"Io"
;
Moon
[
1
]:
"Luna"
;
With the previous input, the output will be the following:
{
"Moon"
:
[
"Io"
,
"Luna"
],
"Planet"
:
[
"Earth"
,
"Mars"
,
"Jupiter"
]
}
Define and call functions
This section describes call functions.
Defining functions
A function in Whistle is a set of mappings that produce a JSON object. The function maps a set of inputs to a set of fields in the resulting JSON object. The following examples use a function to add structure to the planets from the previous section.
Planet
[
0
]:
PlanetName_PlanetInfo
(
"Earth"
);
Planet
[
1
]:
PlanetName_PlanetInfo
(
"Mars"
);
Planet
[
2
]:
PlanetName_PlanetInfo
(
"Jupiter"
);
Moon
[
0
]:
MoonName_MoonInfo
(
"Luna"
);
def
PlanetName_PlanetInfo
(
planetName
)
{
name
:
planetName
;
type
:
"Planet"
}
def
MoonName_MoonInfo
(
moonName
)
{
name
:
moonName
;
type
:
"Moon"
}
With the previous input, the output would be the following:
{
"Moon"
:
[
{
"name"
:
"Luna"
,
"type"
:
"Moon"
}
],
"Planet"
:
[
{
"name"
:
"Earth"
,
"type"
:
"Planet"
},
{
"name"
:
"Mars"
,
"type"
:
"Planet"
},
{
"name"
:
"Jupiter"
,
"type"
:
"Planet"
}
]
}
Calling functions
The following sample demonstrates calling functions, which is similar to C or Python.
A function call looks like FunctionName(a, b, c)
.
You can chain function calls by passing the result of one function to the next one. For
example, SingleParameterFunctionName(FunctionName(a, b, c))
.
Similarly, you can pass the results of a function into a function with multiple parameters.
For example, MultipleParamFunctionName(FunctionName(a, b, c), d)
.
The following sample function generalizes the function from the previous section
by making the celestial body's type
an input.
Planet
[
0
]:
BodyName_BodyType_BodyInfo
(
"Earth"
,
"Planet"
);
Planet
[
1
]:
BodyName_BodyType_BodyInfo
(
"Mars"
,
"Planet"
);
Planet
[
2
]:
BodyName_BodyType_BodyInfo
(
"Jupiter"
,
"Planet"
);
Moon
[
0
]:
BodyName_BodyType_BodyInfo
(
"Luna"
,
"Moon"
);
def
BodyName_BodyType_BodyInfo
(
bodyName
,
bodyType
)
{
name
:
bodyName
type
:
bodyType
}
With the previous input, the output would be the following:
{
"Moon"
:
[
{
"name"
:
"Luna"
,
"type"
:
"Moon"
}
],
"Planet"
:
[
{
"name"
:
"Earth"
,
"type"
:
"Planet"
},
{
"name"
:
"Mars"
,
"type"
:
"Planet"
},
{
"name"
:
"Jupiter"
,
"type"
:
"Planet"
}
]
}
Functions returning a primitive
Functions in Whistle can return arrays, objects, and primitives.
The following sample function sets the Primitive
field to the number 20
:
Primitive: Num_DoubleNum(10);
def Num_DoubleNum(num) 2 * num
With the previous input, the output is the following:
{
"Primitive"
:
20
}
Merge semantics
The following steps show how data objects are merged using Whistle.
Given the following mapping:
Merged: Colour_Colour_MergedColours("red", "blue")
def Colour_Colour_MergedColours(col1, col2) {
field1: "hello";
Colour_Col1(col1);
Colour_Col2(col2);
}
def Colour_Col1(col) {
colour.first: col;
colours[1]: col;
}
def Colour_Col2(col) {
colour.second: col;
colours[1]: col;
}
The output would be the following:
{
"Merged"
:
{
"colour"
:
{
"first"
:
"red"
,
"second"
:
"blue"
},
"colours"
:
[
"blue"
]
"field1"
:
"hello"
},
}
Data objects are merged as follows:
- Arrays are concatenated.
- Hardcoded array indexes are preserved, so in the previous example,
blueoverwritesred. - New fields in objects are added.
- Existing fields in objects are merged recursively.
- Primitives are overwritten.
-
nulland empty values don't overwrite existing values.
Mapping from input data
Given the following source message:
{
"Planets"
:
[
{
"name"
:
"Earth"
},
{
"name"
:
"Mars"
},
{
"name"
:
"Jupiter"
}
],
"Moons"
:
[
{
"name"
:
"Luna"
}
]
}
This data is loaded into an input called $root
. Data loading into an input to
the mapping engine is always in this $root
input. You can use $root
inside
functions, but you should avoid doing so because it can signal non-modular mappings.
Planet
[
0
]:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[
0
],
"Planet"
);
Planet
[
1
]:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[
1
],
"Planet"
);
Planet
[
2
]:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[
2
],
"Planet"
);
Moon
[
0
]:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[
0
],
"Moon"
);
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
name
:
body
.
name
type
:
bodyType
}
The output would be the following:
{
"Planet"
:
[
{
"name"
:
"Earth"
,
"type"
:
"Planet"
},
{
"name"
:
"Mars"
,
"type"
:
"Planet"
},
{
"name"
:
"Jupiter"
,
"type"
:
"Planet"
}
],
"Moon"
:
[
{
"name"
:
"Luna"
,
"type"
:
"Moon"
}
]
}
Arrays
This sections describes the different operations with arrays.
Iteration
The syntax for iterating over an array is to add a []
suffix. You can use this
syntax in the following ways:
-
Function(a[])means "pass each element ofa(one at a time) toFunction". -
Function(a[], b)means "pass each element ofa(one at a time), along withbtoFunction". Ifbis an array of the same length asa, you can iterate over them together. -
Function(a[], b[])means "pass each element ofa(one at a time), along with each element ofb(at the same index) toFunction". -
[]is also allowed after function calls:Function2(Function(a)[])means "pass each element from the result ofFunction(a)(one at a time) toFunction2". - The result of an iterating function call is also an array.
See the following example to iterative over the Planets
and Moons
arrays:
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[],
"Planet"
);
Moon
:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[],
"Moon"
);
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
name
:
body
.
name
type
:
bodyType
}
With the previous input, the output would be the following:
{
"Planet"
:
[
{
"name"
:
"Earth"
,
"type"
:
"Planet"
},
{
"name"
:
"Mars"
,
"type"
:
"Planet"
},
{
"name"
:
"Jupiter"
,
"type"
:
"Planet"
}
],
"Moon"
:
[
{
"name"
:
"Luna"
,
"type"
:
"Moon"
}
]
}
Appending to arrays
You can append to an array using []
in the middle of the path. For example, types[].typeName: ...
is valid and creates types: [{"typeName": ... }]
. You
can also use hardcoded indexes, such as types[0]: ...
. "Out of bounds"
indexes, such as types[153]: ...
, generate missing elements as null
.
Example using index numbers:
Planet
[
0
]:
"Earth"
Planet
[
1
]:
"Mars"
Planet
[
2
]:
"Jupiter"
Moon
[
0
]:
"Luna"
Example using appending:
Planet
[]:
"Earth"
Planet
[]:
"Mars"
Planet
[]:
"Jupiter"
Moon
[]:
"Luna"
Wildcards
The [*]
syntax works the same as specifying an index, except that it returns
an array of values. Multiple arrays mapped through with [*]
, for example a[*].b.c[*].d
, result in one long, non-nested array of the values of d
with
the same item order. Hardcoded array indexes are preserved.
Null values are included through jagged traversal. For example, a[*].b.c[*].d
.
If some instance of a
does not contain b.c
, then a single null value is
returned for that instance.
For example:
PlanetNames
:
$
root
.
Planets
[
*
]
.
name
;
With the previous example, the output is the following:
{
"PlanetNames"
:
[
"Earth"
,
"Mars"
,
"Jupiter"
]
}
Prepend the words "Celestial Body" to the names in PlanetNames
using what you
learned about iterating over arrays:
PlanetNames
:
AddPrefix
(
"Celestial Body "
,
$
root
.
Planets
[]);
def
AddPrefix
(
prefix
,
planet
)
{
prefix
+
planet
.
name
}
With the previous example, the output would be the following:
{
"PlanetNames"
:
[
"Celestial Body Earth"
,
"Celestial Body Mars"
,
"Celestial Body Jupiter"
]
}
Writing to array fields
The following steps show how to write to array fields:
PlanetNames
:
(
$
root
.
Planets
[
*
]
.
name
[]);
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[],
"Planet"
);
Moon
:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[],
"Moon"
);
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
name
:
body
.
name
types
[]:
bodyType
types
[]:
"Body"
}
With the previous input, the output would be the following:
{
"Moon"
:
[
{
"name"
:
"Luna"
,
"types"
:
[
"Moon"
,
"Body"
]
}
],
"Planet"
:
[
{
"name"
:
"Earth"
,
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Mars"
,
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Jupiter"
,
"types"
:
[
"Planet"
,
"Body"
]
}
],
"PlanetNames"
:
[
"Earth"
,
"Mars"
,
"Jupiter"
]
}
Variables
Variables allow mapped data to be reused without re-executing it. The var
keyword indicates that the target field is a variable. Variables have identical
semantics to fields. You can write to or iterate over variables in the same way as any
input, however, variables don't show up in the mapping output.
The following mapping is equivalent to the mapping in Writing to array fields .
PlanetNames
:
$
root
.
Planets
[
*
]
.
name
;
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[],
"Planet"
)
Moon
:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[],
"Moon"
)
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
var
tempName
:
body
.
name
;
name
:
tempName
;
types
[]:
bodyType
;
types
[]:
"Body"
;
}
Whistle containers
A Whistle container is an object used to map a set of fields to values. Containers are equivalent to a Python map or a JSON object containing keys and fields.
The following snippet shows how to declare a container variable:
var PlanetContainer: {
Planet: "Earth"
Moon: "Luna"
Size: "Medium"
Neighbors: ["Mars", "Jupiter"]
}
Conditions
Conditions are values that are only evaluated if a condition is met. Conditions in Whistle are expressed as ternary expressions .
Preparation
In this section, you will use planetary data taken from the NASA factsheets .
{
"Planets"
:
[
{
"name"
:
"Earth"
,
"semiMajorAxis"
:
149.6
},
{
"name"
:
"Mars"
,
"semiMajorAxis"
:
227.92
},
{
"name"
:
"Jupiter"
,
"semiMajorAxis"
:
778.57
}
],
"Moons"
:
[
{
"name"
:
"Luna"
,
"semiMajorAxis"
:
0.3844
}
]
}
Transforming the data into Astronomical Units (AU)
involves converting from the input in millions of km, using the conversion
factor 149.598 million km = 1 AU. To get the distance in AU, use the $Div
predefined function to divide the distance in millions of km by the conversion
constant.
PlanetNames
:
$
root
.
Planets
[
*
]
.
name
;
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[],
"Planet"
);
Moon
:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[],
"Moon"
);
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
name
:
body
.
name
;
types
[]:
bodyType
;
types
[]:
"Body"
;
semiMajorAxisAU
:
body
.
semiMajorAxis
/
149.598
;
}
Conditional mappings
The following sample demonstrates conditional mappings, which are only evaluated if a condition is met.
Add a condition so that the semiMajorAxisAU
field is output on planets and not
moons:
- Use the
==(equal) operator for comparison. - Use the
if ... then ... else ...statement to conditionally execute the mapping.- The expression in the
ifstatement is evaluated and the value afterthenis evaluated and returned if and only if the conditions are true. Otherwise, the value afterelseis evaluated and returned. - The
else ...is optional and defaults toelse {}, meaning that it returns a null value.
- The expression in the
PlanetNames
:
$
root
.
Planets
[
*
]
.
name
[];
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[],
"Planet"
)
Moon
:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[],
"Moon"
)
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
var
bigName
:
body
.
name
name
:
bigName
types
[]:
bodyType
types
[]:
"Body"
semiMajorAxisAU
:
if
bodyType
==
"Planet"
then
body
.
semiMajorAxis
/
149.598
}
In this example, the /
operator divides the million km distance by the
conversion constant to produce the distance in AU.
The output would be the following:
{
"Moon"
:
[
{
"name"
:
"Luna"
,
"types"
:
[
"Moon"
,
"Body"
]
}
],
"Planet"
:
[
{
"name"
:
"Earth"
,
"semiMajorAxisAU"
:
1.0000142656266688
,
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Mars"
,
"semiMajorAxisAU"
:
1.5235511458665132
,
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Jupiter"
,
"semiMajorAxisAU"
:
5.2044191630277785
,
"types"
:
[
"Planet"
,
"Body"
]
}
],
"PlanetNames"
:
[
"Jupiter"
]
}
Conditional blocks
The following sample demonstrates conditional blocks, which wrap a set of
mappings within a condition. Set the semiMajorAxis.unit
to AU
if the bodyType
is a Planet
:
PlanetNames
:
$
root
.
Planets
[
*
]
.
name
[];
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[],
"Planet"
)
Moon
:
BodyName_BodyType_BodyInfo
(
$
root
.
Moons
[],
"Moon"
)
def
BodyName_BodyType_BodyInfo
(
body
,
bodyType
)
{
var
bigName
:
body
.
name
name
:
bigName
types
[]:
bodyType
types
[]:
"Body"
if
(
bodyType
==
"Planet"
)
then
{
semiMajorAxis
.
value
:
body
.
semiMajorAxis
/
149.598
semiMajorAxis
.
unit
:
"AU"
}
else
{
semiMajorAxis
.
value
:
body
.
semiMajorAxis
*
1000000
semiMajorAxis
.
unit
:
"KM"
}
}
With the previous example, the output would be the following:
{
"Moon"
:
[
{
"name"
:
"Luna"
,
"semiMajorAxis"
:
{
"unit"
:
"KM"
,
"value"
:
384400
},
"types"
:
[
"Moon"
,
"Body"
]
}
],
"Planet"
:
[
{
"name"
:
"Earth"
,
"semiMajorAxis"
:
{
"unit"
:
"AU"
,
"value"
:
1.0000142656266688
},
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Mars"
,
"semiMajorAxis"
:
{
"unit"
:
"AU"
,
"value"
:
1.5235511458665132
},
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Jupiter"
,
"semiMajorAxis"
:
{
"unit"
:
"AU"
,
"value"
:
5.2044191630277785
},
"types"
:
[
"Planet"
,
"Body"
]
}
],
"PlanetNames"
:
[
"Jupiter"
]
}
Operators
Similar to Python or C, there are operators available for common arithmetic and
logical operations. This section shows some of the following operators,
where num
is a number input, bool
is a boolean input, and any
is any type
of input:
| Operator | Description |
|---|---|
| num + num | Addition |
| num - num | Subtraction |
| num * num | Multiplication |
| num / num | Division |
| str + any | Concatenation |
| any + str | Concatenation |
| bool and bool | Logical AND |
| bool or bool | Logical OR |
| !bool | Logical NOT |
| any == any | Deep equals - all elements in an array or values in an object must be the same to return true. x = y = z
is a valid expression and is equivalent to (x = y) = z
. If x = y
is true, this checks true = z
. |
| any != any | Not Equal |
| any? | Value exists - is defined, isn't literal null
, and isn't empty. An empty array has 0 elements ( null
s count as elements). An empty object has 0 keys. |
| !any? | Value Does Not Exist |
Filters
Filters have the following properties:
- Filters allow narrowing an array to items that match a condition.
- The
wherekeyword indicates a filter, similar toifindicating a condition. - Each item from the array is loaded one at a time into an input named
$in the filter. - The filter produces a new array. To iterate over the results, use the
[]operator. - Filters can only be the last element in a path. For example,
a.b[where $.color = "red"].cis invalid.
The following steps show how to use filters with the planets example:
Planet
:
BodyName_BodyType_BodyInfo
(
$
root
.
Planets
[
where
$
.
semiMajorAxis
> 200
][],
"Planet"
);
With the previous example, the output would be the following:
{
"Moon"
:
[
{
"name"
:
"Luna"
,
"semiMajorAxis"
:
{
"unit"
:
"KM"
,
"value"
:
384400
},
"types"
:
[
"Moon"
,
"Body"
]
}
],
"Planet"
:
[
{
"name"
:
"Mars"
,
"semiMajorAxis"
:
{
"unit"
:
"AU"
,
"value"
:
1.5235511458665132
},
"types"
:
[
"Planet"
,
"Body"
]
},
{
"name"
:
"Jupiter"
,
"semiMajorAxis"
:
{
"unit"
:
"AU"
,
"value"
:
5.2044191630277785
},
"types"
:
[
"Planet"
,
"Body"
]
}
],
"PlanetNames"
:
[
"Jupiter"
]
}
Caveats
This section describes the null propagation.
Nulls and null propagation
By default, the mapping engine handles and ignores null and missing values or fields, according to the following rules:
- If a field is written with a null or empty value, it's ignored (thus
null,{}, and[]can never show up in the mapping output). - If a non-existent field is accessed, it returns
null. - If a null value is passed to a function, the function is still executed.
The following sample input and mapping demonstrate nulls and null propagation:
Input
{
"Red"
:
{
"Blue"
:
1
}
}
Mapping
def
Root_Example
(
rt
)
{
//
This
field
does
not
appear
in
the
output
excluded
:
rt
.
Abcdefghijklmnop
//
This
array
will
only
contain
the
existing
items
included
[]:
rt
.
Red
.
Blue
included
[]:
rt
.
Abcd
[
123
]
.
efghi
[
*
]
.
jk
[
*
]
.
lmnop
included
[]:
rt
.
Red
.
Blue
//
nested_1
will
appear
with
just
the
constant
,
nested_2
won
't appear
nested_1
:
Nested_Example
(
rt
.
Abcdefghijklmnop
,
"Constant"
)
nested_2
:
Nested_Example
(
rt
.
Abcdefghijklmnop
,
rt
.
Abcdefghijklmnop
)
}
def
Nested_Example
(
one
,
two
)
{
one
:
one
two
:
two
}
The output would be similar to the following sample:
{
"Example"
:
[
"included"
:
[
1
,
1
],
"nested_1"
:
{
"two"
:
"Constant"
}
]
}
Using root
in a function
The following sample demonstrates using the root
keyword inside a function to
send data to the root of the output:
Red
[]:
"Blue"
Complex
:
Hello_World_HelloWorldObject
(
"Hi"
,
"Planet"
)
def
Hello_World_HelloWorldObject
(
hello
,
world
)
{
hello
:
hello
world
:
world
root
Red
[]:
world
root
Complex
.
boo
:
"boo!"
}
With the previous example, the output would be the following:
{
"Complex"
:
[
{
"boo"
:
"boo!"
,
"hello"
:
"Hi"
,
"world"
:
"Planet"
}
],
"Red"
:
[
"Blue"
,
"Planet"
]
}
