Write scripts in Whistle
This guide provides general guidance on writing a Whistle script for a parser. For more information about the Whistle syntax and available functions, see the Whistle reference .
Whistle script writing
Before starting to write a Whistle script, it is recommended to:
-
Understand the schema of the messages in the source message class to which the parser subscribes.To prevent mapping failures and, as a result, messages landing in the dead letter queue, the Whistle script has to be able to map all incoming messages. If message schemas differ from message to message, your Whistle script needs to handle these differences.
Knowing the source message schema helps you understand what fields exist in the source messages and what their data types are, so that you can decide how to map fields in the source messages to the target type schema. It is also important to understand the semantics of the data, so that you can effectively decide which parts of the source message should be mapped to attributes in the target schema.
For example, knowing the frequency of attribute value change helps you decide what data should be mapped as embedded metadata versus cloud metadata. See the sections on modeling source message classes and modeling data .
-
Understand the schema of the type version defined for the parser. The Whistle script in the parser maps source messages to the type version schema that is defined for the parser. To know how to construct a proto record that meets the requirements of the type version, you should look up the type specification. In particular, you should note the schema of the
datafield, as well as any metadata bucket associations. It is particularly important to note if any metadata bucket associations are marked asrequired: true. If you plan to look up metadata instances by value, you should make note of the schemas of the associated buckets. -
Write the Whistle script.The Whistle script performs the actual source-to-target transformation. The source message is loaded into an input called
$root. Consult the Whistle reference for an overview of the language and available functions. Also, see the other guides in this section, such as how to link records to metadata instances .
Best practices
This section outlines the best practices for writing Whistle scripts.
Perform null checks when accessing message properties
It is recommended practice to check for null when accessing a message property, whenever there is a chance that the property might not be defined.
//
Add
metadata
from
source
bucket
if
metadata
.
source
attribute
is
present
if
(
isNotNil
(
input
.
metadata
)
and
isNotNil
(
input
.
metadata
.
source
))
then
{
{
var
metadataArray
[]:
{
bucketReference
:
{
bucketName
:
"source"
;
version
:
1
;
};
naturalKey
:
input
.
metadata
.
source
;
}
}
}
Examples
The following sections show some examples of basic operations using Whistle.
Simple Whistle mapping
Given the following source message...
{
"sensor"
:
"rotation-speed-sensor"
,
"machine"
:
"m-234"
,
"timestamp"
:
"1687973092857"
,
"value"
:
1200
}
And the following Whistle script:
package
mde
[
{
tagName
:
$
root
.
machine
+
" - "
+
$
root
.
sensor
;
data
:
{
numeric
:
$
root
.
value
;
};
timestamps
:
{
eventTimestamp
:
$
root
.
timestamp
;
}
}
]
The parser will produce this proto record output:
[
{
"tagName"
:
"m-234-rotation-speed-sensor"
,
"data"
:
{
"numeric"
:
1200
},
"timestamps"
:
{
"eventTimestamp"
:
"1687973092857"
}
}
]
Simple Whistle mapping using functions
Given the following source message:
{
"sensor"
:
"rotation-speed-sensor"
,
"machine"
:
"m-234"
,
"timestamp"
:
"1687973092857"
,
"value"
:
1200
}
And the following Whistle script:
package
mde
[
{
tagName
:
getTagName
(
$
root
);
data
:
getValue
(
$
root
);
timestamps
:
getTimestamp
(
$
root
)
}
]
def
getTagName
(
input
)
{
input
.
machine
+
"-"
+
input
.
sensor
;
}
def
getTimestamp
(
input
)
{
eventTimestamp
:
input
.
timestamp
;
}
def
getValue
(
input
)
{
numeric
:
input
.
value
;
}
The parser will produce this proto record output:
[
{
"tagName"
:
"m-234-rotation-speed-sensor"
,
"data"
:
{
"numeric"
:
1200
},
"timestamps"
:
{
"eventTimestamp"
:
"1687973092857"
}
}
]
Emitting several proto records from a parser 1
Given the following source message:
{
"tag"
:
"plc-34"
,
"machine"
:
"controller"
,
"timestamp"
:
"1687973092857"
,
"values"
:
[
200
,
499
]
}
And the following Whistle script:
package
mde
var
valueLen
:
listLen
(
$
root
.
values
);
var
indexes
:
range
(
0
,
valueLen
);
[
getProtoRecords
(
$
root
.
values
[],
indexes
[],
$
root
)
]
def
getProtoRecords
(
value
,
index
,
input
)
{
tagName
:
input
.
machine
+
"-"
+
input
.
tag
+
"-"
+
index
;
data
:
{
numeric
:
value
;
};
timestamps
:
{
eventTimestamp
:
input
.
timestamp
;
};
}
The parser will produce this proto record output:
[
{
"tagName"
:
"controller-plc-34-0"
,
"data"
:
{
"value"
:
200.0
},
"timestamps"
:
{
"eventTimestamp"
:
"1687973092857"
}
},
{
"tagName"
:
"controller-plc-34-1"
,
"data"
:
{
"value"
:
499.0
},
"timestamps"
:
{
"eventTimestamp"
:
"1687973092857"
}
}
]
Emitting several proto records from a parser 2
Given the following source message:
{
"machine"
:
"controller"
,
"timestamp"
:
"1687973092857"
,
"sensors"
:
[
{
"tag"
:
"plc-34"
,
"value"
:
200
},
{
"tag"
:
"plc-35"
,
"value"
:
499
}
]
}
And the following Whistle script:
package
mde
[
$$
getProtoRecords
(
$
root
.
sensors
[],
$
root
)
]
def
getProtoRecords
(
sensor
,
input
)
{
tagName
:
input
.
machine
+
"-"
+
sensor
.
tag
;
data
:
{
numeric
:
sensor
.
value
;
};
timestamps
:
{
eventTimestamp
:
input
.
timestamp
;
};
}
The parser will produce this proto record output:
[
{
"tagName"
:
"controller-plc-34"
,
"timestamps"
:
{
"eventTimestamp"
:
"1687973092857"
},
"data"
:
{
"numeric"
:
200
}
},
{
"tagName"
:
"controller-plc-35"
,
"timestamps"
:
{
"eventTimestamp"
:
"1687973092857"
},
"data"
:
{
"numeric"
:
499
}
}
]
