Pipelines
Variable manipulation
The string-eval
step is used to do string based evaluations, which allows a variety of actions to be performed. These include numeric calculations and casting to different data types.
The format will feel familiar to those with database and/or JavaScript experience. The expression is a constant that can use placeholders for input parameters so they can be used in the expression.
An example use case may look like the following:
- id: string-eval
condition: radius_km
params:
param:
bind: radius_miles
expression:
const: 'FLOAT(radius_km)/1.6'
This example takes an input variable radius_km
, evaluates the expression and then writes the output variable to the radius_miles
parameter. This step has the condition associated that ensures it only runs if the radius_km
input exists.
You can also write the output back to one of the input variables. You don’t need to create a new variable. This is useful for augmenting existing parameters in place.
Variables always begin as strings, so you need convert them using INT()
or FLOAT()
casting to use them in mathematical expressions.
Operators
The expression supports AND
, OR
and XOR
operators as well as parentheses. The logic also supports the IF()
function much like you would see in databases and/or Excel (see Conditional functions below).
String concatenation
Much like JavaScript, strings can be concatenated using the +
symbol. For example:
- id: string-eval
params:
param:
bind: concat_output
expression:
const: firstName + ' ' + lastName
The above function would take two inputs and concatenate them with a space in between. Interestingly the JOIN()
function could also have been used here:
- id: string-eval
params:
param:
bind: concat_output
expression:
const: JOIN([firstName, lastName], " ")
Join comes into it’s own once you are dynamically adding things into a list, such as filter expressions. The brackets around the variables indicate a list, of which the elements are joined.
This is very useful for combining filters which may or may not exist in the request. Take the below example, if the bathFilter
and priceFilter
are NULL
and the bedsFilter = “beds = 3”
, then the output filterExpr = “beds = 3”
- id: string-eval
params:
param:
bind: filterExpr
expression:
const: JOIN([bedsFilter, bathFilter, priceFilter], " AND ")
Much like JavaScript, if you don’t convert strings, operators may not do what you are expecting, e.g. 5+6 = 56
if “5” and “6” are strings.
Functions
This step supports a range of functions that can be used in the expression. These functions fall into several groups:
Casting
Convert a string into different data types. This is common for numeric operations.
INT(item)
Int attempts to convert the item to an integer. If the type is a float it will be rounded to the nearest integer.
FLOAT(item)
Float attempts to convert the item into a float.
STRING(item)
String converts the item to a string.
BOOL()
Bool attempts to convert the item to a Boolean. A zero will convert to FALSE
, other numeric values will convert to TRUE
. The string versions of TRUE
and FALSE
will convert as expected.
Conditional
IF(bool, item_if_true, item_if_false)
This takes a bool value (which could be the result of an expression), and returns item_if_true
OR item_if_false
for true/false values respectively.
Arrays
As previously shown above you can create and join lists using the bracket notation, e.g. [a]+[b]
produces a list [a, b]
String functions
String functions allow for strings to be transformed, joined, quoted, trimmed and more.
JOIN(elements, separator)
Join concatenates the elements
of its first argument to create a single string. The separator
string sep is placed between elements in the resulting string.
QUOTE(string)
Quote returns a double-quoted string literal representing string
. The returned string uses escape sequences (\t, \n, \xFF, \u0100) for control characters and non-printable characters.
LENGTH(item)
Length returns the number of bytes if the item is a string
, or the number of elements if the item is an array.
TRIM(string)
Trim returns a slice of the string
with all leading and trailing Unicode code points representing whitespace.
LTRIM(string)
LTrim returns a slice of the string
with all leading Unicode code points representing whitespace.
RTRIM(string)
RTrim returns a slice of the string
with all trailing Unicode code points representing whitespace.
RIGHT(string, num_of_chars)
Right extracts a number of characters num_of_chars
from string
(starting from right)
LEFT(string, num_of_chars)
Left extracts a number of characters num_of_chars
from string
(starting from left)
UPPER(string)
Upper converts string
to upper-case.
LOWER(string)
Lower converts a string to lower-case.
Numeric functions
As the name suggests, numeric functions can be used to perform numeric transforms on inbound parameters.
POWER(base, exponent)
Power takes a base
number and raises it to the power of the exponent
.
ABS(number)
Abs returns the absolute value of number
.
EXP(number)
Exp returns e
raised to the power of number
. The constant e
(2.718281...), is the base of natural logarithms.
CEIL(number)
Ceil returns the smallest integer value that is larger than or equal to number
.
FLOOR(number)
Floor returns the largest integer value that is smaller than or equal to number
.
SQRT(number)
Sqrt returns the square root of number
.
LN(number)
Ln returns the natural logarithm of number
.
For example, you may want to calculate the square root of an input parameter (string):
- id: string-eval
condition: my_number
params:
param:
bind: my_number
expression:
const: SQRT(FLOAT(my_number))
In the above example, the input my_number
must exist for the step to execute. If it does, it is converted from a string to a float, then the square root is calculated and written back into the variable. It could also go into a new parameter, but in this case we are writing to the existing parameter.