Observability Pipelines is not available on the US1-FED Datadog site.
Datadog Processing Language (DPL), or Vector Remap Language (VRL), is a fail-safe language, which means that a DPL/VRL program does not compile unless all potential errors are handled. This ensures that your DPL/VRL programs can handle malformed data.
Compile-time errors
100 Unhandled root runtime error
A root expression is fallible and its runtime error isn’t handled in the DPL
program.
Rationale
DPL is fail safe and thus requires that all possible runtime errors be handled.
This provides important safety guarantees to DPL and helps to ensure that DPL programs
run reliably when deployed.
Resolution
Handle the runtime error by assigning,
coalescing, or raising the
error.
Examples
Unhandled root runtime error (assigning)
DPL program:
How to fix it:
- get_env_var("HOST")
+# .host = get_env_var("HOST")
A regex literal expression is malformed
and thus doesn’t result in a valid regular expression.
Invalid regular expressions don’t compile.
Regular expressions are difficult to write and commonly result in syntax errors. If you’re parsing a common
log format we recommend using one of DPL’s parse_*
functions. If
you don’t see a function for your format please request it. Otherwise, use the
Rust regex tester to test and correct your regular expression.
DPL program:
. |= parse_regex!(.message, r'^(?P<host>[\w\.]+) - (?P<user>[\w]+) (?P<bytes_in>[\d]+) \[?P<timestamp>.*)\] "(?P<method>[\w]+) (?P<path>.*)" (?P<status>[\d]+) (?P<bytes_out>[\d]+)$')
How to fix it:
-. |= parse_regex!(.message, r'^(?P<host>[\w\.]+) - (?P<user>[\w]+) (?P<bytes_in>[\d]+) \[?P<timestamp>.*)\] "(?P<method>[\w]+) (?P<path>.*)" (?P<status>[\d]+) (?P<bytes_out>[\d]+)$')
+. |= parse_common_log!(.message)
102 Non-boolean if expression predicate
An if expression predicate doesn’t
evaluate to a Boolean.
Rationale
DPL doesn’t implement “truthy” values (non-Boolean values that resolve to a Boolean, such as 1
) since these
are common foot-guns that can result in unexpected behavior when used in if expressions. This provides important
safety guarantees in DPL and ensures that DPL programs are reliable once deployed.
Resolution
Adjust your if expression predicate to resolve to a Boolean. Helpful functions to solve this include
exists
and
is_nullish
.
Examples
Non-boolean if expression predicate (strings)
DPL program:
if .message {
. |= parse_key_value!(.message)
}
How to fix it:
-if .message {
+if exists(.message) {
. |= parse_key_value!(.message)
}
103 Unhandled fallible assignment
The right-hand side of this assignment
is fallible (that is, it can produce a runtime error), but the error isn’t
handled.
Rationale
DPL is fail safe and thus requires that all possible runtime errors be handled.
This provides important safety guarantees to DPL and helps to ensure that DPL programs
run reliably when deployed.
Resolution
Handle the runtime error by either
assigning it, coalescing it, or
raising it.
Examples
Unhandled fallible assignment (coalescing)
DPL program:
. = parse_key_value(.message)
How to fix it:
-. = parse_key_value(.message)
+. = parse_key_value(.message) ?? {}
Unhandled fallible assignment (raising)
DPL program:
. = parse_key_value(.message)
How to fix it:
-. = parse_key_value(.message)
+. = parse_key_value!(.message)
Unhandled fallible assignment (assigning)
DPL program:
. = parse_key_value(.message)
How to fix it:
-. = parse_key_value(.message)
+., err = parse_key_value(.message)
104 Unnecessary error assignment
The left-hand side of an assignment expression
needlessly handles errors even though the right-hand side can’t fail.
Rationale
Assigning errors when one is not possible is effectively dead code that makes your program difficult to follow.
Removing the error assignment simplifies your program.
Resolution
Remove the error assignment.
Examples
Unnecessary error assignment (strings)
DPL program:
.message, err = downcase(.message)
How to fix it:
-.message, err = downcase(.message)
+.message = downcase(.message)
105 Undefined function
A function call expression invokes an
unknown function.
Resolution
This is typically due to a typo. Correcting the function name should resolve this.
Examples
Undefined function (typo)
DPL program:
How to fix it:
-parse_keyvalue(.message)
+parse_key_value(.message)
106 Function argument arity mismatch
A function call expression invokes a
function with too many arguments.
Resolution
Remove the extra arguments to adhere to the function’s documented signature.
Examples
Function argument arity mismatch
DPL program:
parse_json(.message, pretty: true)
How to fix it:
-parse_json(.message, pretty: true)
+parse_json(.message)
107 Required function argument missing
A function call expression fails to pass
a required argument.
Resolution
Supply all of the required function arguments to adhere to the function’s documented signature.
Examples
Required function argument missing
DPL program:
parse_timestamp(.timestamp)
How to fix it:
-parse_timestamp(.timestamp)
+parse_timestamp(.timestamp, format: "%D")
108 Unknown function argument keyword
A function call expression passes an
unknown named argument.
Resolution
Correct the name to align with the documented argument names for the function.
Examples
Unknown function argument keyword
DPL program:
parse_timestamp(.timestamp, fmt: "%D")
How to fix it:
-parse_timestamp(.timestamp)
+parse_timestamp(.timestamp, format: "%D")
110 Invalid argument type
An argument passed to a function call expression
isn’t a supported type.
Rationale
DPL is fail safe and thus requires that all possible runtime errors be handled.
This provides important safety guarantees to DPL and helps to ensure that DPL programs
run reliably when deployed.
Resolution
You must guarantee the type of the variable by using the appropriate
type or coercion function.
Examples
Invalid argument type (guard with defaults)
DPL program:
How to fix it:
+.message = string(.message) ?? ""
downcase(.message)
Invalid argument type (guard with errors)
DPL program:
How to fix it:
downcase(string!(.message))
111 Unhandled predicate error
A predicate is fallible and its runtime error isn’t handled in the DPL
program.
Rationale
DPL is fail safe and thus requires that all possible runtime errors be handled.
This provides important safety guarantees to DPL and helps to ensure that DPL programs
run reliably when deployed.
Resolution
Handle the runtime error by assigning,
coalescing, or raising the
error.
Examples
Unhandled predicate error (predicate)
DPL program:
if contains(.field, "thing") {
log("thing")
}
How to fix it:
- if contains(.field, "thing") {
+# if contains(.field, "thing") ?? false {
203 Unrecognized token
Your DPL program contains a token (character) that the DPL parser doesn’t recognize as valid.
Resolution
Use a valid token.
Examples
Unrecognized token
DPL program:
How to fix it:
204 Unrecognized end-of-file (EOF)
The DPL parser has reached the end of the program in an invalid state, potentially due to a
typo or a dangling expression.
Resolution
Make sure that the last expression in the program is valid.
Examples
Unrecognized end-of-file (EOF)
DPL program:
.field1 = "value1"
.field2 =
How to fix it:
-.bar =
+.field2 = "value2"
205 Reserved keyword
You’ve used a name for a variable that serves another purpose in DPL or is reserved for potential future use.
Resolution
Use a different variable name.
Examples
Reserved keyword
DPL program:
How to fix it:
-else = "some value"
+some_non_reserved_name = "some value"
206 Invalid numeric literal
The DPL compiler doesn’t recognize this numeric literal as valid.
207 Invalid string literal
Your DPL program contains a string literal that the DPL parser doesn’t recognize as valid.
Resolution
Make sure that your string is properly enclosed by single or double quotes.
Examples
Invalid string literal
DPL program:
"Houston, we have a problem'
How to fix it:
- "Houston, we have a problem'
+ "Houston, we have a problem"
208 Invalid literal
The DPL compiler doesn’t recognize this literal value as valid.
209 Invalid escape character
Your string includes an escape character that the DPL compiler doesn’t recognize as valid
300 Unexpected type
The DPL compiler expected a value of a specific type but found a different type.
301 Type coercion error
This value can’t be coerced into the desired type.
302 Remainder error
These two types can’t produce a remainder.
303 Multiplication error
These types can’t be multiplied together
304 Division error
The left-hand value can’t be divided by the right-hand value.
305 Divide by zero
You’ve attempted to divide an integer or float by zero.
Rationale
Unlike some other programming languages, DPL doesn’t have any concept of infinity, as it’s
unclear how that could be germane to observability data use cases. Thus, dividing by zero
can’t have any meaningful result.
Resolution
If you know that a value is necessarily zero, don’t divide by it. If a value could be
zero, capture the potential error thrown by the operation:
result, err = 27 / .some_value
if err != null {
# Handle error
}
306 NaN float
Floats in DPL can’t be NaN (not a number).
307 Addition error
These two values can’t be added together.
308 Subtraction error
The right-hand value can’t be subtracted from the left-hand value.
309 Or expression error
These two values can’t be combined into an or expression.
310 And expression error
These two values can’t be combined into an and expression.
311 Greater than error
These two values can’t be used in a greater than expression.
312 Greater than or equal to error
These two values can’t be used in a greater than or equal to expression.
313 Less than error
These two values can’t be used in less than expression.
314 Less than or equal to error
These two values can’t be used in a less than or equal to expression.
315 mutation of read-only value
This value is read-only and cannot be deleted or mutated.
400 Unexpected expression
The DPL compiler encountered an expression type that wasn’t expected here.
401 Invalid enum variant
DPL expects an enum value for this argument, but the value you entered for the enum is
invalid.
Resolution
Check the documentation for this function in the DPL functions
reference to see which enum values are valid for this argument.
402 Expected static expression for function argument
DPL expected a static expression for a function argument, but a dynamic one was provided (such as a variable).
DPL requires static expressions for some function arguments to validate argument types at
compile time to avoid runtime errors.
Resolution
Replace the dynamic argument with a static expression.
403 Invalid argument
An invalid argument was passed to the function. The error string will contain more details about what was invalid.
Resolution
Check the error string for this error to see what was invalid.
601 Invalid timestamp
The provided timestamp literal is properly
formed (i.e. it uses t'...'
syntax) but the timestamp doesn’t adhere to RFC 3339 format.
Rationale
Invalid timestamps don’t compile.
Resolution
Bring the timestamp in conformance with RFC 3339 format.
Examples
DPL program:
.timestamp = format_timestamp!(t'next Tuesday', format: "%v %R")
How to fix it:
-.timestamp = format_timestamp!(t'next Tuesday', format: "%v %R")
+.timestamp = format_timestamp!(t'2021-03-09T16:33:02.405806Z', format: "%v %R")
620 Aborting infallible function
You’ve specified that a function should abort on error even though the function is infallible.
Rationale
In DPL, infallible functions—functions that can’t fail—don’t require error
handling, which in turn means it doesn’t make sense to abort on failure using a !
in the function call.
Resolution
Remove the !
from the function call.
Examples
Aborting infallible function
DPL program:
encode_json!(["one", "two", "three"])
How to fix it:
- encode_json!(["one", "two", "three"])
+# encode_json(["one", "two", "three"])
630 Fallible argument
You’ve passed a fallible expression as an argument to a function.
Rationale
In DPL, expressions that you pass to functions as arguments need to be infallible themselves. Otherwise, the
outcome of the function would be indeterminate.
Resolution
Make the expression passed to the function infallible, potentially by aborting on error using !
, coalescing
the error using ??
, or via some other method.
Examples
Fallible argument
DPL program:
format_timestamp!(to_timestamp("2021-01-17T23:27:31.891948Z"), format: "%v %R")
How to fix it:
- format_timestamp!(to_timestamp("2021-01-17T23:27:31.891948Z"), format: "%v %R")
+ format_timestamp!(to_timestamp!("2021-01-17T23:27:31.891948Z"), format: "%v %R")
631 Fallible abort message expression
You’ve passed a fallible expression as a message to abort.
Rationale
An expression that you pass to abort needs to be infallible. Otherwise, the abort expression could fail at runtime.
Resolution
Make the expression infallible, potentially by handling the error, coalescing the error using ??
, or via some other method.
Examples
Fallible abort message expression
DPL program:
How to fix it:
- abort to_syslog_level(0)
+ abort to_syslog_level(0) ?? "other"
640 No-op assignment
You’ve assigned a value to something that is neither a variable nor a path.
Rationale
All assignments in DPL need to be to either a path or a variable. If you try to assign a value to, for example,
underscore (_
), this operation is considered a “no-op” as it has no effect (and is thus not an assignment at
all).
Resolution
Assign the right-hand-side value to either a variable or a path.
Examples
No-op assignment
DPL program:
_ = "the hills are alive"
How to fix it:
- _ = "the hills are alive"
+# .movie_song_quote = "the hills are alive"
650 Chained comparison operators
You’ve chained multiple comparison operators together in a way that can’t result in a valid expression.
Rationale
Comparison operators can only operate on two operands, e.g. 1 != 2
. Chaining them together, as in
1 < 2 < 3
, produces a meaningless non-expression.
Resolution
Use comparison operators only on a left-hand- and a right-hand-side value. You can chain comparisons together
provided that the expressions are properly grouped. While a == b == c
, for example, isn’t valid,
a == b && b == c
is valid because it involves distinct Boolean expressions.
Examples
Chained comparison operators
DPL program:
How to fix it:
- 1 == 1 == 2
+# (1 == 1) && (1 == 2)
651 Unnecessary error coalescing operation
You’ve used a coalescing operation (??
) to handle an error, but in this case the left-hand
operation is infallible, and so the right-hand value after ??
is never reached.
Rationale
Error coalescing operations are useful when you want to specify what happens if an operation
fails. Here’s an example:
In this example, if op1
is infallible (that is, it can’t error) then the result
variable
if set to the value of op1
while op2
is never reached.
Resolution
If the left-hand operation is meant to be infallible, remove the ??
operator and the
right-hand operation. If, however, the left-hand operation is supposed to be fallible,
remove the !
from the function call and anything else that’s making it infallible.
652 Only objects can be merged
You’re attempting to merge two values together but one or both isn’t an object.
Rationale
Amongst DPL’s available types, only objects can be merged together. It’s not clear what it
would mean to merge, for example, an object with a Boolean. Please note, however,
that some other DPL types do have merge-like operations available:
- Strings can be concatenated together
- Arrays can be appended to other arrays
These operations may come in handy if you’ve used merge
by
accident.
Resolution
Make sure that both values that you’re merging are DPL objects. If you’re not sure whether
a value is an object, you can use the object
function to
check.
660 Non-Boolean negation
You’ve used the negation operator to negate a non-Boolean expression.
Rationale
Only Boolean values can be used with the negation operator (!
). The expression !false
, for example, produces
true
, whereas !"hello"
is a meaningless non-expression.
Resolution
Use the negation operator only with Boolean expressions.
Examples
Non-Boolean negation
DPL program:
How to fix it:
701 Call to Undefined Variable
The referenced variable is undefined.
Rationale
Referencing a variable that is undefined results in unexpected behavior, and is likely due to a typo.
Resolution
Assign the variable first, or resolve the reference typo.
Examples
Undefined variable
DPL program:
How to fix it:
+my_variable = true
my_variable
Wrong variable name
DPL program:
my_variable = true
my_var
How to fix it:
801 Usage of deprecated item
The referenced item is deprecated. Usually an alternative is given that can be used instead.
Rationale
This will be removed in the future.
Resolution
Apply the suggested alternative.