Skip to main content

Conventions

This Guide uses the following conventions for Pipe definition examples.

YAML Formatting​

Examples typically uses four spaces for YAML indentation.

Pipe Snippets​

Some examples are incomplete Pipe definitions, containing only pertinent sections of a Pipe definition. Missing sections are typically indicated as follows:

... # <--- name, input not specified

actions:

- rename:
    oldField: newField

... # <--- output not specified

The echo Input​

The echo input is often used in examples. It is used to generate static input events in a Pipe, without needing a live data source. As such, it is also convenient for testing.

...
input:
    echo:
        event: {"my":"field"}
...

# output:
# {"_raw": "{\"my\":\"field\"}"}

Note input isn't automatically interpreted as JSON. Use json: true to indicate that input events are already valid JSON, which prevents wrapping the input into a _raw field:

...
input:
    echo:
        json: true
        event: {"my":"field"}
...

# output:
# {"my":"field"}

Create multiple input events using YAML's multiline syntax:

...
input:
    echo:
        json: true
        event: |
            {"field":"value A"}
            {"field":"value B"}
            {"field":"value C"}
...

# output:
# {"field":"value A"}
# {"field":"value B"}
# {"field":"value C"}

The default behaviour is to split input lines into individual events. Use ignore-linebreaks: true to treat multiple input lines as a single input event.

The print Output​

Many examples use the print output, which will print output events to the Pipe process's STDOUT file descriptor. While often used as a placeholder in this Guide, it's useful for testing, without needing a live output.

...
output:
    print: STDOUT

Note the following when using this output in a Pipe:

  • When running a Pipe via CLI, output events are printed to the terminal
  • When running a Pipe via the internal Server Agent, output events are printed to STDOUT of the Server process
  • When running a Pipe via an Agent, output events are printed to STDOUT of the Agent process