exec

Execute arbitrary commands

Things to note:

• By default, input data streams through the output process, i.e. works like a shell filter. You cannot have ${field} expansions of the command when streaming, so streaming is switched off. Then the command will be executed once for each input line. With input-field you have to explicitly ask for streaming: true for reasons of backward compatibility.
• With batched output you will have access to the fields of the first event in the batch

Note: This invokes a shell, which is something we might want to avoid at some point

Field Name	Description	Type	Default
batch	Maximum number of events in an output batch. If 'document' send on end of document	integer or the 'document'	-
timeout	Interval after which the batch is sent, to keep throughput going	interval	100ms
header	Put a header line before the batch	templated text	-
footer	Put a header line after the last line of the batch	templated text	-
use-document-marker	Enrich the pipe metadata with a document marker (for document handling in batch mode)	bool	false
input-field	A field from where data is to be read, meaning JSON is assumed	field	-
command	A shell command to be executed	string	-
streaming	Run the command separately for each event	bool	true
retry	For operations that could potentially fail	Retry	-

batch

Maximum number of events in an output batch. If 'document' send on end of document

Type: integer or the 'document'

timeout

Interval after which the batch is sent, to keep throughput going

Type: interval

header

Put a header line before the batch

Type: templated text

footer

Put a header line after the last line of the batch

Type: templated text

use-document-marker

Enrich the pipe metadata with a document marker (for document handling in batch mode)

Type: bool

input-field

A field from where data is to be read, meaning JSON is assumed

Otherwise the whole event is sent

Type: field

Example

input:

{"text":"one","number":1}

action:

exec:
  input-field: text
  command: cat

output:

one

command

A shell command to be executed

If streaming is set to false (or input-field by default) then there can be field expansions

Type: string

Example

input:

{"text":"one","number":1}

action:

exec:
  command: cat

output:

{"text":"one","number":1}

streaming

Run the command separately for each event

Normally, the command runs continuously and events are passed through

Type: bool

Example: streaming must be explicit if input-field is defined

input:

{"msg":"hello dolly"}
{"msg":"we are happy"}

action:

exec:
  input-field: msg
  streaming: true
  command: tr 'h' 'H'

output:

Hello dolly
we are Happy

Example: streaming is implicit if there are field expansions

input:

{"text":"one","number":1}

action:

exec:
  command: echo "${text}-${number}"

output:

one-1

retry

For operations that could potentially fail

Type: Retry

Field Name	Description	Type	Default
count	How many attempts to make before declaring failure	integer	-
pause	How long to pause before re-trying	duration	-
forever	Keep trying until success is declared	bool	false

count

How many attempts to make before declaring failure

Type: integer

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 1

output:

{"_raw":"one two"}

pause

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: duration

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 6
    pause: 10s

output:

{"_raw":"one two"}

forever

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: bool

Example

action:

exec:
  command: echo 'one two'
  retry:
    forever: true

output:

{"_raw":"one two"}