file

Write to a file

Can use field expansions to create the filename. When it changes, you can specify a header and/or footer

Example

action:

file:
  path: '/tmp/out-${now_time_fmt %Y-%M-%D}.log
  input-field: out
  header: Start: ${name}

Field Name	Description	Type	Default
batch	Maximum number of events in an output batch. If 'document' send on end of document	integer or the 'document'	-
timeout	Interval after which the batch is sent, to keep throughput going	interval	100ms
header	Put a header line before the batch	templated text	-
footer	Put a header line after the last line of the batch	templated text	-
use-document-marker	Enrich the pipe metadata with a document marker (for document handling in batch mode)	bool	false
retry	For operations that could potentially fail	Retry	-
path	Path to the file to be written	string	-
input-field	Use the specified field as the content for the file line	field	-
flush-at-end	When the filename changes a new file will be written, force a flush at that point	bool	false
compress-after	Compress the written file after the filename changes	bool	false
create-directories	Create parent directories for the given path if they do not exist	bool	false

batch

Maximum number of events in an output batch. If 'document' send on end of document

Type: integer or the 'document'

timeout

Interval after which the batch is sent, to keep throughput going

Type: interval

header

Put a header line before the batch

Type: templated text

footer

Put a header line after the last line of the batch

Type: templated text

use-document-marker

Enrich the pipe metadata with a document marker (for document handling in batch mode)

Type: bool

retry

For operations that could potentially fail

Type: Retry

Field Name	Description	Type	Default
count	How many attempts to make before declaring failure	integer	-
pause	How long to pause before re-trying	duration	-
forever	Keep trying until success is declared	bool	false

count

How many attempts to make before declaring failure

Type: integer

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 1

output:

{"_raw":"one two"}

pause

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: duration

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 6
    pause: 10s

output:

{"_raw":"one two"}

forever

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: bool

Example

action:

exec:
  command: echo 'one two'
  retry:
    forever: true

output:

{"_raw":"one two"}

path

Path to the file to be written

Type: string

input-field

Use the specified field as the content for the file line

Type: field

flush-at-end

When the filename changes a new file will be written, force a flush at that point

Type: bool

compress-after

Compress the written file after the filename changes

Type: bool

create-directories

Create parent directories for the given path if they do not exist

Type: bool