file

Write to a file

Can use field expansions to create the filename. When it changes, you can specify a header and/or footer

Example

action:

file:
  path: '/tmp/out-${now_time_fmt %Y-%M-%D}.log
  input-field: out
  header: Start: ${name}

Field Name	Description	Type	Default
batch	Maximum number of events in an output batch. If 'document' send on end of document	integer or the 'document'	-
timeout	Interval after which the batch is sent, to keep throughput going	interval	100ms
header	Put a header line before the batch	templated text	-
footer	Put a header line after the last line of the batch	templated text	-
use-document-marker	Enrich the pipe metadata with a document marker (for document handling in batch mode)	bool	false
retry	For operations that could potentially fail	Retry	-
path	Path to the file to be written	string	-
input-field	Use the specified field as the content for the file line	field	-
flush-at-end	When the filename changes a new file will be written, force a flush at that point	bool	false
compress-after	Compress the written file after the filename changes	bool	false
create-directories	Create parent directories for the given path if they do not exist	bool	false
append-to-existing	Append to an output file if it already exists	bool	false

batch

Maximum number of events in an output batch. If 'document' send on end of document

Type: integer or the 'document'

timeout

Interval after which the batch is sent, to keep throughput going

Type: interval

header

Put a header line before the batch

Type: templated text

footer

Put a header line after the last line of the batch

Type: templated text

use-document-marker

Enrich the pipe metadata with a document marker (for document handling in batch mode)

Type: bool

retry

For operations that could potentially fail

Type: Retry

Field Name	Description	Type	Default
count	How many attempts to make before declaring failure	integer	-
pause	How long to pause before re-trying	duration	-
forever	Keep trying until success is declared	bool	false

count

How many attempts to make before declaring failure

Type: integer

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 1

output:

{"_raw":"one two"}

pause

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: duration

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 6
    pause: 10s

output:

{"_raw":"one two"}

forever

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: bool

Example

action:

exec:
  command: echo 'one two'
  retry:
    forever: true

output:

{"_raw":"one two"}

path

Path to the file to be written

Type: string

input-field

Use the specified field as the content for the file line

Type: field

flush-at-end

When the filename changes a new file will be written, force a flush at that point

Type: bool

compress-after

Compress the written file after the filename changes

Type: bool

create-directories

Create parent directories for the given path if they do not exist

Type: bool

append-to-existing

Append to an output file if it already exists

Type: bool