Output: file

Write to a file

Can use field expansions to create the filename. When it changes, you can specify a header and/or footer

Example

Pipe Language Snippet:

file:
  path: '/tmp/out-${now_time_fmt %Y-%M-%D}.log
  input-field: out
  header: Start: ${name}

Field Summary

Field Name	Type	Description	Default
batch	integer or the 'document'	Maximum number of events in an output batch. If 'document' send on end of document	-
timeout	interval	Interval after which the batch is sent, to keep throughput going	100ms
header	templated text	Put a header line before the batch	-
footer	templated text	Put a header line after the last line of the batch	-
use-document-marker	bool	Enrich the pipe metadata with a document marker (for document handling in batch mode)	false
retry	Retry	For operations that could potentially fail	-
path	string	Path to the file to be written	-
input-field	field	Use the specified field as the content for the file line	-
flush-at-end	bool	When the filename changes a new file will be written, force a flush at that point	false
compress-after	bool	Compress the written file after the filename changes	false
create-directories	bool	Create parent directories for the given path if they do not exist	false
append-to-existing	bool	Append to an output file if it already exists	false

Fields

batch

Type: integer or the 'document'

Maximum number of events in an output batch. If 'document' send on end of document

timeout

Type: interval

Default: 100ms

Interval after which the batch is sent, to keep throughput going

header

Type: templated text

Put a header line before the batch

footer

Type: templated text

Put a header line after the last line of the batch

use-document-marker

Type: bool

Default: false

Enrich the pipe metadata with a document marker (for document handling in batch mode)

retry

Type: Retry

For operations that could potentially fail

Field Name	Type	Description	Default
count	integer	How many attempts to make before declaring failure	-
pause	duration	How long to pause before re-trying	-
forever	bool	Keep trying until success is declared	false

  count

Type: integer

How many attempts to make before declaring failure

Example

Pipe Language Snippet:

exec:
  command: echo 'one two'
  retry:
    count: 1

Output:

{"_raw":"one two"}

  pause

Type: duration

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Example

Pipe Language Snippet:

exec:
  command: echo 'one two'
  retry:
    count: 6
    pause: 10s

Output:

{"_raw":"one two"}

  forever

Type: bool

Default: false

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Example

Pipe Language Snippet:

exec:
  command: echo 'one two'
  retry:
    forever: true

Output:

{"_raw":"one two"}

path

Type: string

Path to the file to be written

input-field

Type: field

Use the specified field as the content for the file line

flush-at-end

Type: bool

Default: false

When the filename changes a new file will be written, force a flush at that point

compress-after

Type: bool

Default: false

Compress the written file after the filename changes

create-directories

Type: bool

Default: false

Create parent directories for the given path if they do not exist

append-to-existing

Type: bool

Default: false

Append to an output file if it already exists