Output: file
Write to a file
Can use field expansions to create the filename. When it changes, you can specify a header and/or footer
Example
Pipe Language Snippet:
file:
path: '/tmp/out-${now_time_fmt %Y-%M-%D}.log
input-field: out
header: Start: ${name}
Field Summary
Field Name | Type | Description | Default |
---|---|---|---|
batch | integer or the 'document' | Maximum number of events in an output batch. If 'document' send on end of document | - |
timeout | interval | Interval after which the batch is sent, to keep throughput going | 100ms |
header | templated text | Put a header line before the batch | - |
footer | templated text | Put a header line after the last line of the batch | - |
use-document-marker | bool | Enrich the pipe metadata with a document marker (for document handling in batch mode) | false |
retry | Retry | For operations that could potentially fail | - |
path | string | Path to the file to be written | - |
input-field | field | Use the specified field as the content for the file line | - |
flush-at-end | bool | When the filename changes a new file will be written, force a flush at that point | false |
compress-after | bool | Compress the written file after the filename changes | false |
create-directories | bool | Create parent directories for the given path if they do not exist | false |
append-to-existing | bool | Append to an output file if it already exists | false |
Fields
batch
Type: integer or the 'document'
Maximum number of events in an output batch. If 'document' send on end of document
timeout
Type: interval
Default: 100ms
Interval after which the batch is sent, to keep throughput going
header
Type: templated text
Put a header line before the batch
footer
Type: templated text
Put a header line after the last line of the batch
use-document-marker
Type: bool
Default: false
Enrich the pipe metadata with a document marker (for document handling in batch mode)
retry
Type: Retry
For operations that could potentially fail
Field Name | Type | Description | Default |
---|---|---|---|
count | integer | How many attempts to make before declaring failure | - |
pause | duration | How long to pause before re-trying | - |
forever | bool | Keep trying until success is declared | false |
count
Type: integer
How many attempts to make before declaring failure
Example
Pipe Language Snippet:
exec:
command: echo 'one two'
retry:
count: 1
Output:
{"_raw":"one two"}
pause
Type: duration
How long to pause before re-trying
Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)
Example
Pipe Language Snippet:
exec:
command: echo 'one two'
retry:
count: 6
pause: 10s
Output:
{"_raw":"one two"}
forever
Type: bool
Default: false
Keep trying until success is declared
Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)
Example
Pipe Language Snippet:
exec:
command: echo 'one two'
retry:
forever: true
Output:
{"_raw":"one two"}
path
Type: string
Path to the file to be written
input-field
Type: field
Use the specified field as the content for the file line
flush-at-end
Type: bool
Default: false
When the filename changes a new file will be written, force a flush at that point
compress-after
Type: bool
Default: false
Compress the written file after the filename changes
create-directories
Type: bool
Default: false
Create parent directories for the given path if they do not exist
append-to-existing
Type: bool
Default: false
Append to an output file if it already exists