s3
Write events to a S3 bucket file
| Field Name | Description | Type | Default |
|---|---|---|---|
| retry | For operations that could potentially fail | Retry | - |
| batch | Maximum number of events in an output batch. If 'document' send on end of document | integer or the 'document' | - |
| timeout | Interval after which the batch is sent, to keep throughput going | interval | 100ms |
| header | Put a header line before the batch | templated text | - |
| footer | Put a header line after the last line of the batch | templated text | - |
| use-document-marker | Enrich the pipe metadata with a document marker (for document handling in batch mode) | bool | false |
| bucket-name | Bucket Name | string | - |
| disable-object-name-guid | Do not append a GUID to the output object name | bool | false |
| object-name | Object Name | string | - |
| object-name-field | Field containing the Object Name | field | - |
| region | Region | string | - |
| endpoint | S3 Endpoint | string | - |
| access-key | Access Key ID | string | - |
| secret-key | Secret Key ID | string | - |
| security-token | Security Token | string | - |
| session-token | Session Token | string | - |
retry
For operations that could potentially fail
Type: Retry
| Field Name | Description | Type | Default |
|---|---|---|---|
| count | How many attempts to make before declaring failure | integer | - |
| pause | How long to pause before re-trying | duration | - |
| forever | Keep trying until success is declared | bool | false |
count
How many attempts to make before declaring failure
Type: integer
Example
action:
exec:
command: echo 'one two'
retry:
count: 1
output:
{"_raw":"one two"}
pause
How long to pause before re-trying
Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)
Type: duration
Example
action:
exec:
command: echo 'one two'
retry:
count: 6
pause: 10s
output:
{"_raw":"one two"}
forever
Keep trying until success is declared
Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)
Type: bool
Example
action:
exec:
command: echo 'one two'
retry:
forever: true
output:
{"_raw":"one two"}
batch
Maximum number of events in an output batch. If 'document' send on end of document
Type: integer or the 'document'
timeout
Interval after which the batch is sent, to keep throughput going
Type: interval
header
Put a header line before the batch
Type: templated text
footer
Put a header line after the last line of the batch
Type: templated text
use-document-marker
Enrich the pipe metadata with a document marker (for document handling in batch mode)
Type: bool
bucket-name
Bucket Name
Type: string
disable-object-name-guid
Do not append a GUID to the output object name
Type: bool
object-name
Object Name
Type: string
object-name-field
Field containing the Object Name
Type: field
region
Region
Type: string
endpoint
S3 Endpoint
Type: string
access-key
Access Key ID
Type: string
secret-key
Secret Key ID
Type: string
security-token
Security Token
Type: string
session-token
Session Token
Type: string