s3

Write events to a S3 bucket file

Field Name	Description	Type	Default
retry	For operations that could potentially fail	Retry	-
batch	Maximum number of events in an output batch. If 'document' send on end of document	integer or the 'document'	-
timeout	Interval after which the batch is sent, to keep throughput going	interval	100ms
header	Put a header line before the batch	templated text	-
footer	Put a header line after the last line of the batch	templated text	-
use-document-marker	Enrich the pipe metadata with a document marker (for document handling in batch mode)	bool	false
input-field	Use the specified field as the content for the file line	field	-
bucket-name	Bucket Name	string	-
disable-object-name-guid	Do not append a GUID to the output object name	bool	false
object-name	Object Name	string	-
object-name-field	Field containing the Object Name	field	-
region	Region	string	-
endpoint	S3 Endpoint	string	-
access-key	Access Key ID	string	-
secret-key	Secret Key ID	string	-
security-token	Security Token	string	-
session-token	Session Token	string	-
preprocessors	Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified	OutputPreProcessor	-

retry

For operations that could potentially fail

Type: Retry

Field Name	Description	Type	Default
count	How many attempts to make before declaring failure	integer	-
pause	How long to pause before re-trying	duration	-
forever	Keep trying until success is declared	bool	false

count

How many attempts to make before declaring failure

Type: integer

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 1

output:

{"_raw":"one two"}

pause

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: duration

Example

action:

exec:
  command: echo 'one two'
  retry:
    count: 6
    pause: 10s

output:

{"_raw":"one two"}

forever

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: bool

Example

action:

exec:
  command: echo 'one two'
  retry:
    forever: true

output:

{"_raw":"one two"}

batch

Maximum number of events in an output batch. If 'document' send on end of document

Type: integer or the 'document'

timeout

Interval after which the batch is sent, to keep throughput going

Type: interval

header

Put a header line before the batch

Type: templated text

footer

Put a header line after the last line of the batch

Type: templated text

use-document-marker

Enrich the pipe metadata with a document marker (for document handling in batch mode)

Type: bool

input-field

Use the specified field as the content for the file line

Type: field

bucket-name

Bucket Name

Type: string

disable-object-name-guid

Do not append a GUID to the output object name

Type: bool

object-name

Object Name

Type: string

object-name-field

Field containing the Object Name

Type: field

region

Region

Type: string

endpoint

S3 Endpoint

Type: string

access-key

Access Key ID

Type: string

secret-key

Secret Key ID

Type: string

security-token

Security Token

Type: string

session-token

Session Token

Type: string

preprocessors

Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified

Type: OutputPreProcessor

Field Name	Description	Type	Default
gzip	Gzip the output data	-	-

gzip

Gzip the output data