Skip to main content
Version: 3.3.1

s3

Write events to a S3 bucket file

Field NameDescriptionTypeDefault
retryFor operations that could potentially failRetry-
batchMaximum number of events in an output batch. If 'document' send on end of documentinteger or the 'document'-
timeoutInterval after which the batch is sent, to keep throughput goinginterval100ms
headerPut a header line before the batchtemplated text-
footerPut a header line after the last line of the batchtemplated text-
use-document-markerEnrich the pipe metadata with a document marker (for document handling in batch mode)boolfalse
bucket-nameBucket Namestring-
disable-object-name-guidDo not append a GUID to the output object nameboolfalse
object-nameObject Namestring-
object-name-fieldField containing the Object Namefield-
regionRegionstring-
endpointS3 Endpointstring-
access-keyAccess Key IDstring-
secret-keySecret Key IDstring-
security-tokenSecurity Tokenstring-
session-tokenSession Tokenstring-

retry

For operations that could potentially fail

Type: Retry

Field NameDescriptionTypeDefault
countHow many attempts to make before declaring failureinteger-
pauseHow long to pause before re-tryingduration-
foreverKeep trying until success is declaredboolfalse

count

How many attempts to make before declaring failure

Type: integer

Example

action:

exec:
command: echo 'one two'
retry:
count: 1

output:

{"_raw":"one two"}

pause

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: duration

Example

action:

exec:
command: echo 'one two'
retry:
count: 6
pause: 10s

output:

{"_raw":"one two"}

forever

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Type: bool

Example

action:

exec:
command: echo 'one two'
retry:
forever: true

output:

{"_raw":"one two"}

batch

Maximum number of events in an output batch. If 'document' send on end of document

Type: integer or the 'document'

timeout

Interval after which the batch is sent, to keep throughput going

Type: interval

Put a header line before the batch

Type: templated text

Put a header line after the last line of the batch

Type: templated text

use-document-marker

Enrich the pipe metadata with a document marker (for document handling in batch mode)

Type: bool

bucket-name

Bucket Name

Type: string

disable-object-name-guid

Do not append a GUID to the output object name

Type: bool

object-name

Object Name

Type: string

object-name-field

Field containing the Object Name

Type: field

region

Region

Type: string

endpoint

S3 Endpoint

Type: string

access-key

Access Key ID

Type: string

secret-key

Secret Key ID

Type: string

security-token

Security Token

Type: string

session-token

Session Token

Type: string