Skip to main content
Version: 3.5.0

Output: azure-blob

Send data to a Microsoft Azure Storage Blob (Block Storage)

Field Summary

Field NameTypeDescriptionDefault
container-namestringThe storage service container for created blobs-
blob-namestringThe name for the output blob, this will be suffixed by a GUID by default-
disable-blob-name-guidboolDo not append a GUID to the output blob namefalse
guid-prefixstringGUID Prefix, will be prepended to the GUID, the default value is "/"-
guid-suffixstringGUID Suffix, will be appended to the GUID if specified-
storage-accountstringThe Storage Account Name to be used (credential)-
storage-master-keystringThe Storage Master Key to be used (credential)-
retryRetryFor operations that could potentially fail-
batchinteger or the 'document'Maximum number of events in an output batch. If 'document' send on end of document-
timeoutintervalInterval after which the batch is sent, to keep throughput going100ms
headertemplated textPut a header line before the batch-
footertemplated textPut a header line after the last line of the batch-
use-document-markerboolEnrich the pipe metadata with a document marker (for document handling in batch mode)false
content-typestringSet the created blob the the following content-typetext/plain
input-fieldfieldSend only the content of the specified field to the blob-
scubaScubaIf this Pipe is sending data to Scuba Lite, optionally have the Pipe configure the Scuba Lite instance-
preprocessorsOutputPreProcessorPreprocessors (process data before making it available for upload) these processors will be run in the order they are specified-

Fields

container-name

Type: string

The storage service container for created blobs

blob-name

Type: string

The name for the output blob, this will be suffixed by a GUID by default

disable-blob-name-guid

Type: bool

Default: false

Do not append a GUID to the output blob name

guid-prefix

Type: string

GUID Prefix, will be prepended to the GUID, the default value is "/"

guid-suffix

Type: string

GUID Suffix, will be appended to the GUID if specified

storage-account

Type: string

The Storage Account Name to be used (credential)

storage-master-key

Type: string

The Storage Master Key to be used (credential)

retry

Type: Retry

For operations that could potentially fail

Field NameTypeDescriptionDefault
countintegerHow many attempts to make before declaring failure-
pausedurationHow long to pause before re-trying-
foreverboolKeep trying until success is declaredfalse

  count

Type: integer

How many attempts to make before declaring failure

Example

Pipe Language Snippet:

exec:
command: echo 'one two'
retry:
count: 1

Output:

{"_raw":"one two"}

  pause

Type: duration

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Example

Pipe Language Snippet:

exec:
command: echo 'one two'
retry:
count: 6
pause: 10s

Output:

{"_raw":"one two"}

  forever

Type: bool

Default: false

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Example

Pipe Language Snippet:

exec:
command: echo 'one two'
retry:
forever: true

Output:

{"_raw":"one two"}

batch

Type: integer or the 'document'

Maximum number of events in an output batch. If 'document' send on end of document

timeout

Type: interval

Default: 100ms

Interval after which the batch is sent, to keep throughput going

Type: templated text

Put a header line before the batch

Type: templated text

Put a header line after the last line of the batch

use-document-marker

Type: bool

Default: false

Enrich the pipe metadata with a document marker (for document handling in batch mode)

content-type

Type: string

Default: text/plain

Set the created blob the the following content-type

input-field

Type: field

Send only the content of the specified field to the blob

scuba

Type: Scuba

If this Pipe is sending data to Scuba Lite, optionally have the Pipe configure the Scuba Lite instance

Field NameTypeDescriptionDefault
hoststringThe Scuba Lite instance (for example: https://scuba-lite.example.com)-
insecureboolAllow self-signed certificates for Scuba Lite connectionsfalse
tokenstringThe Scuba Lite API token-
table-namestringThe Scuba Lite table name-
file-patternstringThe Scuba Lite file search pattern-
columnsScubaLiteColumns-

  host

Type: string

The Scuba Lite instance (for example: https://scuba-lite.example.com)

  insecure

Type: bool

Default: false

Allow self-signed certificates for Scuba Lite connections

  token

Type: string

The Scuba Lite API token

  table-name

Type: string

The Scuba Lite table name

  file-pattern

Type: string

The Scuba Lite file search pattern

  columns

Type: ScubaLiteColumns

Field NameTypeDescriptionDefault
namestringThe name of the column-
typeScubaLiteColumnTypeThe type of the column-
actorboolSpecify if the column is an actorfalse

  name

Type: string

The name of the column

  type

Type: ScubaLiteColumnType

The type of the column

Field NameTypeDescriptionDefault
string-
timestring-

  string

  time

Type: string

  actor

Type: bool

Default: false

Specify if the column is an actor

preprocessors

Type: OutputPreProcessor

Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified

Field NameTypeDescriptionDefault
gzipGzip the output data-

  gzip

Gzip the output data