Output: azure-blob

Send data to a Microsoft Azure Storage Blob (Block Storage)

Field Summary

Field Name	Type	Description	Default
container-name	string	The storage service container for created blobs	-
blob-name	string	The name for the output blob, this will be suffixed by a GUID by default	-
disable-blob-name-guid	bool	Do not append a GUID to the output blob name	false
guid-prefix	string	GUID Prefix, will be prepended to the GUID, the default value is "/"	-
guid-suffix	string	GUID Suffix, will be appended to the GUID if specified	-
storage-account	string	The Storage Account Name to be used (credential)	-
storage-master-key	string	The Storage Master Key to be used (credential)	-
retry	Retry	For operations that could potentially fail	-
batch	integer or the 'document'	Maximum number of events in an output batch. If 'document' send on end of document	-
timeout	interval	Interval after which the batch is sent, to keep throughput going	100ms
header	templated text	Put a header line before the batch	-
footer	templated text	Put a header line after the last line of the batch	-
use-document-marker	bool	Enrich the pipe metadata with a document marker (for document handling in batch mode)	false
content-type	string	Set the created blob the the following content-type	text/plain
input-field	field	Send only the content of the specified field to the blob	-
scuba	Scuba	If this Pipe is sending data to Scuba Lite, optionally have the Pipe configure the Scuba Lite instance	-
preprocessors	OutputPreProcessor	Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified	-

Fields

container-name

Type: string

The storage service container for created blobs

blob-name

Type: string

The name for the output blob, this will be suffixed by a GUID by default

disable-blob-name-guid

Type: bool

Default: false

Do not append a GUID to the output blob name

guid-prefix

Type: string

GUID Prefix, will be prepended to the GUID, the default value is "/"

guid-suffix

Type: string

GUID Suffix, will be appended to the GUID if specified

storage-account

Type: string

The Storage Account Name to be used (credential)

storage-master-key

Type: string

The Storage Master Key to be used (credential)

retry

Type: Retry

For operations that could potentially fail

Field Name	Type	Description	Default
count	integer	How many attempts to make before declaring failure	-
pause	duration	How long to pause before re-trying	-
forever	bool	Keep trying until success is declared	false

  count

Type: integer

How many attempts to make before declaring failure

Example

Pipe Language Snippet:

exec:
  command: echo 'one two'
  retry:
    count: 1

Output:

{"_raw":"one two"}

  pause

Type: duration

How long to pause before re-trying

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Example

Pipe Language Snippet:

exec:
  command: echo 'one two'
  retry:
    count: 6
    pause: 10s

Output:

{"_raw":"one two"}

  forever

Type: bool

Default: false

Keep trying until success is declared

Accepts human-friendly formats, like 1m (for 1 minute) and 4h (for 4 hours)

Example

Pipe Language Snippet:

exec:
  command: echo 'one two'
  retry:
    forever: true

Output:

{"_raw":"one two"}

batch

Type: integer or the 'document'

Maximum number of events in an output batch. If 'document' send on end of document

timeout

Type: interval

Default: 100ms

Interval after which the batch is sent, to keep throughput going

header

Type: templated text

Put a header line before the batch

footer

Type: templated text

Put a header line after the last line of the batch

use-document-marker

Type: bool

Default: false

Enrich the pipe metadata with a document marker (for document handling in batch mode)

content-type

Type: string

Default: text/plain

Set the created blob the the following content-type

input-field

Type: field

Send only the content of the specified field to the blob

scuba

Type: Scuba

If this Pipe is sending data to Scuba Lite, optionally have the Pipe configure the Scuba Lite instance

Field Name	Type	Description	Default
host	string	The Scuba Lite instance (for example: https://scuba-lite.example.com)	-
insecure	bool	Allow self-signed certificates for Scuba Lite connections	false
token	string	The Scuba Lite API token	-
table-name	string	The Scuba Lite table name	-
file-pattern	string	The Scuba Lite file search pattern	-
columns	ScubaLiteColumns		-

  host

Type: string

The Scuba Lite instance (for example: https://scuba-lite.example.com)

  insecure

Type: bool

Default: false

Allow self-signed certificates for Scuba Lite connections

  token

Type: string

The Scuba Lite API token

  table-name

Type: string

The Scuba Lite table name

  file-pattern

Type: string

The Scuba Lite file search pattern

  columns

Type: ScubaLiteColumns

Field Name	Type	Description	Default
name	string	The name of the column	-
type	ScubaLiteColumnType	The type of the column	-
actor	bool	Specify if the column is an actor	false

  name

Type: string

The name of the column

  type

Type: ScubaLiteColumnType

The type of the column

Field Name	Type	Description	Default
string			-
time	string		-

  string

  time

Type: string

  actor

Type: bool

Default: false

Specify if the column is an actor

preprocessors

Type: OutputPreProcessor

Preprocessors (process data before making it available for upload) these processors will be run in the order they are specified

Field Name	Type	Description	Default
gzip		Gzip the output data	-

  gzip

Gzip the output data