Skip to main content
Version: 3.5.0

Action: collapse

Converts JSON records to another format, like CSV or key-value pairs

This is the opposite action to expand. You must specify output-field to receive the result.

Field Summary

Field NameTypeDescriptionDefault
conditionexpressionOnly run this action if the condition the specified condition is met-
input-fieldfieldField containing data-
removeboolBy default all other fields are consumed, leaving the output-fieldtrue
delimstringA separator of output values,
output-fieldfieldA field where data is to be written-
document-modeboolCombined with "array" mode to collapse events into a single documentfalse
csvCsvEnumCollapse the fields in the event using CSV mode-
key-valueKeyValueEnumCollapse the fields in the event using Key-Value mode-
arrayArrayEnumTake an array or group of events (when 'document-mode: true') and merge into one text event-

Fields

condition

Type: expression

Only run this action if the condition the specified condition is met

input-field

Type: field

Field containing data

remove

Type: bool

Default: true

By default all other fields are consumed, leaving the output-field

delim

Type: string

Default: ,

A separator of output values

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
csv: true
output-field: row
delim: ' | '

Output:

{"row":"1 | 2"}

output-field

Type: field

A field where data is to be written

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
csv: true
output-field: row

Output:

{"row":"1,2"}

document-mode

Type: bool

Default: false

Combined with "array" mode to collapse events into a single document

csv

Type: CsvEnum

Collapse the fields in the event using CSV mode

Field NameTypeDescriptionDefault
yesboolSimply use the action without any additional optionsfalse
fieldsCsvFieldsJSON to CSV-

  yes

Type: bool

Default: false

Simply use the action without any additional options

  fields

Type: CsvFields

JSON to CSV

Field NameTypeDescriptionDefault
header-fieldfieldWrite the column names to this field-
header-field-typesboolWith header-field, write out column names and typesfalse
header-field-on-changeboolWith 'header-field`, only write out headers if columns changefalse
null-valuestringA substitue string value to be used in the event that a field is null

  header-field

Type: field

Write the column names to this field

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
csv:
header-field: header
output-field: row

Output:

{"row":"1,2","header":"one,two"}

  header-field-types

Type: bool

Default: false

With header-field, write out column names and types

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
csv:
header-field: header
header-field-types: true
output-field: row

Output:

{"row":"1,2","header":"one:num,two:num"}

  header-field-on-change

Type: bool

Default: false

With 'header-field`, only write out headers if columns change

Example

Input:

{"a":1,"b":"one"}
{"a":2,"b":"two"}

Pipe Language Snippet:

collapse:
csv:
header-field: header
header-field-on-change: true
output-field: row

Output:

{"row":"1,one","header":"a,b"}
{"row":"2,two"}

  null-value

Type: string

Default: ``

A substitue string value to be used in the event that a field is null

Example

Input:

{"num":1,"str":"one"}
{"num":2,"str":"two"}
{"num":null,"str":"NaN"}

Pipe Language Snippet:

collapse:
csv:
null-value: '-'
output-field: row

Output:

{"row":"1,one"}
{"row":"2,two"}
{"row":"-,NaN"}

key-value

Type: KeyValueEnum

Collapse the fields in the event using Key-Value mode

Field NameTypeDescriptionDefault
yesboolSimply use the action without any additional optionsfalse
fieldsKeyValueFieldsKey Value collapse options-

  yes

Type: bool

Default: false

Simply use the action without any additional options

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
key-value: true
output-field: row

Output:

{"row":"one=1,two=2"}

  fields

Type: KeyValueFields

Key Value collapse options

Field NameTypeDescriptionDefault
key-value-delimstringSeparator between the key and the value=

  key-value-delim

Type: string

Default: =

Separator between the key and the value

Example: override usual '='

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
key-value:
key-value-delim: ':'
output-field: row

Output:

{"row":"one:1,two:2"}

Example: delimiters and separators can be special or multiple

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
delim: '\n'
key-value:
key-value-delim: '{pipe="metrics"} '
output-field: row

Output:

{"row":"one{pipe=\"metrics\"} 1\ntwo{pipe=\"metrics\"} 2"}

array

Type: ArrayEnum

Take an array or group of events (when 'document-mode: true') and merge into one text event

Field NameTypeDescriptionDefault
yesboolSimply use the action without any additional optionsfalse
fieldsArrayFieldsArray collapse options-

  yes

Type: bool

Default: false

Simply use the action without any additional options

Example

Input:

{"one-two":[1,2]}

Pipe Language Snippet:

collapse:
input-field: one-two
output-field: row
array: true

Output:

{"row":"1,2"}

  fields

Type: ArrayFields

Array collapse options

Field NameTypeDescriptionDefault
collect-over-fieldfieldThis is useful when processing the output of transition-

  collect-over-field

Type: field

This is useful when processing the output of transition

Example

Input:

{"one-two":[{"num":1,"str":"one"},{"num":2,"str":"two"}]}

Pipe Language Snippet:

collapse:
input-field: one-two
array:
collect-over-field: num
output-field: row

Output:

{"row":"1,2"}