Action: collapse

Converts JSON records to another format, like CSV or key-value pairs

This is the opposite action to expand. You must specify output-field to receive the result.

Field Summary

Field Name	Type	Description	Default
condition	expression	Only run this action if the condition the specified condition is met	-
input-field	field	Field containing data	-
remove	bool	By default all other fields are consumed, leaving the output-field	true
delim	string	A separator of output values	,
output-field	field	A field where data is to be written	-
document-mode	bool	Combined with "array" mode to collapse events into a single document	false
csv	CsvEnum	Collapse the fields in the event using CSV mode	-
key-value	KeyValueEnum	Collapse the fields in the event using Key-Value mode	-
array	ArrayEnum	Take an array or group of events (when 'document-mode: true') and merge into one text event	-

Fields

condition

Type: expression

Only run this action if the condition the specified condition is met

input-field

Type: field

Field containing data

remove

Type: bool

Default: true

By default all other fields are consumed, leaving the output-field

delim

Type: string

Default: ,

A separator of output values

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  csv: true
  output-field: row
  delim: ' | '

Output:

{"row":"1 | 2"}

output-field

Type: field

A field where data is to be written

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  csv: true
  output-field: row

Output:

{"row":"1,2"}

document-mode

Type: bool

Default: false

Combined with "array" mode to collapse events into a single document

csv

Type: CsvEnum

Collapse the fields in the event using CSV mode

Field Name	Type	Description	Default
yes	bool	Simply use the action without any additional options	false
fields	CsvFields	JSON to CSV	-

  yes

Type: bool

Default: false

Simply use the action without any additional options

  fields

Type: CsvFields

JSON to CSV

Field Name	Type	Description	Default
header-field	field	Write the column names to this field	-
header-field-types	bool	With header-field, write out column names and types	false
header-field-on-change	bool	With 'header-field`, only write out headers if columns change	false
null-value	string	A substitue string value to be used in the event that a field is null

  header-field

Type: field

Write the column names to this field

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  csv:
    header-field: header
  output-field: row

Output:

{"row":"1,2","header":"one,two"}

  header-field-types

Type: bool

Default: false

With header-field, write out column names and types

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  csv:
    header-field: header
    header-field-types: true
  output-field: row

Output:

{"row":"1,2","header":"one:num,two:num"}

  header-field-on-change

Type: bool

Default: false

With 'header-field`, only write out headers if columns change

Example

Input:

{"a":1,"b":"one"}
{"a":2,"b":"two"}

Pipe Language Snippet:

collapse:
  csv:
    header-field: header
    header-field-on-change: true
  output-field: row

Output:

{"row":"1,one","header":"a,b"}
{"row":"2,two"}

  null-value

Type: string

Default: ``

A substitue string value to be used in the event that a field is null

Example

Input:

{"num":1,"str":"one"}
{"num":2,"str":"two"}
{"num":null,"str":"NaN"}

Pipe Language Snippet:

collapse:
  csv:
    null-value: '-'
  output-field: row

Output:

{"row":"1,one"}
{"row":"2,two"}
{"row":"-,NaN"}

key-value

Type: KeyValueEnum

Collapse the fields in the event using Key-Value mode

Field Name	Type	Description	Default
yes	bool	Simply use the action without any additional options	false
fields	KeyValueFields	Key Value collapse options	-

  yes

Type: bool

Default: false

Simply use the action without any additional options

Example

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  key-value: true
  output-field: row

Output:

{"row":"one=1,two=2"}

  fields

Type: KeyValueFields

Key Value collapse options

Field Name	Type	Description	Default
key-value-delim	string	Separator between the key and the value	=

  key-value-delim

Type: string

Default: =

Separator between the key and the value

Example: override usual '='

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  key-value:
    key-value-delim: ':'
  output-field: row

Output:

{"row":"one:1,two:2"}

Example: delimiters and separators can be special or multiple

Input:

{"one":1,"two":2}

Pipe Language Snippet:

collapse:
  delim: '\n'
  key-value:
    key-value-delim: '{pipe="metrics"} '
  output-field: row

Output:

{"row":"one{pipe=\"metrics\"} 1\ntwo{pipe=\"metrics\"} 2"}

array

Type: ArrayEnum

Take an array or group of events (when 'document-mode: true') and merge into one text event

Field Name	Type	Description	Default
yes	bool	Simply use the action without any additional options	false
fields	ArrayFields	Array collapse options	-

  yes

Type: bool

Default: false

Simply use the action without any additional options

Example

Input:

{"one-two":[1,2]}

Pipe Language Snippet:

collapse:
  input-field: one-two
  output-field: row
  array: true

Output:

{"row":"1,2"}

  fields

Type: ArrayFields

Array collapse options

Field Name	Type	Description	Default
collect-over-field	field	This is useful when processing the output of transition	-

  collect-over-field

Type: field

This is useful when processing the output of transition

Example

Input:

{"one-two":[{"num":1,"str":"one"},{"num":2,"str":"two"}]}

Pipe Language Snippet:

collapse:
  input-field: one-two
  array:
    collect-over-field: num
  output-field: row

Output:

{"row":"1,2"}