collapse

Converts JSON records to another format, like CSV or key-value pairs

This is the opposite action to expand. You must specify output-field to receive the result.

Note: This will remove existing fields

Field Name	Description	Type	Default
condition	Only run this action if the condition the specified condition is met	expression	-
input-field	Field containing data	field	-
remove	By default all other fields are consumed, leaving the output-field	bool	true
delim	A separator of output values	string	,
output-field	A field where data is to be written	field	-
document-mode	Combined with "array" mode to collapse events into a single document	bool	false
csv	Collapse the fields in the event using CSV mode	CsvEnum	-
key-value	Collapse the fields in the event using Key-Value mode	KeyValueEnum	-
array	Take an array or group of events (when 'document-mode: true') and merge into one text event	ArrayEnum	-

condition

Only run this action if the condition the specified condition is met

Type: expression

input-field

Field containing data

Type: field

remove

By default all other fields are consumed, leaving the output-field

Type: bool

delim

A separator of output values

Note: Can be special chars like '\n' (newline) or '\t' (tab) as well

Type: string

Example

input:

{"one":1,"two":2}

action:

collapse:
  csv: true
  output-field: row
  delim: ' | '

output:

{"row":"1 | 2"}

output-field

A field where data is to be written

Type: field

Example

input:

{"one":1,"two":2}

action:

collapse:
  csv: true
  output-field: row

output:

{"row":"1,2"}

document-mode

Combined with "array" mode to collapse events into a single document

Type: bool

csv

Collapse the fields in the event using CSV mode

Type: CsvEnum

Field Name	Description	Type	Default
yes	Simply use the action without any additional options	bool	false
fields	JSON to CSV	CsvFields	-

yes

Simply use the action without any additional options

Type: bool

fields

JSON to CSV

Type: CsvFields

Field Name	Description	Type	Default
header-field	Write the column names to this field	field	-
header-field-types	With header-field, write out column names and types	bool	false
header-field-on-change	With 'header-field`, only write out headers if columns change	bool	false
null-value	A substitue string value to be used in the event that a field is null	string

header-field

Write the column names to this field

Type: field

Example

input:

{"one":1,"two":2}

action:

collapse:
  csv:
    header-field: header
  output-field: row

output:

{"row":"1,2","header":"one,two"}

header-field-types

With header-field, write out column names and types

Type: bool

Example

input:

{"one":1,"two":2}

action:

collapse:
  csv:
    header-field: header
    header-field-types: true
  output-field: row

output:

{"row":"1,2","header":"one:num,two:num"}

header-field-on-change

With 'header-field`, only write out headers if columns change

Type: bool

Example

input:

{"a":1,"b":"one"}
{"a":2,"b":"two"}

action:

collapse:
  csv:
    header-field: header
    header-field-on-change: true
  output-field: row

output:

{"row":"1,one","header":"a,b"}
{"row":"2,two"}

null-value

A substitue string value to be used in the event that a field is null

Type: string

Example

input:

{"num":1,"str":"one"}
{"num":2,"str":"two"}
{"num":null,"str":"NaN"}

action:

collapse:
  csv:
    null-value: '-'
  output-field: row

output:

{"row":"1,one"}
{"row":"2,two"}
{"row":"-,NaN"}

key-value

Collapse the fields in the event using Key-Value mode

Type: KeyValueEnum

Field Name	Description	Type	Default
yes	Simply use the action without any additional options	bool	false
fields	Key Value collapse options	KeyValueFields	-

yes

Simply use the action without any additional options

Type: bool

Example

input:

{"one":1,"two":2}

action:

collapse:
  key-value: true
  output-field: row

output:

{"row":"one=1,two=2"}

fields

Key Value collapse options

Type: KeyValueFields

Field Name	Description	Type	Default
key-value-delim	Separator between the key and the value	string	=

key-value-delim

Separator between the key and the value

Type: string

Example: override usual '='

input:

{"one":1,"two":2}

action:

collapse:
  key-value:
    key-value-delim: ':'
  output-field: row

output:

{"row":"one:1,two:2"}

Example: delimiters and separators can be special or multiple

this converts JSON to Prometheus text format

input:

{"one":1,"two":2}

action:

collapse:
  delim: '\n'
  key-value:
    key-value-delim: '{pipe="metrics"} '
  output-field: row

output:

{"row":"one{pipe=\"metrics\"} 1\ntwo{pipe=\"metrics\"} 2"}

array

Take an array or group of events (when 'document-mode: true') and merge into one text event

Type: ArrayEnum

Field Name	Description	Type	Default
yes	Simply use the action without any additional options	bool	false
fields	Array collapse options	ArrayFields	-

yes

Simply use the action without any additional options

Type: bool

Example

input:

{"one-two":[1,2]}

action:

collapse:
  input-field: one-two
  output-field: row
  array: true

output:

{"row":"1,2"}

fields

Array collapse options

Type: ArrayFields

Field Name	Description	Type	Default
collect-over-field	This is useful when processing the output of transition	field	-

collect-over-field

This is useful when processing the output of transition

Type: field

Example

input:

{"one-two":[{"num":1,"str":"one"},{"num":2,"str":"two"}]}

action:

collapse:
  input-field: one-two
  array:
    collect-over-field: num
  output-field: row

output:

{"row":"1,2"}