Action: filter

Removes events, based on some given conditions

Field Summary

Field Name	Type	Description	Default
schema	array of fields	Accept events that contain only given fields	-
condition	string	Accept events only if given expression is true	-
patterns	array of (field,regex) pairs	Patterns that must match the field values for the event to go through	-
exclude	array of (field,regex) pairs	Patterns that must not match for the event to go through	-
empty		Drop all empty events (events with no fields, empty arrays, or empty strings)	-

Fields

schema

Type: array of fields

Accept events that contain only given fields

This removes any fields that are not listed!

Example

Input:

{"first-name":"John"}
{"first-name":"Jane"}
{"last-name":"Doe"}

Pipe Language Snippet:

filter:
  schema:
  - last-name

Output:

{"last-name":"Doe"}

Example

Input:

{"num":1,"text":"one"}
{"num":2,"text":"two","comment":"is even"}
{"num":3,"text":"three"}

Pipe Language Snippet:

filter:
  schema:
  - num
  - comment

Output:

{"num":2,"comment":"is even"}

condition

Type: string

Accept events only if given expression is true

The expression is Lua. If the field might not exist, guard against warnings like so: num and num > 2

Example

Input:

{"first_name":"John"}
{"first_name":"Jane"}

Pipe Language Snippet:

filter:
  condition: first_name == "Jane"

Output:

{"first_name":"Jane"}

Example

Input:

{"num":1,"text":"one"}
{"num":2,"text":"two","comment":"is even"}
{"num":3,"text":"three"}

Pipe Language Snippet:

filter:
  condition: num > 2

Output:

{"num":3,"text":"three"}

patterns

Type: array of (field,regex) pairs

Patterns that must match the field values for the event to go through

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
  patterns:
  - name: Jane

Output:

{"name":"Jane Doe"}

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
  patterns:
  - name: ^Ja

Output:

{"name":"Jane Doe"}

exclude

Type: array of (field,regex) pairs

Patterns that must not match for the event to go through

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
  exclude:
  - name: Jane

Output:

{"name":"John Doe"}

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
  exclude:
  - name: ^Ja

Output:

{"name":"John Doe"}

empty

Drop all empty events (events with no fields, empty arrays, or empty strings)