Skip to main content
Version: 3.5.2

Action: filter

Removes events, based on some given conditions

Field Summary

Field NameTypeDescriptionDefault
schemaarray of fieldsAccept events that contain only given fields-
conditionstringAccept events only if given expression is true-
patternsarray of (field,regex) pairsPatterns that must match the field values for the event to go through-
excludearray of (field,regex) pairsPatterns that must not match for the event to go through-
emptyDrop all empty events (events with no fields, empty arrays, or empty strings)-

Fields

schema

Type: array of fields

Accept events that contain only given fields

This removes any fields that are not listed!

Example

Input:

{"first-name":"John"}
{"first-name":"Jane"}
{"last-name":"Doe"}

Pipe Language Snippet:

filter:
schema:
- last-name

Output:

{"last-name":"Doe"}

Example

Input:

{"num":1,"text":"one"}
{"num":2,"text":"two","comment":"is even"}
{"num":3,"text":"three"}

Pipe Language Snippet:

filter:
schema:
- num
- comment

Output:

{"num":2,"comment":"is even"}

condition

Type: string

Accept events only if given expression is true

The expression is Lua. If the field might not exist, guard against warnings like so: num and num > 2

Example

Input:

{"first_name":"John"}
{"first_name":"Jane"}

Pipe Language Snippet:

filter:
condition: first_name == "Jane"

Output:

{"first_name":"Jane"}

Example

Input:

{"num":1,"text":"one"}
{"num":2,"text":"two","comment":"is even"}
{"num":3,"text":"three"}

Pipe Language Snippet:

filter:
condition: num > 2

Output:

{"num":3,"text":"three"}

patterns

Type: array of (field,regex) pairs

Patterns that must match the field values for the event to go through

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
patterns:
- name: Jane

Output:

{"name":"Jane Doe"}

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
patterns:
- name: ^Ja

Output:

{"name":"Jane Doe"}

exclude

Type: array of (field,regex) pairs

Patterns that must not match for the event to go through

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
exclude:
- name: Jane

Output:

{"name":"John Doe"}

Example

Input:

{"name":"John Doe"}
{"name":"Jane Doe"}

Pipe Language Snippet:

filter:
exclude:
- name: ^Ja

Output:

{"name":"John Doe"}

empty

Drop all empty events (events with no fields, empty arrays, or empty strings)