Skip to main content
Version: 3.4.0

extract

Extract data from plain text, using a pattern

No auto-conversion of numbers takes place; use convert for that. (You can now use convert instead of output-fields to indirectly invoke 'convert' action) You may use named capture groups instead of output-fields but either way fieldnames are restricted to letters, digits and underscores.

Field NameDescriptionTypeDefault
conditionOnly run this action if the condition the specified condition is metexpression-
input-fieldField containing the datafield_raw
removeRemove the field after usageboolfalse
warningWarn on non-matching eventsboolfalse
dropRemove non-matching eventsboolfalse
patternThe pattern to match onregex-
output-fieldsField names where values are storedarray of fields-
convertInstead of output-fields, invoke convert actionarray of (field,type) pairs-

condition

Only run this action if the condition the specified condition is met

Type: expression

input-field

Field containing the data

Type: field

Example

input:

{"uptime":" 10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23"}

action:

extract:
input-field: uptime
remove: true
pattern: 'load average: (\S+), (\S+), (\S+)'
output-fields:
- m1
- m5
- m15

output:

{"m1":"0.40","m5":"0.28","m15":"0.23"}

remove

Remove the field after usage

Type: bool

Example: Parse output of uptime command

input:

10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23

action:

extract:
input-field: _raw
remove: true
pattern: 'load average: (\S+), (\S+), (\S+)'
output-fields:
- m1
- m5
- m15

output:

{"m1":"0.40","m5":"0.28","m15":"0.23"}

Example: Without input-field removed

input:

10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23

action:

extract:
input-field: _raw
pattern: 'load average: (\S+), (\S+), (\S+)'
output-fields:
- m1
- m5
- m15

output:

{"_raw":"10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23","m1":"0.40","m5":"0.28","m15":"0.23"}

warning

Warn on non-matching events

Type: bool

Example

input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

action:

extract:
input-field: _raw
remove: true
drop: true
warning: true
pattern: '(\S+) ms$'
output-fields:
- latency

output:

{"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}
{"latency":"time=0.060"}
[WARN] extract: no captures with regex action-extract step 1
LINE: {"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}

Example: Without warn

input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

action:

extract:
input-field: _raw
remove: true
pattern: '(\S+) ms$'
output-fields:
- latency

output:

{"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}
{"latency":"time=0.060"}

drop

Remove non-matching events

Type: bool

Example

input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

action:

extract:
input-field: _raw
remove: true
drop: true
pattern: '(\S+) ms$'
output-fields:
- latency

output:

{"latency":"time=0.060"}

Example: Without the drop

input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

action:

extract:
input-field: _raw
remove: true
pattern: '(\S+) ms$'
output-fields:
- latency

output:

{"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}
{"latency":"time=0.060"}

pattern

The pattern to match on

Type: regex

Example

input:

num=1
num=2
num=3

action:

extract:
pattern: 'num=(?P<n>\d+)'

output:

{"_raw":"num=1","n":"1"}
{"_raw":"num=2","n":"2"}
{"_raw":"num=3","n":"3"}

output-fields

Field names where values are stored

Note: Field names must be composed of letters, digits, and underscores

Type: array of fields

Example: extract the round trip time for the ping

input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

action:

extract:
input-field: _raw
remove: true
drop: true
pattern: '(\S+) ms$'
output-fields:
- latency

output:

{"latency":"time=0.060"}

Example: an optional match may or may not set a field

input:

4-01
02

action:

extract:
input-field: _raw
remove: true
pattern: '((\d+)-)*(\d+)'
output-fields: [day, hour]

output:

{"day":"4","hour":"01"}
{"hour":"02"}

convert

Instead of output-fields, invoke convert action

Type: array of (field,type) pairs

Example: Parse output of uptime command

input:

 10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23

action:

extract:
input-field: _raw
remove: true
pattern: 'load average: (\S+), (\S+), (\S+)'
convert:
- m1: num
- m5: num
- m15: num

output:

{"m1":0.40,"m5":0.28,"m15":0.23}