Action: extract

Extract data from plain text, using a pattern

No auto-conversion of numbers takes place; use convert for that. (You can now use convert instead of output-fields to indirectly invoke 'convert' action) You may use named capture groups instead of output-fields but either way fieldnames are restricted to letters, digits and underscores.

Field Summary

Field Name	Type	Description	Default
condition	expression	Only run this action if the condition the specified condition is met	-
input-field	field	Field containing the data	_raw
remove	bool	Remove the field after usage	false
warning	bool	Warn on non-matching events	false
drop	bool	Remove non-matching events	false
pattern	regex	The pattern to match on	-
output-fields	array of fields	Field names where values are stored	-
convert	array of (field,type) pairs	Instead of output-fields, invoke convert action	-

Fields

condition

Type: expression

Only run this action if the condition the specified condition is met

input-field

Type: field

Default: _raw

Field containing the data

Example

Input:

{"uptime":" 10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23"}

Pipe Language Snippet:

extract:
  input-field: uptime
  remove: true
  pattern: 'load average: (\S+), (\S+), (\S+)'
  output-fields:
    - m1
    - m5
    - m15

Output:

{"m1":"0.40","m5":"0.28","m15":"0.23"}

remove

Type: bool
Alias: remove-field
Default: false

Remove the field after usage

Example: Parse output of uptime command

Input:

10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  pattern: 'load average: (\S+), (\S+), (\S+)'
  output-fields:
    - m1
    - m5
    - m15

Output:

{"m1":"0.40","m5":"0.28","m15":"0.23"}

Example: Without input-field removed

Input:

10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23

Pipe Language Snippet:

extract:
  input-field: _raw
  pattern: 'load average: (\S+), (\S+), (\S+)'
  output-fields:
    - m1
    - m5
    - m15

Output:

{"_raw":"10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23","m1":"0.40","m5":"0.28","m15":"0.23"}

warning

Type: bool

Default: false

Warn on non-matching events

Example

Input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  drop: true
  warning: true
  pattern: '(\S+) ms$'
  output-fields:
    - latency

Output:

{"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}
{"latency":"time=0.060"}
[WARN] extract: no captures with regex action-extract step 1
LINE: {"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}

Example: Without warn

Input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  pattern: '(\S+) ms$'
  output-fields:
    - latency

Output:

{"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}
{"latency":"time=0.060"}

drop

Type: bool

Default: false

Remove non-matching events

Example

Input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  drop: true
  pattern: '(\S+) ms$'
  output-fields:
    - latency

Output:

{"latency":"time=0.060"}

Example: Without the drop

Input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  pattern: '(\S+) ms$'
  output-fields:
    - latency

Output:

{"_raw":"PING localhost (127.0.0.1) 56(84) bytes of data."}
{"latency":"time=0.060"}

pattern

Type: regex

The pattern to match on

Example

Input:

num=1
num=2
num=3

Pipe Language Snippet:

extract:
  pattern: 'num=(?P<n>\d+)'

Output:

{"_raw":"num=1","n":"1"}
{"_raw":"num=2","n":"2"}
{"_raw":"num=3","n":"3"}

output-fields

Type: array of fields

Field names where values are stored

Example: extract the round trip time for the ping

Input:

PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.060 ms

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  drop: true
  pattern: '(\S+) ms$'
  output-fields:
    - latency

Output:

{"latency":"time=0.060"}

Example: an optional match may or may not set a field

Input:

4-01
02

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  pattern: '((\d+)-)*(\d+)'
  output-fields: [day, hour]

Output:

{"day":"4","hour":"01"}
{"hour":"02"}

convert

Type: array of (field,type) pairs

Instead of output-fields, invoke convert action

Example: Parse output of uptime command

Input:

 10:34:51 up  2:06,  1 user,  load average: 0.40, 0.28, 0.23

Pipe Language Snippet:

extract:
  input-field: _raw
  remove: true
  pattern: 'load average: (\S+), (\S+), (\S+)'
  convert:
    - m1: num
    - m5: num
    - m15: num

Output:

{"m1":0.40,"m5":0.28,"m15":0.23}