Skip to main content
Version: 3.5.3

Action: enrich

Allows using CSV lookup to enrich data

This step requires special attention, as it needs to have its files attached to the pipe that will use the table lookup mechanisms. This is convenient as updated tables can be bundled with your pipes and managed centrally.

Please see the discussion and the example provided at the end of the Table Lookup section.

Example: Add known network ports to events

File: lookup.csv

port,service
22,ssh
80,http
443,https

Input:

{"port":22}
{"port":80}
{"port":100}
{"port":443}

Pipe Language Snippet:

enrich:
lookup-file: lookup.csv
match:
- type: num
event-field: port
lookup-field: port
add:
event-field: service
lookup-field: service

Output:

{"port":22,"service":"ssh"}
{"port":80,"service":"http"}
{"port":100}
{"port":443,"service":"https"}

Field Summary

Field NameTypeDescriptionDefault
conditionexpressionOnly run this action if the condition the specified condition is met-
lookup-filepathA file containing lookups-
dynamicboolDo not panic if lookup-file does not immediately exist at pipe creation timefalse
addAddEnumDetail on what to add to the event, based on the match.

If there is no default value, then the output field will not be added to the event. |-| |match|Match|Detail on what to match on, associating event fields and lookup fields|-|

Fields

condition

Type: expression

Only run this action if the condition the specified condition is met

lookup-file

Type: path

A file containing lookups

dynamic

Type: bool

Default: false

Do not panic if lookup-file does not immediately exist at pipe creation time

add

Type: AddEnum

Detail on what to add to the event, based on the match. If there is no default value, then the output field will not be added to the event.

Field NameTypeDescriptionDefault
addAddA field value to add to the event-
add-multipleAddMultipleField values to add to the event-

  add

Type: Add

A field value to add to the event

Field NameTypeDescriptionDefault
event-fieldfieldField name to be added to the event-
lookup-fieldfieldField (CSV header) to lookup data to be place in event-field-
default-valuefieldYAML formatted default value if the event is empty-

  event-field

Type: field

Field name to be added to the event

  lookup-field

Type: field

Field (CSV header) to lookup data to be place in event-field

  default-value

Type: field

YAML formatted default value if the event is empty

Example: type=cidr

File: lookup7.csv

office,network
Customer,192.168.26.0/28
Headquarters,192.168.85.0/24

Input:

{"address":"192.168.26.10"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100"}

Pipe Language Snippet:

enrich:
lookup-file: lookup7.csv
match:
- type: cidr
event-field: address
lookup-field: network
add:
event-field: office
lookup-field: office
default-value: unknown

Output:

{"address":"192.168.26.10","office":"Customer"}
{"address":"192.168.26.100","office":"unknown"}
{"address":"192.168.85.100","office":"Headquarters"}

  add-multiple

Type: AddMultiple

Field values to add to the event

Field NameTypeDescriptionDefault
event-fieldsarray of (field,value) pairsAdd multiple fields to a single event based on a single match, providing a default-

  event-fields

Type: array of (field,value) pairs

Add multiple fields to a single event based on a single match, providing a default

The limitation with this shortcut is that the lookup field name should be the same as the event field

Example: type=cidr

File: lookup8.csv

office,network
Customer,192.168.26.0/28
Headquarters,192.168.85.0/24

Input:

{"address":"192.168.26.10"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100"}

Pipe Language Snippet:

enrich:
lookup-file: lookup8.csv
match:
- type: cidr
event-field: address
lookup-field: network
add:
event-fields:
- office: unknown

Output:

{"address":"192.168.26.10","office":"Customer"}
{"address":"192.168.26.100","office":"unknown"}
{"address":"192.168.85.100","office":"Headquarters"}

match

Type: Match

Detail on what to match on, associating event fields and lookup fields

Field NameTypeDescriptionDefault
typestringType of match, one of the following: str, num, cidr, ip, num-range, num-list, str-list-
event-fieldfieldEvent field matched-
lookup-fieldfieldLookup field matched-

  type

Type: string

Possible Values: str, num, cidr, ip, num-range, num-list, str-list

Type of match, one of the following: str, num, cidr, ip, num-range, num-list, str-list

Example: matching strings

File: lookup1.csv

port,service
22,ssh
80,http
443,https

Input:

{"service":"ssh"}
{"service":"http"}
{"service":"unknown"}
{"service":"https"}

Pipe Language Snippet:

enrich:
lookup-file: lookup1.csv
match:
- type: str
event-field: service
lookup-field: service
add:
event-field: port
lookup-field: port

Output:

{"service":"ssh","port":"22"}
{"service":"http","port":"80"}
{"service":"unknown"}
{"service":"https","port":"443"}

Example: matching CIDR

File: lookup2.csv

office,network
Customer,192.168.26.0/28
Headquarters,192.168.85.0/24

Input:

{"address":"192.168.26.10"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100"}

Pipe Language Snippet:

enrich:
lookup-file: lookup2.csv
match:
- type: cidr
event-field: address
lookup-field: network
add:
event-field: office
lookup-field: office

Output:

{"address":"192.168.26.10","office":"Customer"}
{"address":"192.168.26.100"}
{"address":"192.168.85.100","office":"Headquarters"}

Example: IP address (if event match.event-field does not match ip address format, the event will be filtered out)

File: lookup3.csv

fqdn,address
domain.io,206.189.28.194

Input:

{"address":"192.168.26.10"}
{"address":"206.189.28.194"}

Pipe Language Snippet:

enrich:
lookup-file: lookup3.csv
match:
- type: ip
event-field: address
lookup-field: address
add:
event-field: fqdn
lookup-field: fqdn

Output:

{"address":"192.168.26.10"}
{"address":"206.189.28.194","fqdn":"domain.io"}

Example: Type is a range of numbers

File: lookup4.csv

range,grouping
0-3,small numbers
3-10,larger numbers

Input:

{"number":1}
{"number":10}
{"number":100}

Pipe Language Snippet:

enrich:
lookup-file: lookup4.csv
match:
- type: num-range
event-field: number
lookup-field: range
add:
event-field: grouping
lookup-field: grouping

Output:

{"number":1,"grouping":"small numbers"}
{"number":10,"grouping":"larger numbers"}
{"number":100}

Example: Type is a list of strings

File: lookup5.csv

lists,grouping
"zero,two,four",even
"one,three,five",odd

Input:

{"number":"one"}
{"number":"two"}
{"number":"three"}

Pipe Language Snippet:

enrich:
lookup-file: lookup5.csv
match:
- type: str-list
event-field: number
lookup-field: lists
add:
event-field: grouping
lookup-field: grouping

Output:

{"number":"one","grouping":"odd"}
{"number":"two","grouping":"even"}
{"number":"three","grouping":"odd"}

Example: CDIR with multiple matches

File: lookup6.csv

source,destination,label
192.168.26.0/24,192.168.26.0/24,sameSide
192.168.85.0/24,192.168.85.0/24,sameSide
192.168.26.0/24,192.168.85.0/24,sameSide
192.168.26.0/24,0.0.0.0/0,outbound
192.168.85.0/24,0.0.0.0/0,outbound
0.0.0.0/0,192.168.26.0/24,inbound
0.0.0.0/0,192.168.85.0/24,inbound
0.0.0.0/0,0.0.0.0/0,unknown

Input:

{"src":"192.168.26.10","dst":"192.168.26.11"}
{"src":"192.168.26.10","dst":"192.168.85.10"}
{"src":"192.168.85.10","dst":"192.168.85.11"}
{"src":"192.168.26.10","dst":"192.168.86.10"}
{"src":"192.168.86.10","dst":"192.168.26.10"}
{"src":"192.168.86.10","dst":"192.168.86.11"}

Pipe Language Snippet:

enrich:
lookup-file: lookup6.csv
match:
- type: cidr
event-field: src
lookup-field: source
- type: cidr
event-field: dst
lookup-field: destination
add:
event-field: traffic-direction
lookup-field: label

Output:

{"src":"192.168.26.10","dst":"192.168.26.11","traffic-direction":"sameSide"}
{"src":"192.168.26.10","dst":"192.168.85.10","traffic-direction":"sameSide"}
{"src":"192.168.85.10","dst":"192.168.85.11","traffic-direction":"sameSide"}
{"src":"192.168.26.10","dst":"192.168.86.10","traffic-direction":"outbound"}
{"src":"192.168.86.10","dst":"192.168.26.10","traffic-direction":"inbound"}
{"src":"192.168.86.10","dst":"192.168.86.11","traffic-direction":"unknown"}

  event-field

Type: field

Event field matched

  lookup-field

Type: field

Lookup field matched