stream

Create a new field calculated on historical data

Currently, the only operation supported is delta, which gives the difference between the current value and the last value, together with the elapsed time in milliseconds.

If marker is specified, a new event will be created, otherwise the existing event will be enriched

Example

input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

action:

stream:
  input-time: time
  watch: data
  marker:
    - delta
  operation: delta

output:

{"_marker":"delta","delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:06.479Z","data":1}
{"_marker":"delta","delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"_marker":"delta","delta":2,"elapsed":700}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Field Name	Description	Type	Default
condition	Only run this action if the condition the specified condition is met	expression	-
input-time	Use incoming time, instead of current system time	iso time field	-
watch-field	A field to watch for changes	field	-
operation	A stream operation (delta is the only supported value)	string	-
only-changes	Let elapsed field contain the time in milliseconds since the actual change occurred	bool	false
output-field	Use this field to store the difference	field	delta
elapsed-field	Field name to store elapsed time	field	elapsed
group-by	Track differences between field values belonging to different groups	field	-
marker	The marker to use for the newly generated event	(string,string) pair	-

condition

Only run this action if the condition the specified condition is met

Type: expression

input-time

Use incoming time, instead of current system time

Type: iso time field

watch-field

A field to watch for changes

Type: field

operation

A stream operation (delta is the only supported value)

Type: string

only-changes

Let elapsed field contain the time in milliseconds since the actual change occurred

Type: bool

output-field

Use this field to store the difference

Type: field

elapsed-field

Field name to store elapsed time

Type: field

group-by

Track differences between field values belonging to different groups

Type: field

Example: Here the change in state is tracked for each value of name

input:

{"time":"2020-02-03T15:34:45.149Z","name":"john","state":0}
{"time":"2020-02-03T15:34:46.149Z","name":"jane","state":0}
{"time":"2020-02-03T15:34:47.149Z","name":"john","state":1}
{"time":"2020-02-03T15:34:48.149Z","name":"jane","state":1}
{"time":"2020-02-03T15:34:49.149Z","name":"john","state":1}
{"time":"2020-02-03T15:34:50.149Z","name":"jane","state":1}

action:

stream:
  input-time: time
  watch: state
  group-by: name
  operation: delta

output:

{"time":"2020-02-03T15:34:45.149Z","name":"john","state":0,"delta":0,"elapsed":0}
{"time":"2020-02-03T15:34:46.149Z","name":"jane","state":0,"delta":0,"elapsed":0}
{"time":"2020-02-03T15:34:47.149Z","name":"john","state":1,"delta":1,"elapsed":2000}
{"time":"2020-02-03T15:34:48.149Z","name":"jane","state":1,"delta":1,"elapsed":2000}
{"time":"2020-02-03T15:34:49.149Z","name":"john","state":1,"delta":0,"elapsed":2000}
{"time":"2020-02-03T15:34:50.149Z","name":"jane","state":1,"delta":0,"elapsed":2000}

marker

The marker to use for the newly generated event

Type: (string,string) pair

Example: With a marker a new event is created

input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

action:

stream:
  input-time: time
  watch: data
  operation: delta
  marker: [DELTA]

output:

{"_marker":"DELTA","delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:06.479Z","data":1}
{"_marker":"DELTA","delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"_marker":"DELTA","delta":2,"elapsed":700}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Example: Without marker the existing event is enriched

input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

action:

stream:
  input-time: time
  watch: data
  operation: delta

output:

{"time":"2018-07-09T15:09:06.479Z","data":1,"delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:07.479Z","data":2,"delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:08.179Z","data":4,"delta":2,"elapsed":700}