Skip to main content
Version: Next

Action: stream

Create a new field calculated on historical data

Currently, the only operation supported is delta, which gives the difference between the current value and the last value, together with the elapsed time in milliseconds.

If marker is specified, a new event will be created, otherwise the existing event will be enriched

Example

Input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Pipe Language Snippet:

stream:
input-time: time
watch: data
marker:
- delta
operation: delta

Output:

{"_marker":"delta","delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:06.479Z","data":1}
{"_marker":"delta","delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"_marker":"delta","delta":2,"elapsed":700}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Field Summary

Field NameTypeDescriptionDefault
conditionexpressionOnly run this action if the condition the specified condition is met-
input-timeiso time fieldUse incoming time, instead of current system time-
watch-fieldfieldA field to watch for changes-
operationstringA stream operation (delta is the only supported value)-
only-changesboolLet elapsed field contain the time in milliseconds since the actual change occurredfalse
output-fieldfieldUse this field to store the differencedelta
elapsed-fieldfieldField name to store elapsed timeelapsed
group-byfieldTrack differences between field values belonging to different groups-
marker(string,string) pairThe marker to use for the newly generated event-

Fields

condition

Type: expression

Only run this action if the condition the specified condition is met

input-time

Type: iso time field

Use incoming time, instead of current system time

watch-field

Type: field
Alias: watch-field

A field to watch for changes

operation

Type: string

Possible Values: delta

A stream operation (delta is the only supported value)

only-changes

Type: bool

Default: false

Let elapsed field contain the time in milliseconds since the actual change occurred

output-field

Type: field

Default: delta

Use this field to store the difference

elapsed-field

Type: field

Default: elapsed

Field name to store elapsed time

group-by

Type: field

Track differences between field values belonging to different groups

Example: Here the change in state is tracked for each value of name

Input:

{"time":"2020-02-03T15:34:45.149Z","name":"john","state":0}
{"time":"2020-02-03T15:34:46.149Z","name":"jane","state":0}
{"time":"2020-02-03T15:34:47.149Z","name":"john","state":1}
{"time":"2020-02-03T15:34:48.149Z","name":"jane","state":1}
{"time":"2020-02-03T15:34:49.149Z","name":"john","state":1}
{"time":"2020-02-03T15:34:50.149Z","name":"jane","state":1}

Pipe Language Snippet:

stream:
input-time: time
watch: state
group-by: name
operation: delta

Output:

{"time":"2020-02-03T15:34:45.149Z","name":"john","state":0,"delta":0,"elapsed":0}
{"time":"2020-02-03T15:34:46.149Z","name":"jane","state":0,"delta":0,"elapsed":0}
{"time":"2020-02-03T15:34:47.149Z","name":"john","state":1,"delta":1,"elapsed":2000}
{"time":"2020-02-03T15:34:48.149Z","name":"jane","state":1,"delta":1,"elapsed":2000}
{"time":"2020-02-03T15:34:49.149Z","name":"john","state":1,"delta":0,"elapsed":2000}
{"time":"2020-02-03T15:34:50.149Z","name":"jane","state":1,"delta":0,"elapsed":2000}

marker

Type: (string,string) pair

The marker to use for the newly generated event

Example: With a marker a new event is created

Input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Pipe Language Snippet:

stream:
input-time: time
watch: data
operation: delta
marker: [DELTA]

Output:

{"_marker":"DELTA","delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:06.479Z","data":1}
{"_marker":"DELTA","delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"_marker":"DELTA","delta":2,"elapsed":700}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Example: Without marker the existing event is enriched

Input:

{"time":"2018-07-09T15:09:06.479Z","data":1}
{"time":"2018-07-09T15:09:07.479Z","data":2}
{"time":"2018-07-09T15:09:08.179Z","data":4}

Pipe Language Snippet:

stream:
input-time: time
watch: data
operation: delta

Output:

{"time":"2018-07-09T15:09:06.479Z","data":1,"delta":1,"elapsed":0}
{"time":"2018-07-09T15:09:07.479Z","data":2,"delta":1,"elapsed":1000}
{"time":"2018-07-09T15:09:08.179Z","data":4,"delta":2,"elapsed":700}