Skip to main content
Version: 3.3.0

amqp

Read from AMQP queues

Field NameDescriptionTypeDefault
batchFor when a number of output events need to be marked as belonging to a distinct groupBatch-
uriAMQP address to listen tourlamqp://localhost//
queueAn AMQP QueueQueue-
routing-keyUsed in conjunction with topic exchange, to route datastring-
exchangeAn AMQP ExchangeExchange-
persistentLet messages be cached and survive server restartboolfalse
durableLet Queue or Exchange survive server restartboolfalse
rawIf true, treat incoming events as JSON, else wrap in _raw field ({"_raw":...})boolfalse
ignore-line-breaksDo not treat separate lines as distinct eventsboolfalse

batch

For when a number of output events need to be marked as belonging to a distinct group

Type: Batch

Field NameDescriptionTypeDefault
uuid-fieldField where generated uuid, the unique marker for the group, will be storedfield-
invocation-time-fieldField where invocation time will be storedfield-
completion-time-fieldField where completion (end of execution) time will be storedfield-
begin-marker-fieldField used to mark first event in the groupfield-
end-marker-fieldField used to mark last event in the groupfield-
line-count-fieldField used to store the line count of the batchfield-
line-num-fieldField used to store the line number of the batchfield-

uuid-field

Field where generated uuid, the unique marker for the group, will be stored

Type: field

Example

action:

exec:
command: |
for n in $(seq 3)
do
echo $n
done
no-strip-linefeeds: true
batch:
uuid-field: marker
interval: 1m

output:

{"_raw":"foo","line-count":3,"line-num":1,"marker":"f3308aa9-6f56-4cc1-8782-c4231ff254b8"}
{"_raw":"2","line-count":3,"line-num":2,"marker":"f3308aa9-6f56-4cc1-8782-c4231ff254b8"}
{"_raw":"3","line-count":3,"line-num":3,"marker":"f3308aa9-6f56-4cc1-8782-c4231ff254b8"}

Example: For cases where event count is known, a simple counter is used, instead of uuid (useful for testing)

action:

exec:
command: echo foo
no-strip-linefeeds: true
count: 3
batch:
uuid-field: marker
interval: 1m

output:

{"_raw":"1","line-count":3,"line-num":1,"marker":"1"}
{"_raw":"2","line-count":3,"line-num":2,"marker":"1"}
{"_raw":"3","line-count":3,"line-num":3,"marker":"1"}
{"_raw":"1","line-count":3,"line-num":1,"marker":"2"}
{"_raw":"2","line-count":3,"line-num":2,"marker":"2"}
{"_raw":"3","line-count":3,"line-num":3,"marker":"2"}
{"_raw":"1","line-count":3,"line-num":1,"marker":"3"}
{"_raw":"2","line-count":3,"line-num":2,"marker":"3"}
{"_raw":"3","line-count":3,"line-num":3,"marker":"3"}

invocation-time-field

Field where invocation time will be stored

Type: field

Example

action:

exec:
command: |
for n in $(seq 3)
do
echo $n
done
no-strip-linefeeds: true
batch:
invocation-time-field: begin
interval: 1m

output:

{"_raw":"1","line-count":3,"line-num":1,"begin":"2020-01-17T09:55:09.135Z"}
{"_raw":"2","line-count":3,"line-num":2,"begin":"2020-01-17T09:55:09.135Z"}
{"_raw":"3","line-count":3,"line-num":3,"begin":"2020-01-17T09:55:09.135Z"}

completion-time-field

Field where completion (end of execution) time will be stored

Type: field

Example

action:

exec:
command: |
for n in $(seq 3)
do
echo $n
done
no-strip-linefeeds: true
batch:
invocation-time-field: begin
completion-time-field: end
interval: 1m

output:

{"_raw":"1","begin":"2020-01-17T10:02:14.302Z","end":"2020-01-17T10:02:14.301Z","line-count":3,"line-num":1}
{"_raw":"2","begin":"2020-01-17T10:02:14.302Z","end":"2020-01-17T10:02:14.301Z","line-count":3,"line-num":2}
{"_raw":"3","begin":"2020-01-17T10:02:14.302Z","end":"2020-01-17T10:02:14.301Z","line-count":3,"line-num":3}

begin-marker-field

Field used to mark first event in the group

Type: field

Example

action:

exec:
command: |
for n in $(seq 3)
do
echo $n
done
no-strip-linefeeds: true
batch:
begin-marker-field: begin
interval: 1m

output:

{"_raw":"1","begin":true,"line-count":3,"line-num":1}
{"_raw":"2","line-count":3,"line-num":2}
{"_raw":"3","line-count":3,"line-num":3}

end-marker-field

Field used to mark last event in the group

Type: field

Example

action:

exec:
command: |
for n in $(seq 3)
do
echo $n
done
no-strip-linefeeds: true
batch:
begin-marker-field: begin
end-marker-field: end
interval: 1m

output:

{"_raw":"1","begin":true,"line-count":3,"line-num":1}
{"_raw":"2","line-count":3,"line-num":2}
{"_raw":"3","end":true,"line-count":3,"line-num":3}

line-count-field

Field used to store the line count of the batch

Type: field

line-num-field

Field used to store the line number of the batch

Type: field

uri

AMQP address to listen to

Type: url

Example

action:

amqp:
queue:
name: queue-name
uri: amqp://localhost//

queue

An AMQP Queue

Type: Queue

Field NameDescriptionTypeDefault
nameQueue namestring-
passiveAssume Queue has already been declared, else errorboolfalse

name

Queue name

Type: string

Example

action:

amqp:
queue:
name: queue-name

passive

Assume Queue has already been declared, else error

Type: bool

Example

action:

amqp:
queue:
name: some-queue
passive: true

routing-key

Used in conjunction with topic exchange, to route data

Type: string

exchange

An AMQP Exchange

Type: Exchange

Field NameDescriptionTypeDefault
nameExchange namestring-
typeExchange typestring-
passiveAssume Exchange has already been declared, else errorboolfalse

name

Exchange name

Type: string

Example

action:

amqp:
exchange:
name: some-exchange
type: fanout

type

Exchange type

There are 2 supported types:

  • fanout: For when same messages are to be consumed by multiple comsumers
  • topic: Topic-based pub-sub
  • direct: For when routing keys are to be used

Type: string

Example

action:

amqp:
exchange:
name: some-exchange
type: topic
routing-key: some.key

passive

Assume Exchange has already been declared, else error

Type: bool

Example

action:

amqp:
exchange:
name: some-exchange
passive: true

persistent

Let messages be cached and survive server restart

Type: bool

Example

action:

amqp:
queue:
name: some-queue
persistent: true

durable

Let Queue or Exchange survive server restart

Type: bool

Example

action:

amqp:
queue:
name: some-queue
durable: true

raw

If true, treat incoming events as JSON, else wrap in _raw field ({"_raw":...})

Type: bool

Example

action:

amqp:
queue:
name: queue-name
raw: true

ignore-line-breaks

Do not treat separate lines as distinct events

Type: bool

Example

action:

amqp:
queue:
name: some-queue
ignore-line-breaks: true