Internal Messages

The Server, Agents and Pipes generates telemetry data in the form of internal messages. By using the internal-messages input, users may access and leverage these internal messages in Pipes. This special input subscribes to a real-time feed of all or some internal messages generated by and from system components.

note

The internal message subsystem is available in versions 3.5 and later.

Subscribing to internal messages

This example subscribes to all internal messages available on the system, by including all available message sources in the system.

name: example-feed-all

input:
  internal-messages:
    filter-source:
    - server
    - agent
    - pipe

output:
  print: STDOUT

This Pipe will print internal messages in a standardized JSON format, which looks like this:

{"id":"cd70d6f4-1ac8-48de-a52e-be8e2d03ecba","kind":"system","message_type":"heart-beat","nanoseconds_since_epoch":1688656612679319371,"pipe_name":"example-feed-all","source":"pipe","version":"0.1"}
{"id":"b20a4335-847a-4e6d-84af-f8af108f43e7","kind":"system","message_type":"heart-beat","nanoseconds_since_epoch":1688656613680944280,"pipe_name":"example-feed-all","source":"pipe","version":"0.1"}
{"agent_id":"built-in-agent","id":"29aadf6b-6225-4f84-bbd8-e9f1b74db48e","kind":"system","message_type":"agent-heart-beat","nanoseconds_since_epoch":1688656614137398340,"source":"agent","version":"0.1"}

Messages contain the fields:

field	description
id	unique, internal ID of the message
kind	Either "system" or "user"
message_type	Identifies the type of message
nanoseconds_since_epoch	A Unix timestamp with nanosecond resolution
pipe_name*	The applicable pipe, if any
agent_id*	The applicable agent, if any
source	From which component the message was generated; server, agent or pipe
version	Internal designation for the structure of the message payload
message_data	message data payload, which may vary between message_type values

* These fields may not be present on all messages.

The ability to subscribe to an event feed of the system's behaviour is useful for a variety of observability-related purposes. It is furthermore easy to send these feeds to external destinations, by means of any supported output available to Pipes.