sql

Query a SQL database

Currently only Postgres and Sqlite (embedded) are supported

Field Name	Description	Type	Default
interval	How often to run the command	duration	-
cron	How often to run the command. Note that Hotrod uses a different format than Cron it includes a column for seconds. See full discussion	cron	-
immediate	Run as soon as invoked, instead of waiting for the specified cron interval	bool	false
random-offset	Sets a random offset to the schedule, then sticks to it	duration	0s
window	For resources that need a time window to be specified	Window	-
block	Block further input schedules from triggering if the pipe output is retrying	bool	false
query	An SQL SELECT statement	string	-
source	SQL connection options	SqlSources	-

interval

How often to run the command

By default, interval: 0s which means: once. Note that scheduled inputs set document markers. See full discussion

Type: duration

Example

action:

exec:
  command: echo 'once a day'
  interval: 1d

cron

How often to run the command. Note that Hotrod uses a different format than Cron it includes a column for seconds. See full discussion

Type: cron

Example: Once a day

action:

exec:
  command: echo 'once a day'
  cron: '0 0 0 * * *'

Example: Once a day, using a convenient shortcut

action:

exec:
  command: echo 'once a day'
  cron: '@daily'

immediate

Run as soon as invoked, instead of waiting for the specified cron interval

Type: bool

Example: Run immediately on invocation, and thereafter at 10h every morning

action:

exec:
  command: echo 'hello'
  immediate: true
  cron: '0 0 10 * * *'

random-offset

Sets a random offset to the schedule, then sticks to it

This can help avoid the thundering herd problem, where you do not, for example, want to overload some service at 00:00:00

Type: duration

Example: Would fire up to a minute after every hour

action:

exec:
  command: echo 'hello'
  random-offset: 1m
  cron: '0 0 * * * *'

window

For resources that need a time window to be specified

Type: Window

Field Name	Description	Type	Default
size	Window size	duration	-
offset	Window offset	duration	0s
start-time	Allows the windowing to start at a specified time	time	-
highwatermark-file	Specify file where timestamp would be stored in order to resume, for when Pipe has been restarted	path	-

size

Window size

Type: duration

Example

action:

exec:
  command: echo 'one two'
  window:
    size: 1m

offset

Window offset

Type: duration

Example

action:

exec:
  command: echo 'one two'
  window:
    size: 1m
    offset: 10s

start-time

Allows the windowing to start at a specified time

It should in the following format: 2019-07-10 18:45:00.000 +0200

Type: time

Example

action:

exec:
  command: echo 'one two'
  window:
    size: 1m
    start-time: 10s

highwatermark-file

Specify file where timestamp would be stored in order to resume, for when Pipe has been restarted

Type: path

Example

action:

exec:
  command: echo 'one two'
  window:
    size: 1m
    highwatermark-file:: /tmp/mark.txt

block

Block further input schedules from triggering if the pipe output is retrying

Type: bool

query

An SQL SELECT statement

Type: string

source

SQL connection options

Type: SqlSources

Field Name	Description	Type	Default
postgres	Specify kind and connection for an SQL database	SqlSource	-
sqlite	Specify kind and connection for an SQL database	SqlSource	-
my-sql	Specify kind and connection for an SQL database	SqlSource	-
odbc	SQL options	OdbcSqlSource	-

postgres

Specify kind and connection for an SQL database

Type: SqlSource

Field Name	Description	Type	Default
connection	For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"	connection-string	-

connection

For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"

Type: connection-string

sqlite

Specify kind and connection for an SQL database

Type: SqlSource

Field Name	Description	Type	Default
connection	For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"	connection-string	-

connection

For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"

Type: connection-string

my-sql

Specify kind and connection for an SQL database

Type: SqlSource

Field Name	Description	Type	Default
connection	For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"	connection-string	-

connection

For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"

Type: connection-string

odbc

SQL options

Type: OdbcSqlSource

Field Name	Description	Type	Default
quick	Simple connection to ODBC (assumes system or user configuration)	DsnFields	-
connection-string	Create a compatible connection string to connect to ODBC this may vary between ODBC drivers	DriverFields	-
max-str-len	The maximum string buffer to allocate when parsing returned strings this is specified in bytes and should be large enough to accommodate the largest possible string returned	integer	-
max-bin-len	The maximum string buffer to allocate when parsing returned binary blobs this is specified in bytes and should be large enough to accommodate the largest possible binary blobs returned	integer	-

quick

Simple connection to ODBC (assumes system or user configuration)

Type: DsnFields

Field Name	Description	Type	Default
dsn	The configured DSN name	string	-
username	The username for the ODBC connection	string	-
password	The password for the ODBC connection	string	-

dsn

The configured DSN name

Type: string

username

The username for the ODBC connection

Type: string

password

The password for the ODBC connection

Type: string

connection-string

Create a compatible connection string to connect to ODBC this may vary between ODBC drivers

Type: DriverFields

Field Name	Description	Type	Default
driver	The ODBC driver to use	string	-
attributes	Attributes in a "String: Value" format	Map	-

driver

The ODBC driver to use

Type: string

attributes

Attributes in a "String: Value" format

Type: Map

max-str-len

The maximum string buffer to allocate when parsing returned strings this is specified in bytes and should be large enough to accommodate the largest possible string returned

Type: integer

max-bin-len

The maximum string buffer to allocate when parsing returned binary blobs this is specified in bytes and should be large enough to accommodate the largest possible binary blobs returned

Type: integer