Skip to main content
Version: 3.3.1

sql

Query a SQL database

Currently only Postgres and Sqlite (embedded) are supported

Field NameDescriptionTypeDefault
intervalHow often to run the commandduration-
cronHow often to run the command. Note that Hotrod uses a different format than Cron it includes a column for seconds. See full discussioncron-
immediateRun as soon as invoked, instead of waiting for the specified cron intervalboolfalse
random-offsetSets a random offset to the schedule, then sticks to itduration0s
windowFor resources that need a time window to be specifiedWindow-
blockBlock further input schedules from triggering if the pipe output is retryingboolfalse
queryAn SQL SELECT statementstring-
sourceSQL connection optionsSqlSources-

interval

How often to run the command

By default, interval: 0s which means: once. Note that scheduled inputs set document markers. See full discussion

Type: duration

Example

action:

exec:
command: echo 'once a day'
interval: 1d

cron

How often to run the command. Note that Hotrod uses a different format than Cron it includes a column for seconds. See full discussion

Type: cron

Example: Once a day

action:

exec:
command: echo 'once a day'
cron: '0 0 0 * * *'

Example: Once a day, using a convenient shortcut

action:

exec:
command: echo 'once a day'
cron: '@daily'

immediate

Run as soon as invoked, instead of waiting for the specified cron interval

Type: bool

Example: Run immediately on invocation, and thereafter at 10h every morning

action:

exec:
command: echo 'hello'
immediate: true
cron: '0 0 10 * * *'

random-offset

Sets a random offset to the schedule, then sticks to it

This can help avoid the thundering herd problem, where you do not, for example, want to overload some service at 00:00:00

Type: duration

Example: Would fire up to a minute after every hour

action:

exec:
command: echo 'hello'
random-offset: 1m
cron: '0 0 * * * *'

window

For resources that need a time window to be specified

Type: Window

Field NameDescriptionTypeDefault
sizeWindow sizeduration-
offsetWindow offsetduration0s
start-timeAllows the windowing to start at a specified timetime-
highwatermark-fileSpecify file where timestamp would be stored in order to resume, for when Pipe has been restartedpath-

size

Window size

Type: duration

Example

action:

exec:
command: echo 'one two'
window:
size: 1m

offset

Window offset

Type: duration

Example

action:

exec:
command: echo 'one two'
window:
size: 1m
offset: 10s

start-time

Allows the windowing to start at a specified time

It should in the following format: 2019-07-10 18:45:00.000 +0200

Type: time

Example

action:

exec:
command: echo 'one two'
window:
size: 1m
start-time: 10s

highwatermark-file

Specify file where timestamp would be stored in order to resume, for when Pipe has been restarted

Type: path

Example

action:

exec:
command: echo 'one two'
window:
size: 1m
highwatermark-file:: /tmp/mark.txt

block

Block further input schedules from triggering if the pipe output is retrying

Type: bool

query

An SQL SELECT statement

Type: string

source

SQL connection options

Type: SqlSources

Field NameDescriptionTypeDefault
postgresSpecify kind and connection for an SQL databaseSqlSource-
sqliteSpecify kind and connection for an SQL databaseSqlSource-
my-sqlSpecify kind and connection for an SQL databaseSqlSource-
odbcSQL optionsOdbcSqlSource-

postgres

Specify kind and connection for an SQL database

Type: SqlSource

Field NameDescriptionTypeDefault
connectionFor 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"connection-string-

connection

For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"

Type: connection-string

sqlite

Specify kind and connection for an SQL database

Type: SqlSource

Field NameDescriptionTypeDefault
connectionFor 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"connection-string-

connection

For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"

Type: connection-string

my-sql

Specify kind and connection for an SQL database

Type: SqlSource

Field NameDescriptionTypeDefault
connectionFor 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"connection-string-

connection

For 'sqlite', a path to a file, for 'postgres' strings like "host=localhost user=postgres"

Type: connection-string

odbc

SQL options

Type: OdbcSqlSource

Field NameDescriptionTypeDefault
quickSimple connection to ODBC (assumes system or user configuration)DsnFields-
connection-stringCreate a compatible connection string to connect to ODBC this may vary between ODBC driversDriverFields-
max-str-lenThe maximum string buffer to allocate when parsing returned strings this is specified in bytes and should be large enough to accommodate the largest possible string returnedinteger-
max-bin-lenThe maximum string buffer to allocate when parsing returned binary blobs this is specified in bytes and should be large enough to accommodate the largest possible binary blobs returnedinteger-

quick

Simple connection to ODBC (assumes system or user configuration)

Type: DsnFields

Field NameDescriptionTypeDefault
dsnThe configured DSN namestring-
usernameThe username for the ODBC connectionstring-
passwordThe password for the ODBC connectionstring-

dsn

The configured DSN name

Type: string

username

The username for the ODBC connection

Type: string

password

The password for the ODBC connection

Type: string

connection-string

Create a compatible connection string to connect to ODBC this may vary between ODBC drivers

Type: DriverFields

Field NameDescriptionTypeDefault
driverThe ODBC driver to usestring-
attributesAttributes in a "String: Value" formatMap-

driver

The ODBC driver to use

Type: string

attributes

Attributes in a "String: Value" format

Type: Map

max-str-len

The maximum string buffer to allocate when parsing returned strings this is specified in bytes and should be large enough to accommodate the largest possible string returned

Type: integer

max-bin-len

The maximum string buffer to allocate when parsing returned binary blobs this is specified in bytes and should be large enough to accommodate the largest possible binary blobs returned

Type: integer