Skip to main content
Version: 3.4.0

files

Read from multiple files, in order of creation

Example

action:

files:
path: /var/log/dpkg.*

Field NameDescriptionTypeDefault
pathA path with wildcards identifying files to be readpath-
data-dirThe directory where checkpoints are writtenpaththe data directory for this user (e.g ~/.local/share/hotrod/PIPENAME on Linux)
encodingThe text encoding for the monitored filesencodingUTF-8
includeAny other paths to be includedarray of paths-
excludePaths to be excludedarray of paths-
ignore-older-thanIgnore files older than this (epoch time in seconds)integer-
glob-minimum-cooldownPause after scanning for new files (in milliseconds)integer300ms
stop-reading-afterDo not wait for new filesboolfalse
ignore-line-breaksConsume each file as one eventboolfalse
remove-afterRemove files that are done reading after this period (seconds)integer-
fingerprintingFile fingerprinting strategyFingerprinterDevInode
oldest-firstDefault is youngest firstboolfalse
file-path-fieldFill this field with the field we are currently reading fromfield-
file-basenameThe file-path-field path will be the basename (e.g. '/path/frodo.txt' becomes just 'frodo')boolfalse
start-at-beginningIgnore stored checkpoints and re-read all specified filesboolfalse
rawDisable 'quoting' of lines with _rawboolfalse
max-read-sizeWill not attempt to read files larger than thisinteger100MB
max-line-sizeWill not attempt to read lines longer than thisinteger1MB

path

A path with wildcards identifying files to be read

Type: path

data-dir

The directory where checkpoints are written

Type: path

encoding

The text encoding for the monitored files

Type: encoding

include

Any other paths to be included

Type: array of paths

exclude

Paths to be excluded

Type: array of paths

ignore-older-than

Ignore files older than this (epoch time in seconds)

Type: integer

glob-minimum-cooldown

Pause after scanning for new files (in milliseconds)

Type: integer

stop-reading-after

Do not wait for new files

Available from Hotrod: 2.8

Type: bool

ignore-line-breaks

Consume each file as one event

Available from Hotrod: 2.8

Type: bool

remove-after

Remove files that are done reading after this period (seconds)

Type: integer

fingerprinting

File fingerprinting strategy

Type: Fingerprinter

Field NameDescriptionTypeDefault
checksumFingerprint strategy is set to a checksum of the start of the file determined by the bytes that are skipped (ignored-header-bytes) and then the fingerprint size in bytesFingerprinterChecksum-
first-line-checksumFingerprint strategy that uses the first line of the file to detect if a file has changedFingerprinterFirstLineChecksum-
dev-inodeFingerprint strategy is set to the inode of the file causing the file monitor to detect new files when a file is recreatedFingerprinterDevInode-

checksum

Fingerprint strategy is set to a checksum of the start of the file determined by the bytes that are skipped (ignored-header-bytes) and then the fingerprint size in bytes

Type: FingerprinterChecksum

Field NameDescriptionTypeDefault
fingerprint-bytesThe number of bytes to use for the fingerprint, starts after ignore-header-bytes if specifiedinteger-
ignored-header-bytesThe number of bytes from the start of the file to ignore before starting the fingerprintinteger-

fingerprint-bytes

The number of bytes to use for the fingerprint, starts after ignore-header-bytes if specified

Type: integer

ignored-header-bytes

The number of bytes from the start of the file to ignore before starting the fingerprint

Type: integer

first-line-checksum

Fingerprint strategy that uses the first line of the file to detect if a file has changed

Type: FingerprinterFirstLineChecksum

Field NameDescriptionTypeDefault
max-line-lengthThe maximum number of bytes to read from the first line of the file to calculate the fingerprintinteger-

max-line-length

The maximum number of bytes to read from the first line of the file to calculate the fingerprint

Type: integer

dev-inode

Fingerprint strategy is set to the inode of the file causing the file monitor to detect new files when a file is recreated

Type: FingerprinterDevInode

oldest-first

Default is youngest first

Type: bool

file-path-field

Fill this field with the field we are currently reading from

Type: field

file-basename

The file-path-field path will be the basename (e.g. '/path/frodo.txt' becomes just 'frodo')

Available from Hotrod: 2.8

Type: bool

start-at-beginning

Ignore stored checkpoints and re-read all specified files

Type: bool

raw

Disable 'quoting' of lines with _raw

Type: bool

max-read-size

Will not attempt to read files larger than this

Type: integer

max-line-size

Will not attempt to read lines longer than this

Type: integer