Collector settings

The collector settings are used by the collector to successfully parse CSV files.

Basic settings

Delimiter

Description: The delimiter of the csv file.
Required: yes

CSVHeader

Description: Indicates if the csv files contain a header or not. If a header is present, the header record will be skipped when parsing the csv file.
Required: yes

TimestampColumn

Description: Index of column holding the timestamp (starts from 0)
Required: yes

TimestampLayout

Description: The layout of the timestamp of the measurements, default RFC3339 will be used. More info: https://pkg.go.dev/time#Layout
Required: no

StatusColumn

Description: Index of column holding the status (starts from 0)
Required: no

TrimString

Description: The string to trim (leading and trailing) from the csv cells used (values, mappings, status, …)
Required: no

BasePath

Description: Base directory, upon which the default incoming, processed and error directory are constructed.
Required: yes
Example: c:\plantdata\

IncomingDirectoryPath

Description: Override for the directory that needs to be monitored by the CSV collector.
Required: no
Example: c:\plantdata\incoming

ProcessedDirectoryPath

Description: Override for the directory in which processed files will be put. Processed files are files which had at least one measurement sent to Influx.
Required: no
Example: c:\plantdata\processed

Processed files are files which had at least one measurement sent to Influx.

Note: when a CSV measurement name is not found in the measurements configuration, the file will still be marked as processed successfully.

ErrorDirectoryPath

Description: Override for the directory in which files that contained an error will be put. Invalid CSV files will be stored here.
Required: no
Example: c:\plantdata\failed

FileMask

Description: File mask of the files to use.
Required: yes

The file mask for which the collector will monitor files.

Advanced settings

ProcessInterval

Description: Delay in milliseconds for processing new files after the were detected
Required: no
Default value: 10000

ProcessDelay

Description: Delay in milliseconds for processing new files after the were detected
Required: no
Default value: 3000

StartupDelay

Description: Initial delay for checking the incoming directory in milliseconds
Required: no
Default value: 9000

ProcessedMaxAge

Description: The maximum time in minutes to keep processed files. 0 will keep the files indefinitely
Required: no
Default value: 43200

FailedMaxAge

Description: The maximum time in minutes to keep failed files. 0 will keep the files indefinitely
Required: no
Default value: 129600

SourceTimeZone

Description: The timezone for the data points coming from the source device can be overwritten using this setting. Available timezones can be found here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
Required: no