Collector settings

The collector settings are used by the collector to successfully parse CSV files.

Collector settings

Basic settings

Delimiter

Description: The delimiter of the csv file.
Required: yes

CSVHeaderRows

Description: Indicates the number of header rows in the csv files. If a header is present, the header records will be skipped when parsing the csv file.
Required: yes

TimestampColumns

Description: Indices of columns holding the timestamp (starts from 0), separated by semicolon
Required: yes

TimestampLayout

Description: The layout of the timestamp of the measurements, default RFC3339 will be used. More info: https://pkg.go.dev/time#Layout
Required: no

StatusColumn

Description: Index of column holding the status (starts from 0)
Required: no

TrimString

Description: The string to trim (leading and trailing) from the csv cells used (values, mappings, status, …)
Required: no

The default value is " \"", trimming all spaces and double quotes from the start and end of the csv cell contents.

Locale

Description: The locale with which numbers should be parsed. Default is empty, meaning no locale parsing will be done.
Required: no

This table lists some examples of number formats and the locale with which they can be correctly parsed to a number the collector can process.
The output in these examples is always the same: 1234.56

input locale
1 234.56 en-GB
1,234.56 en-US
1.234,56 de-DE
1 234,56 fr-FR
1234,56 nl-BE

BasePath

Description: Base directory, upon which the default incoming, processed and error directory are constructed.
Required: yes
Example: c:\plantdata\

IncomingDirectoryPath

Description: Override for the directory that needs to be monitored by the CSV collector.
Required: no
Example: c:\plantdata\incoming

ProcessedDirectoryPath

Description: Override for the directory in which processed files will be put. Processed files are files which had at least one measurement sent to Influx.
Required: no
Example: c:\plantdata\processed

Processed files are files which had at least one measurement sent to Influx.

Note: when a CSV measurement name is not found in the measurements configuration, the file will still be marked as processed successfully.

ErrorDirectoryPath

Description: Override for the directory in which files that contained an error will be put. Invalid CSV files will be stored here.
Required: no
Example: c:\plantdata\failed

FileMask

Description: File mask of the files to use.
Required: yes

The file mask for which the collector will monitor files.

Advanced settings

Advanced settings

ProcessInterval

Description: Delay in milliseconds for processing new files after the were detected
Required: no
Default value: 10000

ProcessDelay

Description: Delay in milliseconds for processing new files after the were detected
Required: no
Default value: 3000

StartupDelay

Description: Initial delay for checking the incoming directory in milliseconds
Required: no
Default value: 9000

ProcessedMaxAge

Description: The maximum time in minutes to keep processed files. 0 will keep the files indefinitely
Required: no
Default value: 43200

FailedMaxAge

Description: The maximum time in minutes to keep failed files. 0 will keep the files indefinitely
Required: no
Default value: 129600

Advanced settings

SourceTimeZone

Description: The timezone for the data points coming from the source device can be overwritten using this setting. Available timezones can be found here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
Required: no