Collector settings
The collector settings are used by the collector to successfully parse CSV files.
Basic settings
Delimiter
Description: The delimiter of the csv file.
Required: yes
CSVHeaderRows
Description: Indicates the number of header rows in the csv files. If a header is present, the header records will be skipped when parsing the csv file.
Required: yes
TimestampColumns
Description: Indices of columns holding the timestamp (starts from 0), separated by semicolon
Required: yes
TimestampLayout
Description: The layout of the timestamp of the measurements, default RFC3339 will be used. More info:
https://pkg.go.dev/time#Layout
Required: no
StatusColumn
Description: Index of column holding the status (starts from 0)
Required: no
TrimString
Description: The string to trim (leading and trailing) from the csv cells used (values, mappings, status, …)
Required: no
The default value is " \"", trimming all spaces and double quotes from the start and end of the csv cell contents.
Locale
Description: The locale with which numbers should be parsed. Default is empty, meaning no locale parsing will be done.
Required: no
This table lists some examples of number formats and the locale with which they can be correctly parsed to a number the collector can process.
The output in these examples is always the same: 1234.56
| input | locale |
|---|---|
| 1 234.56 | en-GB |
| 1,234.56 | en-US |
| 1.234,56 | de-DE |
| 1 234,56 | fr-FR |
| 1234,56 | nl-BE |
BasePath
Description: Base directory, upon which the default incoming, processed and error directory are constructed.
Required: yes
Example: c:\plantdata\
IncomingDirectoryPath
Description: Override for the directory that needs to be monitored by the CSV collector.
Required: no
Example: c:\plantdata\incoming
ProcessedDirectoryPath
Description: Override for the directory in which processed files will be put. Processed files are files which had at least one measurement sent to Influx.
Required: no
Example: c:\plantdata\processed
Processed files are files which had at least one measurement sent to Influx.
Note: when a CSV measurement name is not found in the measurements configuration, the file will still be marked as processed successfully.
ErrorDirectoryPath
Description: Override for the directory in which files that contained an error will be put. Invalid CSV files will be stored here.
Required: no
Example: c:\plantdata\failed
FileMask
Description: File mask of the files to use.
Required: yes
The file mask for which the collector will monitor files.
Advanced settings
ProcessInterval
Description: Delay in milliseconds for processing new files after the were detected
Required: no
Default value: 10000
ProcessDelay
Description: Delay in milliseconds for processing new files after the were detected
Required: no
Default value: 3000
StartupDelay
Description: Initial delay for checking the incoming directory in milliseconds
Required: no
Default value: 9000
ProcessedMaxAge
Description: The maximum time in minutes to keep processed files. 0 will keep the files indefinitely
Required: no
Default value: 43200
FailedMaxAge
Description: The maximum time in minutes to keep failed files. 0 will keep the files indefinitely
Required: no
Default value: 129600
SourceTimeZone
Description: The timezone for the data points coming from the source device can be overwritten using this setting. Available timezones can be found here:
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
Required: no