Custom Time Series Data Store v2

The Custom Time Series Data Store v2 is an enhanced version of Custom Time Series Data Store that includes the modified history functionality. This means that the data store can archive multiple values with the same timestamp, essentially allowing for multiple versions of data points if they are modified (previously, any modification would overwrite the stored value meaning only the most recent modification would be available).

Modified history data can be viewed in the History Grid and History Trend displays (when a Custom Time Series Data Store v2 is selected) and can also be retrieved using the Lua API getrawhistory function (again, when a Custom Time Series Data Store v2 is specified in the call).

The object can be created in the I/O Model beneath the Core or a Data Store Group object.

Once created, the Custom Time Series Data Store v2 exists as a separate MongoDB database instance in the connected MongoDB component. The data store can then be selected to archive Time Series data when configuring the Archive Selector property of an I/O item or other object that produces time series data.

To add Custom Time Series Data Stores v2 to Data Store Sets, the user must have LIST permissions on the Custom Time Series Data Store v2 object. For more information on permissions, see the Permissions section.

Data stores can also be configured to use Server Timestamps for data retention.

Quick Configuration

Right click on the Core or a Data Store Group object and select Admin New Data Stores Custom Time Series Data Store v2 from the context menu to open the Create Object wizard.
Give the object a unique name and enter a description if necessary.

The properties that configure the time series buffer (buffer size, latency etc.) can be changed here, however, the default settings are usually sufficient for most cases.
Select the Disabled SaF Mode. The default is "Discard" which means that any data in the Saf buffer will be purged if the data store is disabled. Please see the property page for more details.
Click on the Time Series Data Store options from the wizard sidebar to view the data store settings.
Generated Database Name and Collection properties are filled automatically (Database Name property is deprecated and is only used to maintain backwards compatibility with Custom Data Stores created in versions before v1.96).
The Purge Options control if data will be purged from the data store and if so, how long the data is retained before it is purged. The Purge Orphaned Data controls what happens to the data associated with an object if the object is deleted. Please see the Purge Options page for more details.
The Server Timestamp settings can be configured here. This enable the user to record the time that the data is persisted to the data store as a timestamp (optionally this can be used instead of the source timestamp). Please read the Server Timestamp page for more details before configuring any of these properties.
If necessary, the MongoDB connection settings can also be changed (for example, if you wish to connect to a remote MongoDB instance or configure replication sets).
Click Create to create the object in the I/O model tree.
To add the new Time Series Data Store v2 as an archive option for the Archive Selector property for I/O items under the Core, it is also necessary to configure the Data Store Sets table property in the Core object.
Select the Core in the I/O Model tree and open the Data Store Configuration property compound in the Object Properties panel.
Open the Data Store Sets table property (the property grid should be empty if this is the first data source being configured).
Enter a unique name in the 'Name' column. This is the name that will be displayed in the drop down menu of the Archive Selector property for I/O items.
From the drop down menu in the 'Data Stores' column, select the name of the Time Series Data Store 2 object that you just created. You can also select the System Production/Test archive, to archive the data in the centralized System archives.
Add a description to the 'Description' column if required.
Click OK to close the property grid.
Click Apply in the Core Object Properties panel to apply the changes.

To check that the new Time Series Data Store v2 is now available, select an I/O item (or other time series data producing object) in the namespace below the respective Core and navigate to the Archive Selector property in the Object Properties panel.

If you are configuring I/O items that are below a Local Core to historize in a Custom Time Series Data Store 2 that is below the Master Core, the Data Store Sets property on the Local Core must also be configured for the Custom Data Store to be available as an archive option. To do this, you must be logged into the Master Core with DataStudio when configuring the Local Core properties because the Data Store will not be visible if you are logged into the Local Core.

Select the data store from the drop-down menu and click Apply to archive the data for that object in the new Time Series Data Store v2.

Object Properties

Common

Object Name: The user-modifiable object name. This name overrides the name which has been supplied by the external system. It must be unique within the collection of objects of the parent object.

Object Description: This is the user-modifiable object description. This text overrides the description which has been supplied by the external system.

Disabled SaF Mode: SaF behavior if the object is disabled.

Display Alias: Alternate label for objects to be used for easier identification in the displays.

Attachments: File attachments stored in MongoDB file store.

Time Series Buffer

Custom time series buffer configuration.

Latency: The maximum duration to buffer time-series data in memory.

Minimum Buffer Size: The minimum size of the time-series buffer, in data values, eligible for immediate writing.

Maximum Buffer Size: The maximum size of the time-series buffer, in data values.

Parallelism: The number of threads used when writing time-series data.

Maximum Future Offset: Data with timestamps beyond the specified offset from the current time will be rejected. Data with significant future offsets degrade the purge performance of the test store, potentially causing it to keep data longer than the configured retention time.

Allocation Step: A value used internally by the time-series buffer. This should only be changed when advised by Product Support.

Time Series Data Store

Custom time series data store configuration.

Generated Database Name: If not empty, this value will be used as the database name.

Database Name: This name will only be used to identify the database if the 'Generated Database Name' is empty.

Collection: The collection storing time-series data.

Purge Mode

Do not purge: Do not purge data from the store.
Purge when idle: Purge data of deleted objects from the store, only when it is idle.
Purge continuously: Purge data of deleted objects from the store, even if it is busy.
Purge: Purge data from the store.

Retention Period: The retention period for the particular custom data store, a value of 0 will set it to unlimited retention.

Purge Size: The maximum number of BSON documents purged in a single batch operation.

Purge Orphaned Data: Delete data owned by objects which have been deleted.

Purge When Idle: Inhibits purging when time-series data are being persisted.

Purge Moderation: A delay between successive purge batches.

Server Timestamp

Server Timestamp Settings.

Mode

Record: The Server Timestamp is recorded whenever data are written (updated), only for the data being written.
Use for Retention: As in the Record mode, plus the Server Timestamp is used instead of the Source Timestamp for data retention. If, during the purge cycle, data are encountered without a Server Timestamp, the data are updated with the current server time as the Server Timestamp.

MongoDB Connection

This property compound combines the properties of the MongoDB Connection e.g. the connection strings to the mongoDB.

Connection String: The connection string for the external system; depending on the type of connection.

Replication Set Name: Configuration name for MongoDB Replica Set.

Replication Set 1: The connection string for the core service to replication set 1.

Replication Set 2: The connection string for the core service to replication set 2.

Authentication Mode

No Authentication: No Authentication is performed when the System connects to a certain instance of MongoDB.
Challenge-Response: MONGODB-CR is the default authentication mechanism supported by MongoDB. Username and Password needs to be provided.
SCRAM-SHA-1 Authentication: Salted Challenge Response Authentication Mechanism (SCRAM) means that the password is never actually sent over the wire, but rather a computed proof that the client password is the same as the password the server knows.
X.509 Certificate: MONGODB-X509 mechanism authenticates a username derived from the distinguished subject name of a X.509 certificate.
LDAP (Lightweight Directory Access Protocol): Lightweight Directory Access Protocol (LDAP) service is used for MongoDB authentication.
SCRAM-SHA-256 Authentication: SCRAM-SHA-256 authentication means that the password is never actually sent over the wire, but rather a computed proof that the client password is the same as the password the server knows (non-ASCII characters in passwords are not supported).
Auto: Detect the default authentication mechanism for users with stored SCRAM-SHA-1 and SCRAM-SHA-256 credentials (requires MongoDB 4.0 or newer).

User Name: User Name.

Password: Password.

Compression: Compress MongoDB data on the wire.

Secure connections: Enable TLS for MongoDB connections.

Allow Invalid Hostnames: Allow invalid hostnames in the server certificate for TLS connections.

Allow Invalid Certificates: Allow invalid server certificates for TLS connections.

MongoDB URI: If not empty, the MongoDB URI takes precedence, overriding all other MongoDB connection settings. The string needs to be URL-encoded.

Effective MongoDB URI: The actual MongoDB URI used for current connection.

Retry Period: The time after which an unsuccessful operation shall be retried.

Error Latency: The time during which successive errors are not reported.

Custom Options

Compound to hold various structures to customize the object and to be read and written to by Lua-Script code or external interfaces.

Custom String: A generic string buffer to be used programmatically for custom purposes.

Custom Properties

This is an extensible set of named strings which can be used programmatically for custom purposes.

Property Name: A custom property name which can be used programmatically.

Property Value: The value of the custom property which can be read and written programmatically.

Custom Tables

This is an extensible set of named tables which can be used programmatically for custom purposes.

Table Name: A custom table name which can be used programmatically.

Table Data: Handles an entire table organized in columns and rows. The data can easily (cut, copy and paste) be exchanged with table-oriented data of other software products, e.g. MS Excel.