Custom Event Data Store

The Custom Event Data Store object allows the user to create and configure multiple event data stores in addition to the already existing System Event (A&E) DataStore. The object can be created in the I/O Model beneath the Core or a Data Store Group object.

A Custom Event Data Store will by default connect to the local MongoDB instance but can also be configured to connect to a different MongoDB historian. The Filter Options allow the user to choose the filter conditions for Alarms & Events to selectively store certain alarms and events in the Custom Event Data Store.

Quick Configuration

  1. Right click on the Core or a Data Store Group object and select Admin  New  Data Stores  Custom Event Data Store from the context menu to open the Create Object wizard.

  2. Give the object a unique name and enter a description if necessary.

  3. Select the Disabled SaF Mode. The default is "Discard" which means that any data in the Saf buffer will be purged if the data store is disabled. Please see the property page for more details.

  4. For now, leave the Processing Mode set to "Configuration".

  5. Click on the Custom Event Data Store options from the wizard sidebar to view the data store settings.

  6. The default database name can be changed here, along with data retention period and purge settings (what happens to the object data if the object is deleted). If the retention period is set to <null> then the event data will be retained permanently.

  7. If necessary, the MongoDB connection settings can also be changed (for example, if you wish to connect to a remote MongoDB instance or configure replication sets).

  8. Click Create to create the object in the I/O model tree.

Historization Options

There are two mechanisms to specify which events a Custom Event Data Store receives. Each Event Object (Event Stream, Script Event, Internal Event Stream and Redundant Event Stream) has an Archive Selector property to select a Data Store Set with one or more Event Stores, which determines to which stores an event is sent. Additionally, the Filter Options of the Custom Event Data Store can be used to filter out events it received. Events that are filtered out are not historized by the store.

Historize using Archive Selection on Event Objects

To add the new Custom Event Data Store as an archive option for the Archive Selector property for Event Objects, it is also necessary to configure the Data Store Sets table property in the relevant Core object(s).

  1. Select the Core in the I/O Model tree and open the Data Store Configuration property compound in the Object Properties panel.

  2. Open the Data Store Sets table property (the property grid will be empty if this is the first data store set being configured).

  3. Enter a unique name in the 'Name' column. This is the name that will be displayed in the drop down menu of the Archive Selector property for Event Objects.

  4. From the drop down menu in the 'Data Stores' column, select the name of the Custom Event Data Store object that you just created. You can also select other System/Custom Event stores, to archive the data.

  5. Add a description to the 'Description' column if required.

  6. Click OK to close the property grid.

  7. Click Apply in the Core Object Properties panel to apply the changes.

  8. The data store store set is now available in the drop-down menu of the Archive Selector property of an Event Object.

  9. Select the data store set from the drop-down menu and click Apply to archive the data for that object in the new Custom Event Data Store.

    If you are configuring Event Objects that are below a Local Core to historize in a Custom Event Data Store that is below the Master Core, the Data Store Sets property on the Local Core must also be configured for the Data Store to be available as an archive option. To do this, you must be logged into the Master Core with DataStudio when configuring the Local Core properties because the Data Store will not be visible if you are logged into the Local Core.

Historize using Filter Options

  1. Select the newly created Custom Event Data Store in the I/O Model and look at the Object Properties panel. Open the Filter Options property compound to see all the options for filtering A&E for storage in the Custom Store.

  2. The Filter Options can be applied via a number of categories. Firstly, Event Objects can be added to the Stream Objects filter by opening the table and adding the path to an Event Objects (you can also drag and drop objects from the I/O Model into the table).

    The Stream Objects table also accepts RegEx strings of object paths. For a disambiguated path containing the caret escape character (^) such as:

    • ^/System/Core/object 1 ^/object 2

    Some valid regex expressions for would be:

    • \\^/System/Core/object 1 \\^/object 2

    • (\\^?)/System/Core/(.*)

  3. The following filters can also be applied by adding relevant string(s) entry to the tables

    • Messages

    • Sources

    • Categories

    • Conditions

    • SubConditions

      These filters all accept RegEx strings as entries and will then match them against the appropriate field in the event to filter them into the Custom Event Data Store.

      The Filter Options table properties are all generated as tab separated strings of RegEx patterns that are then processed internally. For this reason, and to avoid processing errors, users should avoid adding strings to the filter table that contain the following expression: ‘\t’(tab)
  4. Other Filter Options include adding different Severity ranges and various check boxes that can be applied to Event State Changes. It is also possible to add a custom Lua script to the Filter Script field to fully customize your filtering options.

  5. When all filtering options have been applied, return to the Common property compound and change the Processing Mode to "Operation" and click Apply to begin archiving.

  6. Event History Data can be retrieved selectively from a Custom Event Data Store using the Lua API geteventhistory function. The Custom Event Data Store object can be specified in the options table argument for the function.

Systems Created Before Version 1.84

The Archive Selector was added to Event Objects with version 1.84. Old Event Objects have the All Event Stores value selected here automatically after an upgrade, which sends the event to all event stores on the master core. If an Event Object is on a local core then all event stores on this local core also receive these events. This is in accordance with the previous behavior where filtering was only possible with the Filter Options. As long as there is at least one Event Object that still has this option enabled, you should use the Filter Options on all Custom Event Stores to make sure they only receive the desired events.

Object Properties

Common

Object Name

The user-modifiable object name. This name overrides the name which has been supplied by the external system. It must be unique within the collection of objects of the parent object.

Object Description

This is the user-modifiable object description. This text overrides the description which has been supplied by the external system.

Disabled SaF Mode

SaF behavior if the object is disabled.

Display Alias

Alternate label for objects to be used for easier identification in the displays.

Processing Mode

Configure the processing mode of the object.

  • Configuration: In this mode no data is processed.

  • Operation: In this mode the object is operational.

Attachments

File attachments stored in MongoDB file store.

Event Data Store

Custom event data store configuration.

Database

The user-defined database storing event data.

Collection

The collection storing event data.

Retention Time

The maximum age of data in the archive.

Purge Size

The maximum number of BSON documents purged in a single batch operation.

Purge Mode

Controls what happens to an object’s data in the store after the object’s deletion.

  • Do not purge: Do not purge data of deleted objects from the store.

  • Purge when idle: Purge data of deleted objects from the store, only when it is idle.

  • Purge continuously: Purge data of deleted objects from the store, even if it is busy.

MongoDB Connection

This property compound combines the properties of the MongoDB Connection e.g. the connection strings to the mongoDB.

Connection String

The connection string for the external system; depending on the type of connection.

Replication Set Name

Configuration name for MongoDB Replica Set.

Replication Set 1

The connection string for the core service to replication set 1.

Replication Set 2

The connection string for the core service to replication set 2.

Authentication Mode

Authentication mode to be used connecting the MongoDB instance.

  • No Authentication: No Authentication is performed when the System connects to a certain instance of MongoDB.

  • Challenge-Response: MONGODB-CR is the default authentication mechanism supported by MongoDB. Username and Password needs to be provided.

  • SCRAM-SHA-1 Authentication: Salted Challenge Response Authentication Mechanism (SCRAM) means that the password is never actually sent over the wire, but rather a computed proof that the client password is the same as the password the server knows.

  • X.509 Certificate: MONGODB-X509 mechanism authenticates a username derived from the distinguished subject name of a X.509 certificate.

  • LDAP (Lightweight Directory Access Protocol): Lightweight Directory Access Protocol (LDAP) service is used for MongoDB authentication.

  • SCRAM-SHA-256 Authentication: SCRAM-SHA-256 authentication means that the password is never actually sent over the wire, but rather a computed proof that the client password is the same as the password the server knows (non-ASCII characters in passwords are not supported).

  • Auto: Detect the default authentication mechanism for users with stored SCRAM-SHA-1 and SCRAM-SHA-256 credentials (requires MongoDB 4.0 or newer).

User Name

User Name.

Password

Password.

Compression

Compress MongoDB data on the wire.

Secure connections

Enable TLS for MongoDB connections.

Allow Invalid Hostnames

Allow invalid hostnames in the server certificate for TLS connections.

Allow Invalid Certificates

Allow invalid server certificates for TLS connections.

MongoDB URI

If not empty, the MongoDB URI takes precedence, overriding all other MongoDB connection settings. The string needs to be URL-encoded.

Effective MongoDB URI

The actual MongoDB URI used for current connection.

Filter Options

Options to filter events.

Stream Objects

List of event stream object paths.

Sources

List of RegEx strings (eg. '\\^/System/Core/obj1 \\^/obj2') which will be evaluated against the event source field.

Messages

List of RegEx strings which will be evaluated against the message field.

Event Types

Event type filters to select one or multiple event types.

  • Filter Simple Events: If switched on, simple events (OPC_SIMPLE_EVENTS) will be filtered by the server and not acquired by the EventStream object.

  • Filter Condition Events: If switched on, condition events (OPC_CONDITION_EVENTS) will be filtered by the server and not acquired by the EventStream object.

  • Filter Tracking Events: If switched on, tracking events (OPC_TRACKING_EVENTS) will be filtered by the server and not acquired by the EventStream object.

Categories

List of RegEx strings which will be evaluated against the event categorie field.

Severities

Allows to filter for multiple severity ranges.

Severity From

Lower limit for the severity filter of events.

Severity To

Upper limit for the severity filter of events.

Conditions

List of RegEx strings which will be evaluated against the event conditions field.

Sub-Conditions

List of RegEx strings which will be evaluated against the event sub-conditions field.

Filter Script

A Lua script returning true or false when passed in a event document.

Custom Options

Compound to hold various structures to customize the object and to be read and written to by Lua-Script code or external interfaces.

Custom String

A generic string buffer to be used programmatically for custom purposes.

Custom Properties

This is an extensible set of named strings which can be used programmatically for custom purposes.

Property Name

A custom property name which can be used programmatically.

Property Value

The value of the custom property which can be read and written programmatically.

Custom Tables

This is an extensible set of named tables which can be used programmatically for custom purposes.

Table Name

A custom table name which can be used programmatically.

Table Data

Handles an entire table organized in columns and rows. The data can easily (cut, copy and paste) be exchanged with table-oriented data of other software products, e.g. MS Excel.