Custom File Data Store

The Custom File Data Store object works in conjunction with the File Transfer objects (File Source and File Sink) to allow the user to store files sourced from different host locations (under different components) in the Mongo repository. The Data Store can be created in the I/O Model beneath a Core or a Data Store Group object.

The Custom File Data Store can be selected as a long term storage option when configuring the File Storage Type property on the File Sink object class.

The Custom File Data Store is designed to use with the File Transfer objects (File Source and File Sink) and is not analogous to the System File Store. It is therefore not available for use with File Pointer Properties and inversely the System File Store is not available for use with the File Transfer objects.
Unlike other Custom Data Stores (for example the Custom Time Series Data Store) that are used to historize data from objects in the system, the Custom File Data Store does not need to be configured in the Data Store Sets property of the Core to be available.

Use Cases

  • Long term storage of files transferred from other components.

Quick Configuration

  1. Right click on a Core or a Data Store Group object and select Admin  New  Data Stores  Custom File Data Store from the context menu to open the Create Object wizard.

  2. Give the object a unique name and enter a description if necessary.

  3. Click on the Upload Restrictions options from the wizard sidebar, here you can select certain file extensions to be whitelisted or blacklisted for upload into the Custom File Data Store. A maximum file size can also be configured here.

  4. Click on the File Data Store options from the wizard sidebar. The default database can be changed here, along with data retention period and purge settings (what happens to the object data if the object is deleted).

  5. If necessary, the MongoDB connection settings can also be changed (for example, if you wish to connect to a remote MongoDB instance or configure replication sets).

    It might be necessary to configure the Authentication Mode and User Name/Password if the MongoDB instance you are using also has this configured. The default connection is to the localhost/port but the other connection settings are not provided automatically.
  6. Click Create to create the object in the I/O model tree.

  7. Visit the How to use File Transfer Objects (File Source and File Sink) section of the documentation to learn how to configure objects to store files in the Custom File Data Store.

Object Properties

Common

Object Name

The user-modifiable object name. This name overrides the name which has been supplied by the external system. It must be unique within the collection of objects of the parent object.

Object Description

This is the user-modifiable object description. This text overrides the description which has been supplied by the external system.

Display Alias

Alternate label for objects to be used for easier identification in the displays.

Attachments

File attachments stored in MongoDB file store.

Upload Restrictions

Settings to restrict which files can be uploaded to the file store.

File Extensions Whitelist

Comma separated list of allowed file extensions.

File Extensions Blacklist

Comma separated list of disallowed file extensions.

Max File Size

Maximum allowed upload size per file.

File Data Store

Custom file data store configuration.

Database

The database storing files.

Retention Time

The number of days files are retained in the MongoDB store.

Partial Files Purge Time

The partially uploaded files before the specified time are purged.

Purge Files

If enabled, files from store are purged.

MongoDB Connection

This property compound combines the properties of the MongoDB Connection e.g. the connection strings to the mongoDB.

Connection String

The connection string for the external system; depending on the type of connection.

Replication Set Name

Configuration name for MongoDB Replica Set.

Replication Set 1

The connection string for the core service to replication set 1.

Replication Set 2

The connection string for the core service to replication set 2.

Authentication Mode

Authentication mode to be used connecting the MongoDB instance.

  • No Authentication: No Authentication is performed when the System connects to a certain instance of MongoDB.

  • Challenge-Response: MONGODB-CR is the default authentication mechanism supported by MongoDB. Username and Password needs to be provided.

  • SCRAM-SHA-1 Authentication: Salted Challenge Response Authentication Mechanism (SCRAM) means that the password is never actually sent over the wire, but rather a computed proof that the client password is the same as the password the server knows.

  • X.509 Certificate: MONGODB-X509 mechanism authenticates a username derived from the distinguished subject name of a X.509 certificate.

  • LDAP (Lightweight Directory Access Protocol): Lightweight Directory Access Protocol (LDAP) service is used for MongoDB authentication.

  • SCRAM-SHA-256 Authentication: SCRAM-SHA-256 authentication means that the password is never actually sent over the wire, but rather a computed proof that the client password is the same as the password the server knows (non-ASCII characters in passwords are not supported).

  • Auto: Detect the default authentication mechanism for users with stored SCRAM-SHA-1 and SCRAM-SHA-256 credentials (requires MongoDB 4.0 or newer).

User Name

User Name.

Password

Password.

Compression

Compress MongoDB data on the wire.

Secure connections

Enable TLS for MongoDB connections.

Allow Invalid Hostnames

Allow invalid hostnames in the server certificate for TLS connections.

Allow Invalid Certificates

Allow invalid server certificates for TLS connections.

MongoDB URI

If not empty, the MongoDB URI takes precedence, overriding all other MongoDB connection settings. The string needs to be URL-encoded.

Effective MongoDB URI

The actual MongoDB URI used for current connection.

Custom Options

Compound to hold various structures to customize the object and to be read and written to by Lua-Script code or external interfaces.

Custom String

A generic string buffer to be used programmatically for custom purposes.

Custom Properties

This is an extensible set of named strings which can be used programmatically for custom purposes.

Property Name

A custom property name which can be used programmatically.

Property Value

The value of the custom property which can be read and written programmatically.

Custom Tables

This is an extensible set of named tables which can be used programmatically for custom purposes.

Table Name

A custom table name which can be used programmatically.

Table Data

Handles an entire table organized in columns and rows. The data can easily (cut, copy and paste) be exchanged with table-oriented data of other software products, e.g. MS Excel.