Dropzone Basics

The Datasource object represents external data sources in the I/O model of the system and can be configured to be the interface to OPC servers or any other data producing source. When the Datasource is configured as a Dropzone it provides an interface to any file-based data you wish to import into your system. The Dropzone can be created underneath a Connector object like any other datasource object and must be linked to a "Disk Folder" that will be the "Drop" location for any files that are imported. The Dropzone must then be configured to specifically process the type and format of your import files In the following examples, we will demonstrate how to create and configure a Dropzone data source to handle different file formats. Although it is impossible to cover every individual file format that you may encounter, this guide should enable you (with some trial and error) to identify the steps you need to take to successfully import data files.

Creating a Dropzone Datasource

The first step is to create a Dropzone Datasource under a Connector in your I/O model in DataStudio (see the Infrastructure Hands On guide for information on creating a Connector object). You can create the Dropzone under either a local or a remote Connector, however, if using a remote Connector remember that the Disk Folder where you copy the files for import should be on the same machine as the Connector.

  1. To create a Dropzone datasource in DataStudio, select a Connector in the I/O model, right-click and select Admin  New  External Interfaces → Datasource to open the Create Object wizard:

  2. In the Create Object wizard, give the datasource a name, e.g. Dropzone Example, and click the Next.

  3. On the Server Type page of the wizard, select Dropzone as the server type from the drop-down menu.

  4. In the Disk Folder property, enter the filepath location of the folder you wish to use as the drop point to import files. In the _inmation.root folder there is a drop folder that contains various sub-folders that can be used as the disk folder, but you can choose an alternative folder location if you wish. If the folder does not already exist, it will be created when the Dropzone object is created.

    The configured Disk Folder directory must be local to the parent Connector instance. Remote access over a network to a file is not supported, either via UNC path ("\...") or over a mapped network drive ("H:\.."). If it is not possible to install a Connector instance at the source, the files can be copied from one location to another, for example, using Robocopy and shared directories. In these cases it is recommended to use Dropzone in conjunction with a File Monitor with the in built functionality to feed the Dropzone files for processing. Please see the Dropzone: Use with File Monitor and File Transfer section of the Jumpstart for more details.

  5. The rest of the options on this page determine the types of files that can be imported by the dropzone object (txt and csv are selected as default, please see below for more information) and the configuration to parse the import file. The following sections will cover the Dropzone Processing options in detail. If you already know the Dropzone Processing option suitable for your file type, select it from the drop-down menu, otherwise select Column Based from the Dropzone.

  6. Click on the arrow by Dropzone Options to expand the configuration options. Then expand the Dropzone New Item Preset options and the Value Storage Strategy sections (you may need to scroll down) and check the Raw History checkbox.

    File data imported using Dropzone is fundamentally historical data, therefore, the Dropzone preset options should usually be configured to archive history. Otherwise any history calls made on items will indicate that there is no data in the archive.

  7. Click Create to create the Dropzone object in the I/O Model tree.

File Types

The file type flags options determine which types of file (determined by file extension) will be processed by the Dropzone datasource. Currently the options are:

  • txt

  • csv

  • xml

  • json

  • xlsx

  • accdb

The Dropzone will look for the file extension when the file is put into the disk folder. If the file extension matches one of the selected file types then it will be processed and moved to the "work" directory in the disk folder. If the file extension is not recognized or has not been selected then it will not be processed and instead transferred to the "noproc" directory in the disk folder.

If no file type flags are selected, then no files will be processed (unless the Wildcard property field is used).

Wildcard Option

The Wildcard property field can be used to match and process Dropzone files based on filename. The Wildcard field accepts the Windows Wildcard characters "*" and "?" (representing a string of characters or a single character respectively) and can be used alone or alongside the file types options explained above to providing further options to match and selectively process files dropped in the Dropzone disk folder.

The filename is matched against the Wildcard field if it doesn’t match any of the configured file types. Therefore if the csv file type is flagged then all csv files will be processed regardless of the content of the Wildcard field.

The Wildcard field can be used to specify file extensions that are not available in the file type flags, (for example using "*.ext" ) or even to allow all file extensions and file names to be matched for processing (by entering just the "*"). More usually, it can be used to match certain file names, for example, "ab*" would match all file names that begin with "ab".

Using the "?" wildcard character, it is possible to selectively match filenames for processing that have similar filenames that might change by one or more characters. For example, numbered files such as "001-testfile", "010-testfile", "090-testfile" could all be matched for processing by using the Wildcard string "???-testfile".