How to Export History Data

system:inmation provides two convenient ways to export large amounts of historical time-series VQT data, stored in the repository, to different file formats. Both methods are covered here and the chosen method depends on whether the history export is likely to be a one-time event or a recurring process.

The two methods described here are suitable for creating dump files for large amounts of history data. To make export files for smaller amounts of history data, the HistoryGrid Export feature can be used to create files in a number of different formats.

History Exporter - This object can be used for a simple, one-time history file export. The object can then be deleted after the export files have been created.
History Controller - This object can be used if you wish to export history data continuously to file. New files will be created as the history data is archived.

For theses examples, we will use historical data from the Demo Data items, please go to the Using Lua Scripting Jump Start document for details on how to configure these items.

Figure 1. Demo Data Namespace used in the examples

Export History Data with History Exporter object

The History Exporter object is ideal for creating one-time exports to file for selected object’s history data. The data is exported as a tab delimited text file to a directory of your choice.

To create a History Exporter object, right-click on the Core object and select Admin New History History Exporter from the context menu.
In the Create Object wizard, give the object a name and click Create.
Select the History Exporter in the I/O Model and open the Exporter Configuration property compound in the Object Properties panel. From there, select the Tag Configuration table property.
Tags from the I/O Model can be added to the table via drag and drop, or imported from a external file (an Excel, CSV or JSON file). Drag and drop some selected tags from the Demo Data objects into the table to fill the Object Path column. The Object Name can also be used to fill the Target ID column, but this is only strictly necessary if using the Exporter with a History Sink attached to an external historian. Click OK when finished.

To add more rows to the Tag Configuration table, click the "Add row" icon in the menu bar.
Return to the Exporter Configuration property compound in the Object Properties panel and enter a directory location for the Export Folder. The export files will be created here.
Open the Aggregation property compound in the Object Properties panel and select the "Raw Data" aggregate from the drop-down menu. Other Aggregates can be selected here for the file export. Also, the time period of history data included in the file export can be configured with the Start and End times. If the values are left as <null> then the entire history data for the selected objects will be exported.
The Asset Hierarchy property compound gives options to export contextual asset data alongside the history data. The Output Options property compound allows you to configure the format of the exported data, including decimal places, timestamp and quality format options. For this simple example we will stick to the default settings however.
To begin the history export immediately, return to the Exporter Configuration property compound and change the Processing Mode from "Configuration" to "Export" and click Apply to begin.
The progress of the export can be followed in the Diagnostics Exporter Status field that will display the percentage of data exported. Any errors will also be displayed here.
When the export is complete, go to the Export Folder directory to find the tab delimited export file.
Open the file in Excel (import wizard will detect tab delimited file) or any other text editor) to view the contents.

Export History Data with History Controller

The History Transfer Controller object is used to oversee history transfer management operations. They control the transfer of data from one historian system to another, through History Transporter and History Sink objects that can be created and configured in the History Transfer Controller object.

They are useful for exporting of large amounts of history data over a longer period of time and for the continuous export of history data for selected objects.

In this example, we will; use the History Transfer Controller to export data from the MongoDB historian, to multiple JSON files. This requires a History Sink to be created as part of the History Transfer Controller configuration.

To configure the History Transfer Controller for history data export requires that you have a Connector service running and have created a corresponding Connector object in the I/O Model.

To create a History Transfer Controller object, right-click on the Core object and select Admin New History History Transfer Controller from the context menu.
In the Create Object wizard, give the object a name and change the Recurrence property to "Each minute" (this configures how often the data transfer operation will be done). Click Create to create the object in the I/O Model.
Select the History Transfer Controller in the I/O Model and go to the Object Configuration property compound in the Object Properties panel. Click on the + button to add an Object Configuration block. Enter a Block Name and set the Depth to (the default "*-100days" will export the last 100 days of historical data).
Click on BLock Configuration to open the table property. Tags from the I/O Model can be added to the table via drag and drop, or imported from a external file (an Excel, CSV or JSON file). Drag and drop some selected tags from the Demo Data objects into the table to fill the Object Path column. Use the tag’s Object Name to fill the Target ID column. Click OK when finished.
Open the Sink Configuration property compound in the Object Properties panel. Here we configure the name and essential properties for the History Sink object that will export the history data. Firstly, specify the Connector that will be the parent object of the History Sink by selecting the object using the object picker (for this example we use the local Connector). Change the History Sink Interface to "Disk (JSON Dump File)" and enter a directory path to export the file to. Click Apply to save the changes.
To begin the history export immediately, go to the Common property compound and change the Processing Mode from "Configuration" to "Export" and click Apply to begin.
The configured History Sink Object is created underneath the specified Connector along with some children variable groups to monitor performance.
The progress of the history export can be followed in the Diagnostics section to the History Controller properties.
The progress of the history data export for each tag can also be followed by opening the Controller Status table in the Common property compound. Any errors in the export process relating to individual tags will be visible here.
Each cycle (running every minute) will result in the creation of a JSON export file in the chosen file path, named with a timestamp corresponding to the time each cycle was started. The cycles will continue exporting the history data to file with the maximum number of points exported in each cycle corresponding to the History Limit property setting in the Common property compound.

The History Controller can also be configured to connect and export data to a MQTT publisher. Please visit the History Controller with MQTT publisher Jumpstart section of the documentation for more details.