Advanced Transporter Options

The previous pages regarding the Transporter Configuration property compounds in the History Transfer Controller and jumpstarts:using-history-transfer-objects:history-transport-manager-config.adoc#the_transporter_configuration_property_compound[History Transport Manager, window="_blank"] addressed the essential configuration options for use cases where these objects are involved.

In addition to the information provided on those pages, the present page covers more advanced configuration options.

To avoid confusion, do not make any configuration changes directly on History Transporter objects which are managed by History Transfer Controller objects. Always use the Transporter Configuration property compound of the managing object (History Transfer Controller or History Transport Manager) to change the behavior of managed History Transporter objects.

Restricting History Transfer Cycles

The history transfer process is executed in recurring cycles to avoid an overburdening of the system, the source historian, and the network.

These properties are means to control the amount of data that is transferred during each of these history transfer cycles:

  • Slicing - defines how much of the current transfer cycle (which is defined in the Recurrence property) is allowed to be used for active data processing. For example, if set to 50%, this means that out of a Recurrence setting of 1 minute only 30 seconds can be used for active data processing.

    The current history cycle is finished as soon as either the configured Slicing or the Hedging limit is reached.

    Decreasing this value allows to release the stress on the external historian system but will increase the time needed to complete the history transfer.

    • The default value for the Slicing property is 35%.

    • The maximum value is 75%.

    Using a small Slicing value with a small Recurrence value can potentially block the transfer of historical completely.

  • Hedging - specifies the absolute limit of data points to be transferred within a single history transfer cycle. This includes historical data and up-to-date data. The current history cycle is finished as soon as either the configured Hedging or the Slicing limit is reached.

    Decreasing the value for Hedging allows to release the stress on the external historian system but will increase the time needed to complete the history transfer.

    The maximum Hedging value is 1.000.000 data points.
  • Throttling - yields another fragment of CPU time after each tag data transfer. As this occurs with each tag, there is a dependency on the number of tags that are added to the Tag Configuration table. This should be kept in mind before using the property as a control mechanism as the throttling will increase considerably when making data transfers for large numbers of tags

Timing History Transfer Cycles

If multiple history transfer processes are implemented, their transfer cycles eventually overlap due their Recurrenc settings. These overlapps cause peaks of system strain.

In the Continuous Processing Mode the following properties can be used to control when the individual history transfer cycles take place. For Processing Mode Subscription they have not effect.

  • Time Shift - is used when transferring data using the Continuous Processing Mode where near time data is transferred through history calls.

    Time Shift provides an offset (positive or negative) for the end time of the history calls.
    This can be used to suppress unwanted data or as a balancing mechanism to even out time synchronicity differences between different computers. A negative Time Shift value will move the query end time backwards (towards history) whereas a positive value moves the end time forwards (towards the future).

  • Invocation Delay - In the Continuous Processing Mode, to prevent the cumulation of strain caused by multiple History Transporter objects executing their history transfers at the same time due to overlapping settings in their Recurrence properties, setting the Invocation Delay property differently for all the involved History Transporter objects can be used to postpone the triggering of the individual history transfer cycles and therefore to distribute the strain more evenly.

    On History Transporter objects, this property is located inside the Scheduling compound.
  • Delay Reflection - If Delay Reflection is enabled, the data which is added to the source historian during the time span between the unaltered starting point of the history transfer cycle (as defined by the selected option for the Recurrenc property) and the actual starting point which is delayed for the amount of time specified Invocation Delay property, is also included in the transfer to have latest additions to the source historian transferred as soon as possible.

Limiting Transferred Content

The following properties control which information is transferred for the datapoints:

  • Transfer Context - If enabled, the context information (description, limits, engineering unit) is included in the history transfer. Also see the Context Revalidation property below.

  • Transfer UTD - If enabled, data which occurred on the source historian after the history transfer was launched, keeps being transferred. If switched off, the history transfer is stopped when all data historical data as specified in the Depth property is transferred.

  • Transfer History - If enabled, historical data from the source tag is included in the history transfer.

  • Context Revalidation - Context information for data tags hardly ever changes and therefore it’s not necessary to transfer it for all the data points. Context Revalidation defines how frequently context information for a tag is transferred - if the Transfer Context property is enabled.

  • Historization Mode - How historical and up-to-date data are brought into the system is defined by the Historization Mode.
    Options are:

    • Always - All values are transferred by inserting them as the current value.

    • Recent Values [default] - Only the very last value is inserted as the current value of the item, everything else is transferred as historical data without affecting the current value of the item.

    • Continuous Values - Historical data transfer works as in Recent Values mode, the transfer of up-to-date data works as in Always mode

Acquisition Options

  • Acquisition Frame - Specify the number of days a single fetch shall retrieve from the target system. Increase this value with care to avoid overloading the target system.

  • Basket Calls - If enabled, more than one tag can be requested from the source historian on a single request. Tags should be of the same data type (based on the context) and time range.

  • Basket Limit (Historical) - Defines the maximum number of tags to be requested by a Basket Call for historical data.

  • Basket Limit (Continuous) - Defines the maximum number of tags to be requested by a Basket Call for up-to-date data.

  • Overfetch - Activate overfetching. Overfetching extends the time range of the up-to-date queries beyond the last time the history transfer (should have) occurred according to configuration of the Recurrence property, up until the limit defined in Overfetch Maximum or the Last Known Good - whichever is reached earlier.
    The Last Known Good is the timestamp of the last successful history transfer cycle.

    This option can be used to secure the retrieval of data from the source historian which is provided with a delay.

  • Overfetch Minimum - This property defines the lower (or most recent) limit of the time range for which up-to-date data is (re-)retrieved by the Overfetch functionality.

  • Overfetch Maximum - This property defines the maximum limit (furthest back in time) of the time range for which up-to-date data is (re-)retrieved by the Overfetch functionality.
    Overfetching will be restricted by the timestamp of the last successful history transfer cycle (the Last Known Good) or this value - whichever is more recent. To ignore the Last Known Good and enforce this value, enable the Overlap Last Known Good property.

  • Overlap Last Known Good - To avoid the duplication of data points, by default Overfetch does not go further back in time than the Last Known Good - the timestamp of the last successful history transfer cycle. If Overfetch is meant to (re-) retrieve up-to-date data from prior to this timestamp, Overlap Last Known Good needs to be enabled.

  • Enforce Context Query - If Enforce Context Query is enabled, the History Transporter object will execute a context query for all streams in the first history transfer cycle, independent of the next context query time in its state table.