Performance Indicators

The SaF sub-system provides performance indicators for monitoring the persistent buffer. For each data type (real-time data, log entries, and events) there are four different performance indicators:

Buffer

The current number of items in the buffer for this data type.

Inserts

The item insertion rate (number of items per second) for this data type.

Sent (applies to Connectors only)

The rate at which items for this data type are send to the Core component.

Received (applies to Cores only)

The rate at which items for this data type are received from a remote component with a persistent SaF sub-system configuration.

On service start, the number of data items in the persistent buffer is counted. In case of a large buffer, the Buffer indicators may show a steady increase until the initial counting is finished. Received only shows data forwarded from a remote component if that component uses a SaF sub-system with a persistent buffer.

The following usage scenarios show the different behavior of the performance counters. Unless otherwise stated, the system setup involves a functional Core - Connector communication and data being transported through the SaF sub-system (archived real-time data, log messaged, or events). Further, there is no malfunction in the system other than the one described in the individual scenario.

Scenario 1: The MongoDB repository is unavailable

If the MongoDB repository becomes unavailable, the performance indicators will show:

Core

  1. Buffer: Continuous growth (if Inserts is non-zero)

  2. Inserts: Stable (on average), non-zero

  3. Received: Stable (on average), non-zero

Connector

  1. Buffer: Continuous growth (if Inserts is non-zero)

  2. Inserts: Stable (on average)

  3. Sent: Stable (on average), non-zero

When the MongoDB repository becomes available again, the Core Buffer counter will steadily decrease, forwarding its data to the Mongo database.

Scenario 2: No Connector Connection

Assume that there is only one configured Connector in the system. If the connection to the Connector component is lost, the performance indicators will show:

Core

  1. Buffer: Stable (on average)

  2. Inserts: Stable (on average). Shows insert from Core items only

  3. Received: Zero

Connector

  1. Buffer [not observable]: Continuous growth (if Inserts is non-zero)

  2. Inserts [not observable]: Stable (on average)

  3. Sent [not observable]: Zero

When the connector connection is re-established, the Connector buffer is drained and the Core buffer forwards data to MongoDB. This results in spikes of the Inserts, Received, and Sent counters. The Connector* Buffer decreases and the Core *Buffer may increase depending on the throughput difference between the Connector - Core and the Core - MongoDB communication.

Scenario 3: Insufficient Network Bandwidth

Assume that the Core - MongoDB connection can handle any work load, but the Core - Connector connection does not provide enough bandwidth to transport all data from the Connector to the Core. The performance indicators will show:

Core

  1. Buffer: Stable (on average)

  2. Inserts: Stable (on average), non-zero

  3. Received: Stable (on average), non-zero

Connector

  1. Buffer: Continuous growth

  2. Inserts: Stable (on average), non-zero

  3. Sent: Stable (on average), non-zero

If more bandwidth becomes available, such that all Connector data can be transported, the Connector Buffer will decrease until it is stable again. Otherwise the persistent buffer grows until it reaches its configured limit, resulting in data being dropped and warning messages.

Depending on the configuration, the warning messages generated by the Connector might get dropped if the SaF persistent buffer reached its limits.

Scenario 4: Persistent Size Limit Reached

If the Connector or Core SaF sub-system reached a limit for its persistent buffer, the performance indicators will show:

Core

  • Buffer:

    1. Retain new items mode: Continuous growth for 30 seconds, followed by a drop.

    2. Retain old items mode: Stable (on average), no new items stored.

  • Inserts:

    1. Retain new items mode: Stable (on average), non-zero.

    2. Retain old items mode: Stable (on average), no new items stored.

  • Inserts

    1. Retain new items mode: Stable (on average), non-zero.

Until some counter action is taken, the SaF sub-system will keep dropping data and log warning messages.

Scenario 5: Change from persistent to volatile buffering

When changing from a persistent (hard disk) buffer to a volatile (main memory) buffer, the persistent buffer is either properly drained or completely cleared by deleting the data.

If the SaF sub-system in its persistent buffer configuration is overloaded (steady increase of the Buffer performance indicator) when switching to a volatile buffer, it is cleared. The Buffer performance counter will show a rapid decline until it is zero and a warning messaged containing the number of deleted items is logged.

Otherwise, the persistent buffer is drained in parallel with the volatile buffer and the Buffer performance indicator will show a steady decrease until it is zero.