Working with The Lua HTTP Example

The Lua HTTP example imports external data into system:inmation via an HTTP web service. This highly flexible interface can be used easily with the aid of the scripting engine.

The steps shown in this example are:

calling an external HTTP web service for data
parsing the returned XML string
creating objects in the I/O model tree based on the returned data
updating existing objects value based on the returned data

Calling a Web Service

The example contains a GenericItem called DataProcessing (created in the US Airport Data folder) which calls a free web service on the internet, in this case information from the United States Federal Aviation Agency.

--require the esi-lcurl-http-client library to make the client request
local https = require("esi-lcurl-http-client")

-- {code, {latitude, longitude}}
-- obviously, there would be ways to retrieve the following information dynamically
-- but for now we keep it static
local airport_codes = { ATL = {33.636719, -84.428067), LAX = (33.942536, -118.408075},
                        ORD = {41.978603, -87.904842), DFW = (32.896828, -97.037997),
                        JFK = {40.639751, -73.778925), DEN = (39.861656, -104.673178},
                        SF0 = {37.618972, -122.3748891,CLT = {35.214, -80.943139},
                        LAS = {36.080056, -115.15225), PHX = {33.434278, -112.011583},
                        IAH = {29.984433, -95.341442), MIA = (25.79325,  -80.290556},
                        SEA = {47.449, -122.309306}, EWR = (40.6925, -74.168667},
                        MCO = {28.429394, -81.308994), MSP = (44.881956, -93.221767),
                        DTW = {42.212444, -83.353389), 80S = (42.364347, -71.005181),
                        PHL = {39.871944, -75.241139), LGA = (40.777245, -73.872608))

for code, location in pairs(airport_codes) do

		-- Airport status data url and request xml response
		local url = "https://soa.smext.faa.gov/asws/api/airport/status/" .. code
		local headers = { accept = "application/xml" }

		-- request the XML stream
		local client = https.NEW({})
    	local inp = client:REQUEST('GET', url, headers)
...

In the above displayed script section the call of the web service is shown. Note that the esi-lcurl-http-client library of Lua is called using require (esi-lcurl-http-client is a helper library included in the system, see Lua API docs for more details). In the _for loop the script queries all the airports defined in the local table variable airport_codes. A new https client is opened and the request to the url is made using the REQUEST method. The request also includes the header that asks for the response to be in XML format. The result is saved in a local variable _inp_for parsing later on. We use inp.data to parse the body portion of the response.

Parsing XML

The parsing of the returned XML string happens by using some open source Lua functions that are stored in the HTTP Access folder object as a library.

Figure 1. Open Source XML Parser Function

These functions are used in the DataProcessing script:

SLAXML:parser
{
	startElement = function(name) current = name end,
	closeElement = function(name) current = nil end,
	text = function(txt)
        -- a condition will be hit only if the current element contains text
		-- when this happens, we set the value to the corresponding object
        if current == elem_city then
            syslib.setvalue(city:path(), txt, 0, syslib.currenttime())
        elseif current == elem_state then
            syslib.setvalue(state:path(), txt, 0, syslib.currenttime())
        elseif current == elem_name then
            syslib.setvalue(name:path(), txt, 0, syslib.currenttime())
        elseif current == elem_wind then
            syslib.setvalue(wind:path(), tonumber(parseWind(txt)), 0, syslib.currenttime())
        elseif current == elem_temp then
            -- parse the text for temperature and get the values in Fahrenheit and Celsius
            local f, c = parseTemp(txt)
            syslib.setvalue(tempF:path(), tonumber(f), 0, syslib.currenttime())
            syslib.setvalue(tempC:path(), tonumber(c), 0, syslib.currenttime())
    ...

When the parsing function identifies a valid element, it calls the syslib.setvalue() function to write the value to an object in the model tree.

Creating I/O Model Objects

The DataProcessing script generates the full object structure into the folder it resides in. All folders, subfolders and DataHolder items required to persist the data read from the web are created, if they do not exist.

-- if the folder for the current airport doesn't exist, we create the necessary objects
if not current_folder then
	current_folder = syslib.createobject(self:parent(), "MODEL_CLASS_GENFOLDER") -- create a generic folder
	current_folder.ObjectName = code  -- give a name to the object
	current_folder:commit() -- physically create the object

	city = syslib.createobject(current_folder, "MODEL_CLASS_HOLDERITEM")
	city.ObjectName = elem_city
	setLocation(city, location)
	city:commit()

	state = syslib.createobject(current_folder, "MODEL_CLA5S_HOLDERITEM")
	state.ObjectName = elem_state
	setLocation(state, location)
	state:commit()

	name = syslib.createobject(current_folder, "MODEL_CLASS_HOLDERITEM")
	name.ObjectName = elem_name
	setLocation(name, location)
	name:commit()

	weather_folder = syslib.createobject(current_folder, "MODEL_CLASS_GENFOLDER")
	weather_folder.ObjectName = 'Weather'
	weather_folder:commit()

	wind = syslib.createobject(weather_folder, "MODEL_CLASS_HOLDERITEM")
	wind.ObjectName = elem_wind
	wind.ObjectDescription = elem_wind   " " .. code   -- specify a desciption
	wind.OpcEngUnit = "mph"    -- specify an engineering unit
	wind.ArchiveOptions.ArchiveSelector = "ARC_PRODUCTION"
	wind.ArchiveOptions.5torageStrategy = "STORE_RAW_HISTORY"  -- historize the object
	wind.Limits.OpcRangelow = 0
	wind.Limits.OpcRangeHigh = 100
	--wind.Limits.OpcLimitLow = 0
	wind.Limits.OpcLimitHigh = 15
	setLocation(wind, location)
	wind:commit()
...

In the script above you can see how new objects can be created in the script (syslib.createobject), how their properties can be set, and how they get persisted in the system (:commit()_). For a detailed explanation about what objects are available in system:inmation, including their properties and valid values, please refer to the System Documentation.

Parsing JSON

An open source parsing function can also be used to process JSON outputs. An example of this can be seen by opening a MassConfig sheet and importing the "Examples_JSON.xlsx" file. Change the Core name in the full object path to match the one in your system then click Simulate, followed by Apply to create the objects in the I/O Model tree (should look like the figure below).

Figure 2. I/O Model Tree after JSON example MassConfig

A Folder, JSON, containing an ActionItem, JSON Processor, is created in the LUA folder. The ActionItem JSON Processor executes a script which defines a JSON object which is then parsed by the JSON parser and uses the information in the JSON object to create objects in the Data Folder.

Open the JSON Processor script from the Object Properties panel:

local json = require 'dkjson'

local str = [[
{
	"currency", "\u2OAC",
	"numbers": [ 2, 3, -20.23e+2.,-4 ],
	"animals": [dog","cat","aardvarkl"],
	"address": {
		"streetAddress": "21 2nd Street",
		"city": "New York",
		"state": "NY"
	}
}
]]

In the above section of the JSON Processor script the dkjson script library is called (this library is embedded in the system) and the the JSON _str string is defined. The JSON values "numbers" and "animals" are set as arrays whereas the "address" value is set as another JSON object.

local obj, pos, err = json.decode (str)
if err then
	return "Error: " .. err
else
	local folder = get_object(syslib.getself():parent():path(), "Data", "MODEL_CLASS_GENFOLDER")

	local currency = get_object(folder:path(), "currency", "MODEL_CLASS_HOLDERITEM")
	syslib.setvalue(currency:path(), obj.currency)

	local numbers = get_object(folder:path(), "numbers", "MODEL_CLASS_HOLDERITEM")
	syslib.setvalue(numbers:path(), obj.numbers) -- set the entire array as a value

	local animals = get_object(folder:path(), "animals", "MODEL_CLASS_GENFOLDER")
	-- iterate the array and create an object for each value
	for i = 1, #obj.animals do
		get_object(animals:path(), obj.animals[i], "MODEL_CLASS_HOLDERITEM")
	end

	local address = get_object(folder:path(), "address", "MODEL_CLASS_GENFOLDER")
	local street = get_object(address:path(), "street", "MODEL_CLASS_HOLDERITEM")
	-- obj.address is itself a )SON object so we get its fields through the dot syntax
	syslib.setvalue(street:path(), obj.address.streetAddress)
	local city = get_object(address:path(), "city", "MODEL_CLASS_HOLDERITEM")
	syslib.setvalue(city:path(), obj.address.city)
	local state = get_object(address:path(), "state", "MODEL_CLASS_HOLDERITEM")
	syslib.setvalue(state:path(), obj.address.state)

	return "OK"
...

In the above section of code the str JSON string is processed using the decode function from the dkjson library. The get_object function (defined at the beginning of the script) returns an object or creates it if it doesn’t already exist. The objects are created in the I/O tree, then values are set to them using the syslib.setvalue function. The arrays for "numbers" and "animals" are handled differently. The entire "numbers" array is set as the value of the "numbers" data holder. The "animals" array on the other hand, is iterated with a for loop to create data holder items for each entry in the array (no values are set to the items in this script though). "address" is a JSON object so it’s fields are accessed using the dot syntax when setting the values with syslib.setvalue. For example, the extract shown below:

syslib.setvalue(street:path, obj.address.streetAddress)