Azure Machine Learning Studio Using Python
Azure Machine Learning Studio supports Python script execution, enabling the user to read and write data in system:inmation. Scripts can be compressed as zip files and zip files can be imported and saved as datasets in the Azure ML Studio.
The example script below contains a few basic functions for accessing data stored in the system. The script should be saved as a python file(*.py) and then added to a zip archive. The zip file can then be uploaded from the local computer as data set to Azure ML Studio.
| Make a note of the saved filename as you will need it to reference in subsequent scripts that access the functions. For the following examples we use the name pythonclientexample. |
import json
import requests
import datetime
import pandas as pd
class Client:
baseURL = "http://localhost:8002"
options = {
"auth": {
"username": '',
"password": '',
"authority": 'inmation',
"grant_type": "password"
}
}
def __init__(self, baseURL, options):
self.baseURL = baseURL
self.options = options
def reqHeaders(self):
headers = {
"Content-Type": 'application/json'
}
options = self.options or {}
auth = options["auth"] or {}
headers["username"] = auth["username"]
headers["password"] = auth["password"]
headers["authority"] = auth["authority"]
headers["grant_type"] = auth["grant_type"]
return headers
def createHistoryRequestItem(self, path, aggregate = None):
agg = aggregate or 'AGG_TYPE_INTERPOLATIVE'
return {
"p": path,
"aggregate": agg
}
def createItemValue(self, p, v, q, t):
return {
"p": p,
"v": v,
"q": q,
"t": t
}
def readHistoricalData(self, items, start_time, end_time, intervals_no):
reqBody = {
"start_time": start_time,
"end_time": end_time,
"intervals_no": intervals_no,
"items": items
}
headers = self.reqHeaders()
url = "{}/api/v2/readhistoricaldata".format(self.baseURL)
print("inmation Web API URL: '{}'".format(url))
print("inmation Web API headers: '{}'".format(headers))
print("inmation Web API body: '{}'".format(json.dumps(reqBody)))
r = requests.post(url, data = json.dumps(reqBody), headers = headers)
return r
def write(self, itemvalues):
reqBody = {
"items": itemvalues
}
headers = self.reqHeaders()
url = "{}/api/v2/write".format(self.baseURL)
print("inmation Web API URL: '{}'".format(url))
print("inmation Web API headers: '{}'".format(headers))
print("inmation Web API body: '{}'".format(json.dumps(reqBody)))
r = requests.post(url, data = json.dumps(reqBody), headers = headers)
return rInvoking function
Functions from the Zip file can be invoked by adding Execute Python Script tile. Example code for invoking the ReadHistoricalData endpoint:
import pandas as pd
from inmationhttpclient import Client
baseURL = "http://localhost:8002"
options = {
"auth": {
"username": '%USERNAME%',
"password": '%PASSWORD%',
"authority": 'inmation',
"grant_type": "password"
}
}
client = Client(baseURL, options)
def azureml_main(dataframe1 = None, dataframe2 = None):
start_time = "2018-01-01T00:00:00.000Z"
end_time = "2018-01-02T00:00:00.000Z"
intervals_no = 31
items = [
{
"p": "%PATH%",
"aggregate": "AGG_TYPE_INTERPOLATIVE"
},
{
"p": "%PATH%",
"aggregate": "AGG_TYPE_INTERPOLATIVE"
}
]
r = client.readHistoricalData(items, start_time, end_time, intervals_no)
print("JSON")
jsonRes = r.json()
print(type(jsonRes))
dataFrameData = {}
if isinstance(jsonRes, dict):
print("r.json is an dictionary!!")
data = jsonRes['data']
if isinstance(data, dict):
print("data is an dictionary!!")
items = data['items']
if isinstance(items, list):
print("items is a list!!")
for item in items:
objPath = item['p']
itempath = objPath.split('/')
objName = itempath[-1]
intervals = item['intervals']
v = []
q = []
t = []
print(type(v))
dataFrameData[objName + '_v'] = v
dataFrameData[objName + '_q'] = q
dataFrameData[objName + '_t'] = t
print(type(v))
if isinstance(intervals, list):
print("intervals is a list!!")
for interval in intervals:
v.append(interval['V'])
q.append(interval['Q'])
t.append(interval['T'])
print(v, q, t)
dataframe = pd.DataFrame(data = dataFrameData)
return dataframeThe following arguments should be provided: item paths, start and end time, intervals and aggregation.
The zip file, which stores the inmation http Python client creation script should be connected to the Python script tile. After these files have been connected click run on the Python script tile.
Access input data
After successful execution, a green tick mark will be displayed at the top right corner of the screen.
To view the imported data, click the right mouse button, choose Result Data set, Visualize option. Various statistics about each column are displayed. Data can be modified and filtered, missing values deleted or replaced.
Store model data to system:inmation
After incoming data has been shaped and the machine learning module created, the output can be written to system:inmation.
Deploy another script execution tile in ML studio, connect the tile with data output and the zip fil. Create a new item in the I/O model, copy the full property path, fill it in the example script item path place holder and click on run.
After successful run, values are set to the item in the DataStudio.
import pandas as pd
from pythonclientexample import Client
import json
baseURL = "http://localhost:8002"
options = {
"auth": {
"username": '%USERNAME%',
"password": '%PASSWORD%',
"authority": 'inmation',
"grant_type": "password"
}
}
client = Client(baseURL, options)
p = "%PATH%"
def azureml_main(dataframe1 = None, dataframe2 = None):
colnames = dataframe1.columns
scores = dataframe1[colnames[6]]
timestamps = dataframe1[colnames[0]]
itemValues = []
for index, timestamp in timestamps.iteritems():
v = int(scores[index])
t = int(timestamp)
itemValues.append(
{
"p": p,
"v": v,
"t": t
}
)
print(itemValues)
r = client.write(itemValues)
print(r.json())
return dataframe1