Transform the raw data into business data
The sales system API expects the traffic data in a specific format. You currently have all the necessary raw data, but you can not send that as is since the API would not understand that format.
You decide to implement a keyword to handle the transformation of raw data to sales system API payloads:
producer.robot:
*** Settings ***
Documentation Inhuman Insurance, Inc. Artificial Intelligence System robot.
... Produces traffic data work items.
Library Collections
Library RPA.HTTP
Library RPA.JSON
Library RPA.Tables
*** Variables ***
${TRAFFIC_JSON_FILE_PATH}= ${OUTPUT_DIR}${/}traffic.json
*** Tasks ***
Produce traffic data work items
Download traffic data
${traffic_data}= Load traffic data as table
${filtered_data}= Filter and sort traffic data ${traffic_data}
${filtered_data}= Get latest data by country ${filtered_data}
${payloads}= Create work item payloads ${filtered_data}
*** Keywords ***
Download traffic data
Download
... https://github.com/robocorp/inhuman-insurance-inc/raw/main/RS_198.json
... ${TRAFFIC_JSON_FILE_PATH}
... overwrite=True
Load traffic data as table
${json}= Load JSON from file ${TRAFFIC_JSON_FILE_PATH}
${table}= Create Table ${json}[value]
RETURN ${table}
Filter and sort traffic data
[Arguments] ${table}
${max_rate}= Set Variable ${5.0}
${rate_key}= Set Variable NumericValue
${gender_key}= Set Variable Dim1
${both_genders}= Set Variable BTSX
${year_key}= Set Variable TimeDim
Filter Table By Column ${table} ${rate_key} < ${max_rate}
Filter Table By Column ${table} ${gender_key} == ${both_genders}
Sort Table By Column ${table} ${year_key} False
RETURN ${table}
Get latest data by country
[Arguments] ${table}
${country_key}= Set Variable SpatialDim
${table}= Group Table By Column ${table} ${country_key}
${latest_data_by_country}= Create List
FOR ${group} IN @{table}
${first_row}= Pop Table Row ${group}
Append To List ${latest_data_by_country} ${first_row}
END
RETURN ${latest_data_by_country}
Create work item payloads
[Arguments] ${traffic_data}
${payloads}= Create List
FOR ${row} IN @{traffic_data}
${payload}=
... Create Dictionary
... country=${row}[SpatialDim]
... year=${row}[TimeDim]
... rate=${row}[NumericValue]
Append To List ${payloads} ${payload}
END
RETURN ${payloads}
- The
Create work item payloadskeyword loops the list of traffic data - essentially rows. - For each row, you create a new dictionary (a data structure that supports named keys).
- You append the dictionaries to a list that you then return from the keyword.
After running the robot, the log provides a small snippet of the payloads:
${payloads} = [{'country': 'VCT', 'year': 2011, 'rate': 3.69293}, {'country': 'SWError', 'year': 2019, 'rate': 3.13947}, {'country': 'SWE', 'year': 2018, 'rate': 3.61718}...
Looks good. But what's that smell? SpatialDim, TimeDim, NumericValue appear in the code more than once. Duplication? In your code? Better deal with it immediately.
You decide to create variables for the data keys to avoid duplicating the values:
producer.robot:
*** Settings ***
Documentation Inhuman Insurance, Inc. Artificial Intelligence System robot.
... Produces traffic data work items.
Library Collections
Library RPA.HTTP
Library RPA.JSON
Library RPA.Tables
*** Variables ***
${TRAFFIC_JSON_FILE_PATH}= ${OUTPUT_DIR}${/}traffic.json
# JSON data keys:
${COUNTRY_KEY}= SpatialDim
${GENDER_KEY}= Dim1
${RATE_KEY}= NumericValue
${YEAR_KEY}= TimeDim
*** Tasks ***
Produce traffic data work items
Download traffic data
${traffic_data}= Load traffic data as table
${filtered_data}= Filter and sort traffic data ${traffic_data}
${filtered_data}= Get latest data by country ${filtered_data}
${payloads}= Create work item payloads ${filtered_data}
*** Keywords ***
Download traffic data
Download
... https://github.com/robocorp/inhuman-insurance-inc/raw/main/RS_198.json
... ${TRAFFIC_JSON_FILE_PATH}
... overwrite=True
Load traffic data as table
${json}= Load JSON from file ${TRAFFIC_JSON_FILE_PATH}
${table}= Create Table ${json}[value]
RETURN ${table}
Filter and sort traffic data
[Arguments] ${table}
${max_rate}= Set Variable ${5.0}
${both_genders}= Set Variable BTSX
Filter Table By Column ${table} ${RATE_KEY} < ${max_rate}
Filter Table By Column ${table} ${GENDER_KEY} == ${both_genders}
Sort Table By Column ${table} ${YEAR_KEY} False
RETURN ${table}
Get latest data by country
[Arguments] ${table}
${table}= Group Table By Column ${table} ${COUNTRY_KEY}
${latest_data_by_country}= Create List
FOR ${group} IN @{table}
${first_row}= Pop Table Row ${group}
Append To List ${latest_data_by_country} ${first_row}
END
RETURN ${latest_data_by_country}
Create work item payloads
[Arguments] ${traffic_data}
${payloads}= Create List
FOR ${row} IN @{traffic_data}
${payload}=
... Create Dictionary
... country=${row}[${COUNTRY_KEY}]
... year=${row}[${YEAR_KEY}]
... rate=${row}[${RATE_KEY}]
Append To List ${payloads} ${payload}
END
RETURN ${payloads}
That's better!