Create output work items

Currently, the API payloads are created only in memory - they are not persisted. It's time to save the payloads as work items. You decide to create one work item per API payload:

producer.robot:

*** Settings ***
Documentation     Inhuman Insurance, Inc. Artificial Intelligence System robot.
...               Produces traffic data work items.
Library           Collections
Library           RPA.HTTP
Library           RPA.JSON
Library           RPA.Robocorp.WorkItems
Library           RPA.Tables

*** Variables ***
${TRAFFIC_JSON_FILE_PATH}=    ${OUTPUT_DIR}${/}traffic.json
# JSON data keys:
${COUNTRY_KEY}=    SpatialDim
${GENDER_KEY}=    Dim1
${RATE_KEY}=      NumericValue
${YEAR_KEY}=      TimeDim

*** Tasks ***
Produce traffic data work items
    Download traffic data
    ${traffic_data}=    Load traffic data as table
    ${filtered_data}=    Filter and sort traffic data    ${traffic_data}
    ${filtered_data}=    Get latest data by country    ${filtered_data}
    ${payloads}=    Create work item payloads    ${filtered_data}
    Save work item payloads    ${payloads}

*** Keywords ***
Download traffic data
    Download
    ...    https://github.com/robocorp/inhuman-insurance-inc/raw/main/RS_198.json
    ...    ${TRAFFIC_JSON_FILE_PATH}
    ...    overwrite=True

Load traffic data as table
    ${json}=    Load JSON from file    ${TRAFFIC_JSON_FILE_PATH}
    ${table}=    Create Table    ${json}[value]
    [Return]    ${table}

Filter and sort traffic data
    [Arguments]    ${table}
    ${max_rate}=    Set Variable    ${5.0}
    ${both_genders}=    Set Variable    BTSX
    Filter Table By Column    ${table}    ${RATE_KEY}    <    ${max_rate}
    Filter Table By Column    ${table}    ${GENDER_KEY}    ==    ${both_genders}
    Sort Table By Column    ${table}    ${YEAR_KEY}    False
    [Return]    ${table}

Get latest data by country
    [Arguments]    ${table}
    ${table}=    Group Table By Column    ${table}    ${COUNTRY_KEY}
    ${latest_data_by_country}=    Create List
    FOR    ${group}    IN    @{table}
        ${first_row}=    Pop Table Row    ${group}
        Append To List    ${latest_data_by_country}    ${first_row}
    END
    [Return]    ${latest_data_by_country}

Create work item payloads
    [Arguments]    ${traffic_data}
    ${payloads}=    Create List
    FOR    ${row}    IN    @{traffic_data}
        ${payload}=
        ...    Create Dictionary
        ...    country=${row}[${COUNTRY_KEY}]
        ...    year=${row}[${YEAR_KEY}]
        ...    rate=${row}[${RATE_KEY}]
        Append To List    ${payloads}    ${payload}
    END
    [Return]    ${payloads}

Save work item payloads
    [Arguments]    ${payloads}
    FOR    ${payload}    IN    @{payloads}
        Save work item payload    ${payload}
    END

Save work item payload
    [Arguments]    ${payload}
    Create Output Work Item
    Set Work Item Variable    traffic_data    ${payload}
    Save Work Item

Save work item payloads is your custom keyword that takes the list of payloads as an argument and calls another custom keyword (Save work item payload) to save each payload as a work item.

The RPA.Robocorp.WorkItems library provides three useful keywords for our current need:

  1. Create Output Work Item creates a new empty output work item.
  2. Set Work Item Variable saves our payload with the given variable name (traffic_data).
  3. Save Work Item persists the work item. If you don't call it, your work item exists only in memory and is not saved.

After running the producer task, a devdata/work-items-out/run-1/work-items.json file is automatically created. A new run-n folder is created every time you run your robot. Those output work items come in handy later when implementing and testing your consumer robot since they can be used as test input for the consumer.

The work-items.json contains your output work items:

[
    {
        "payload": {
            "traffic_data": {
                "country": "VCT",
                "year": 2011,
                "rate": 3.69293
            }
        },
        "files": {}
    },
    {
        "payload": {
            "traffic_data": {
                "country": "SWError",
                "year": 2019,
                "rate": 3.13947
            }
        },
        "files": {}
    },...

Each work item has the payload and files properties. In your case, only the payload contains data (the traffic data your robot created). The files property is empty since this robot does not use files.

Your producer robot is now complete!

The producer:

  • downloads the raw traffic data.
  • transforms the raw data into a business data format.
  • saves the business data as work items that can be consumed later.

Well done, you! Time to consume stuff, next.