Webinar

October 12, 2021 12:00 PM EDT
The future of RPA: Migrating to Gen-2 toolsOctober 12, 2021 12:00 PM EDT
Are you wondering if it is worth the time and money to switch RPA platforms? Join our RPA experts as they discuss all things migration & Gen-2 tools.

Using work items

What is a Work Item?

Steps in the same process can share data during their execution, thanks to the concept of a Work Item. Each step execution receives work items as inputs from the previous step and creates work items as output to the next one. During the execution, each step can freely read and update the data contained in an item. It's also possible to trigger processes with custom work items via an API call, and the contents can be used in the first step of the process.

A work item contains a payload of JSON-based data and optionally attached files.

On step inputs and outputs

There are two main ways to approach building processes with work items:

  1. Having one work item for each process run that is passed between steps and optionally modified at each stage.
  2. Having each step of the process be responsible for creating output items, with the option of creating multiple items at once.

The first option is the current default and is easier for composing simple sequences of steps. For every trigger of the process, it's ensured that all steps run one after the other if no errors are encountered. The second option is more powerful and enables different patterns such as producer-consumer.

One example use-case of the latter would be a step that receives a large Excel file of orders and splits it into work items per row (the producer), and a second step that processes each work item individually (the consumer). With this pattern the producer can wait to handle incoming requests, and the consumer can run in parallel and potentially process a large volume of data.

Another advantage of splitting work into individual items is the separation of concern. If one row of the input data is wrong, it can be manually fixed and re-run without affecting all other rows.

Example #1: PDF invites printer with one work item

An example of a process with multiple steps where one item is modified and passed between steps can be seen in the PDF Invites printer article.

Example #2: Web store order processor with multiple work items

Get the code and run this example in your favorite editor on our Portal!

Dependencies

The primary way to interact with work items is through the RPA.Robocorp.WorkItems library included in RPA Framework. We start by importing it in our tasks.robot file:

*** Settings ***
Library           RPA.Robocorp.WorkItems

By default, it uses Control Room for storing work items and passing them between steps, but it optionally supports local files for easier development. More on that later.

First task: Creating output work items

In the example linked above, the first task looks like this:

*** Tasks ***
Split orders file
    [Documentation]    Read orders file from input item and split into outputs
    Get work item file    ${ORDER_FILE_NAME}
    Open workbook    ${ORDER_FILE_NAME}
    ${table}=    Read worksheet as table    header=True
    ${groups}=    Group table by column    ${table}    Name
    FOR    ${products}    IN    @{groups}
        Create output work item
        ${rows}=    Export table    ${products}
        Set work item variable    products    ${rows}
        Save work item
    END

The section we are interested in, for now, is the FOR loop. For every row in the table, we call the following keywords:

All changes to work items are stored locally until the keyword Save work item is called.

Second task: Reading work item data

In the second task, we want to read the contents of the incoming work item:

*** Tasks ***
Process order
    [Documentation]    Order all products in input item
    ${rows}=    Get work item variable    products
    ${products}=    Create table    ${rows}
    Open Swag Labs
    Wait until keyword succeeds    3x    1s    Login
    Process order    ${products}
    [Teardown]    Close browser

The most important keyword here is Get work item variable, which reads values from a work item. We read the same variable we set in the first step and can use the contained information to process our order.

The input work item is automatically loaded by the RPA.Robocorp.WorkItems library when execution starts

A keen reader might have seen an example of reading files in work items in the first task! We used the keyword Get work item file to receive the initial Excel file of our orders. How the first step receives this item is described in the following sections.

Configuring Control Room

To run the full process in Control Room, we create a new process under the Workforce section. The process will have the two steps described earlier:

Two step process in Control Room

And because we want to control how output items are created, we will enable the Explicit Output option:

Explicit output configuration

Without this option, the initial work item that triggered the process is automatically copied from one step to the next.

Another feature of Control Room that we can use with our current process is concurrency, the ability to run multiple work items in parallel in the same step. This can be changed under the Advanced section of a process step:

Concurrency limit of a step

In this example we changed it to the value 5. If there are more than our set limit of work items, they will wait until previous items have been processed.

Triggering the process

This robot expects to receive an Excel file of a certain format, named orders.xlsx. There is an example of such an input inside the devdata folder of the robot repository.

You can accomplish this in two ways:

  • Triggering a process manually with a custom work item, using the Control Room UI.
  • Triggering a process via a Control Room API call, passing the work item content.

Manipulating the whole item payload

As we have seen, the Get Work Item Variable and Set Work Item Variable keywords allow you to manipulate the values in your work item conveniently.

There are cases, for example, when receiving a webhook from an external system, when you might want or have to structure the data differently. In these cases, you can take advantage of the Get Work Item Payload and Set Work Item Payload keywords to work with the whole item payload.

Assuming you have a work item structured like this:

[
  {
    "payload": ["Hello", "World"]
  }
]

Note how the item payload is no longer a map of key-value pairs

Because you do not have a typical dictionary as the payload, the Get Work Item Variable, Set Work Item Variable, and Get Work Item Variables keywords are not helpful in this case. Instead, you will need to access the full payload item and then target the data elements you need by directly manipulating the data.

An example of manipulating raw payloads:

*** Settings ***
Documentation     A simple robot to demonstrate manipulating the item payload directly.
Library           RPA.Robocorp.WorkItems
Library           Collections

*** Tasks ***
Demonstrate how to read and write the full payload in the work item
    ${payload}=    Get Work Item Payload
    FOR    ${value}    IN    @{payload}
        Log    Received value: ${value}
    END

    Append to list    ${payload}    And    Friends
    Set Work Item Payload    ${payload}
    Save Work Item
  • In the *** Settings *** section, we have added the standard Robot Framework Collections library, which allows us to manipulate data structures like lists and dictionaries.
  • We are using the Get Work Item Payload keyword to get the whole payload into a variable that we call ${payload}.
  • We can loop over the raw payload using a FOR loop because it is a list instead of a typical dictionary.
  • To add new values to the payload, we can use the Append to list keyword that we got from the Collections library.
  • We can then set the work item payload using the Set Work Item Payload, and finally make the change permanent using the Save Work Item keyword.

Using this method, you can work on payloads with arbitrary structures at the price of adding a bit of complexity: for this reason, if you can create your payload using the default structure, we recommend you do so.

Working with files in the work item

In addition to structured data, the work item can also be used to store and share files. The RPA.Robocorp.WorkItems library provides keywords to add, retrieve, list, and delete work item files. Like all other data stored in the work item, the files will be available to all robot steps added to the same process. Files added to the work item are stored by Control Room and can be inspected using its user interface.

Control Room enforces a size limit of 50 MB per file added to the work item.

Developing with work items locally

When executing a robot in a cloud environment like Control Room, the RPA.Robocorp.WorkItems library will store the work item in the cloud environment, sharing its contents between steps defined in the same process, without any configuration needed.

However, when developing our robot and running it locally, we want the library to store the data in a JSON file and provide the required parameters to simulate the cloud environment.

Create a new file called items.json on your file system, for example, inside the devdata folder of the current robot directory.

Paste this content into your items.json file:

[
  {
    "payload": {
      "message": "Hello World!"
    }
  }
]

Create a devdata/env.json file in your robot structure and paste in the following configuration:

{
  "RPA_WORKITEMS_ADAPTER": "RPA.Robocorp.WorkItems.FileAdapter",
  "RPA_WORKITEMS_PATH": "%{ROBOT_ROOT}/devdata/items.json"
}

Edit the RPA_WORKITEMS_PATH variable to point to the items.json file that you created, if needed. Note that you can use Robot Framework variables inside the path definition.

The library will read these environment variables and use the given files instead of reading work items from Control Room. The variable message will be available to typical keywords such as Get work item variable.

The file devdata/env.json is read automatically by Robocorp Lab and the Robocorp VS Code extension but will not be used when run inside Control Room.

Local files

The library can also read files as part of the defined JSON database.

An example of a work item with files:

[
  {
    "payload": {
      "message": "Hello World!"
    },
    "files": {
      "orders-file": "orders.xlsx"
    }
  }
]

This means that the file orders.xlsx in the same folder as the items.json can be accessed with the name orders-file. Alternatively, it can point to an absolute path on the file system, but that will make it less portable and possibly not work on other devices.

Learn more about the libraries mentioned on this page:

August 31, 2021