How to create UIs for humans to interact with Robocorp robots

Using the RPA.Dialogs library, you can add a user interface to your robots.

The human operator will be able to interact and work together with the robot to complete its tasks. This is especially useful in combination with the Assistant features of Control Room and Robocorp Assistant.

On a technical level, the library will present operating system specific dialogs to the user. See RPA.Dialogs library documentation pages for more information.

Example: Configurable Google Images search robot

When run, the robot will:

• present a dialog to the user to input the search query for the Google image search interface
• open a web browser
• search for images according to the search query
• save the first returned image

rpaframework version 14.1.1 or newer is recommended.

Run this robot as an assistant in Robocorp Assistant

Follow these instructions to upload the robot to Control Room and set up Robocorp Assistant to execute it on your local machine.

Robot script

*** Settings ***
Documentation       Example robot that allows a human to search for a specific
...                 search query in Google Images.

Library             RPA.Browser.Selenium
Library             RPA.Dialogs

Suite Teardown      Close All Browsers


*** Tasks ***
Save the first image for a search query collected from the user
    ${search_query}=    Collect search query from user
    Search Google Images    ${search_query}
    Collect the first search result image


*** Keywords ***
Collect search query from user
    Add text input    search    label=Search query
    ${response}=    Run dialog
    RETURN    ${response.search}

Accept Google consent
    Click Element    xpath://button/div[contains(text(), 'I agree')]

Search Google Images
    [Arguments]    ${search_query}
    Open Available Browser    https://images.google.com
    Run Keyword And Ignore Error    Accept Google Consent
    Input Text    name:q    ${search_query}
    Submit Form

Collect the first search result image
    Wait Until Element Is Visible    css:div[data-ri="0"]
    Screenshot    css:div[data-ri="0"]
    ...    filename=%{ROBOT_ROOT}${/}output${/}image_from_google.png

Robot script explained

Settings

*** Settings ***
Documentation       Example robot that allows a human to search for a specific
...                 search query in Google Images.

Library             RPA.Browser.Selenium
Library             RPA.Dialogs

Suite Teardown      Close All Browsers

The *** Settings *** section provides short Documentation for the script and imports libraries (Library) that add new keywords for the robot to use. In this case, we will add the RPA.Browser.Selenium library to control the web browser and the RPA.Dialogs library to build the UI for the user.

Tasks

*** Tasks ***
Save the first image for a search query collected from the user
    ${search_query}=    Collect search query from user
    Search Google Images    ${search_query}
    Collect the first search result image

• Save the first image for a search query collected from the user is the name of the robot's only task.
• ${search_query}= Collect search query from user: We are creating a variable to hold the search query that we collect from the user using theCollect search query from user keyword.
• Search Google Images ${search_query}: We pass the value of the variable to the Search Google Images keyword.
• with the Collect the first search result image we take a screenshot of the first returned image.

Keywords

*** Keywords ***
Collect search query from user
    Add text input    search    label=Search query
    ${response}=    Run dialog
    RETURN    ${response.search}

• The dialog is created automatically under the hood by the RPA.Dialogs library. You can start adding inputs without creating the dialog implicitly.
• We add a new text input field to our dialog (Add text input) setting its name and label.
• The Run dialog keyword allows us to show the dialog to the user, and to assign the values that the user will enter into the dialog into the result object ${response}, which will contain a property with the name that we assigned to our dialog field (search). The keyword then returns the value.

Here's how our dialog will look like:

*** Keywords ***
Search Google Images
    [Arguments]    ${search_query}
    Open Available Browser    https://images.google.com
    Run Keyword And Ignore Error    Accept Google Consent
    Input Text    name:q    ${search_query}
    Submit Form

This keyword accepts a search query as an argument, opens Google Images search in a browser, and executes the search by submitting the search form.

*** Keywords ***
Collect the first search result image
    Wait Until Element Is Visible    css:div[data-ri="0"]
    Screenshot    css:div[data-ri="0"]
    ...    filename=%{ROBOT_ROOT}${/}output${/}image_from_google.png

This keyword will take a screenshot of the first search result image in the Google Images search result page.