How to create UIs for humans to interact with Robocorp robots

Using the RPA.Dialogs library, you can add a user interface to your robots.

The human operator will be able to interact and work together with the robot to complete its tasks. This is especially useful in combination with the Assistant features of Robocorp Cloud and Robocorp App.

On a technical level, the library will start a local webserver and present HTML forms to the user. You can read more about the library on its documentation pages.

Example: Configurable Google Images search robot

Get the code and run this example in your favorite editor on our Portal!

When run, the robot will:

  • present a form to the user to input the search query and to choose a language for the Google image search interface
  • open a web browser
  • search for images according to the search query and the user language interface
  • save the first returned image

The RPA.Dialogs library was added in the 5.2.0 version of the rpaframework, so make sure you are using version 5.2.0 or newer.

Run this robot as an assistant in Robocorp App

Follow these instructions to upload the robot to Robocorp Cloud and set up Robocorp App to execute it locally as a Robot Assistant.

Robot script

*** Settings ***
Documentation     Example robot that allows a human to search for a specific
...               search query in Google Images
Library           RPA.Dialogs
Library           RPA.Browser
Suite Teardown    Close All Browsers

*** Keywords ***
Collect Search Query From User
    Create Form    Search form
    Add Text Input    Search query    search
    &{response}=    Request Response
    [Return]    ${response["search"]}

*** Keywords ***
Search Google Images For Requested Query
    [Arguments]    ${search_query}
    Open Available Browser    https://images.google.com
    Input Text    name:q    ${search_query}
    Submit Form

*** Keywords ***
Collect The First Search Result Image
    Wait Until Element Is Visible    css:div[data-ri="0"]
    Screenshot    css:div[data-ri="0"]
    ...    filename=%{ROBOT_ROOT}${/}output${/}image_from_google.png

*** Tasks ***
Save The First Image For a Search Query Collected From The User
    ${search_query}=    Collect search query from user
    Search Google Images For Requested Query    ${search_query}
    Collect The First Search Result Image

Robot script explained

Settings

*** Settings ***
Documentation     Example robot that allows a human to search for a specific
...               search query in Google Images
Library           RPA.Dialogs
Library           RPA.Browser
Suite Teardown    Close All Browsers

The *** Settings *** section provides short Documentation for the script and imports libraries (Library) that add new keywords for the robot to use. In this case, we will add the RPA.Browser library to control the web browser and the RPA.Dialogs library to build the UI for the user.

Tasks

*** Tasks ***
Save The First Image For a Search Query Collected From The User
    ${search_query}=    Collect search query from user
    Search Google Images For Requested Query    ${search_query}
    Collect The First Search Result Image
  • Save The First Image For a Search Query Collected From The User is the name of the robot's only task.
  • ${search_query}= Collect search query from user: We are creating a variable to hold the search query that we collect from the user using theCollect search query from user keyword.
  • Search Google Images For Requested Query ${search_query}: We pass the value of the variable to the Search Google Images For Requested Query keyword.
  • with the Collect The First Search Result Image we take a screenshot of the first returned image.

Keywords

*** Keywords ***
Collect Search Query From User
    Create Form    Search form
    Add Text Input    Search query    search
    &{response}    Request Response
    [Return]    ${response["search"]}
  • We create a form using the (you guessed it!) Create Form keyword) from the RPA.Dialogs library.
  • We add a new text input field to our form (Add Text Input) setting its label and name.
  • The Request Response keyword allows us to show the form to the user, and to assign the values that the user will enter into the form into the dictionary variable &{response}, which will contain an item with the name that we assigned to our form field (search). The keyword then returns the value.

Here's how our form will look like:

Search form

For more complex forms, you can also specify your form structure in JSON format. Read more on the documentation pages for the library.

*** Keywords ***
Search Google Images For Requested Query
    [Arguments]    ${search_query}
    Open Available Browser    https://images.google.com
    Input Text    name:q    ${search_query}
    Submit Form

This keyword accepts a search query as an argument, opens Google Images search in a browser, and executes the search by submitting the search form.

*** Keywords ***
Collect The First Search Result Image
    Wait Until Element Is Visible    css:div[data-ri="0"]
    Screenshot    css:div[data-ri="0"]
    ... filename=%{ROBOT_ROOT}${/}output${/}image_from_google.png

This keyword will take a screenshot of the first search result image in the Google Images search result page.

Running robot

Dialogs Google Images Search Robot running