How to create UIs for humans to interact with Robocorp robots

Using the RPA.Assistant library, you can add a user interface to your robots.

The human operator will be able to interact and work together with the robot to complete its tasks. This is especially useful in combination with the Assistant features of Control Room and Robocorp Assistant.

On a technical level, the library will present operating system specific dialogs to the user. See RPA.Assistant library documentation pages for more information.

Example: Configurable Google Images search robot

Get the code and run this example in your favorite editor on our Portal!

When run, the robot will:

  • present a dialog to the user to input the search query for the Google image search interface
  • open a web browser
  • search for images according to the search query
  • save the first returned image

rpaframework version 22.0.0 or newer is recommended.
rpaframework-assistant version 2.2.2 or newer is recommended.

Run this robot as an assistant in Robocorp Assistant

Follow these instructions to upload the robot to Control Room and set up Robocorp Assistant to execute it on your local machine.

Robot script

*** Settings *** Documentation Example robot that allows a human to search for a specific ... search query in Google Images. Library RPA.Browser.Selenium Library RPA.Assistant Suite Teardown Close All Browsers *** Tasks *** Save the first image for a search query collected from the user TRY ${search_query}= Collect search query from user Search Google Images ${search_query} Collect the first search result image EXCEPT Capture Page Screenshot %{ROBOT_ARTIFACTS}${/}error.png Fail Checkout the screenshot: error.png END *** Keywords *** Collect search query from user Add text input search label=Search query Add Submit Buttons buttons=Submit default=Submit ${response}= Run dialog ... height=200 RETURN ${response.search} Reject Google Cookies Click Element If Visible xpath://button/div[contains(text(), 'Reject all')] Accept Google Consent Click Element If Visible xpath://button/div[contains(text(), 'Accept all')] Close Google Sign in if shown Click Element If Visible No thanks Search Google Images [Arguments] ${search_query} Open Available Browser https://images.google.com Close Google Sign in if shown Reject Google Cookies Accept Google Consent Input Text name:q ${search_query} Submit Form Collect the first search result image Wait Until Element Is Visible css:div[data-ri="0"] timeout=15 Screenshot css:div[data-ri="0"] ... filename=%{ROBOT_ROOT}${/}output${/}image_from_google.png

Robot script explained

Settings

*** Settings *** Documentation Example robot that allows a human to search for a specific ... search query in Google Images. Library RPA.Browser.Selenium Library RPA.Assistant Suite Teardown Close All Browsers

The *** Settings *** section provides short Documentation for the script and imports libraries (Library) that add new keywords for the robot to use. In this case, we will add the RPA.Browser.Selenium library to control the web browser and the RPA.Assistant library to build the UI for the user.

Tasks

*** Tasks *** Save the first image for a search query collected from the user TRY ${search_query}= Collect search query from user Search Google Images ${search_query} Collect the first search result image EXCEPT Capture Page Screenshot %{ROBOT_ARTIFACTS}${/}error.png Fail Checkout the screenshot: error.png END
  • Save the first image for a search query collected from the user is the name of the robot's only task.
  • ${search_query}= Collect search query from user: We are creating a variable to hold the search query that we collect from the user using theCollect search query from user keyword.
  • Search Google Images ${search_query}: We pass the value of the variable to the Search Google Images keyword.
  • with the Collect the first search result image we take a screenshot of the first returned image.
  • everything is surrounded in a try-catch clause so the robot will take a screenshot in case one of the above keywords fail

Keywords

*** Keywords *** Collect search query from user Add text input search label=Search query Add Submit Buttons buttons=Submit default=Submit ${response}= Run dialog ... height=200 RETURN ${response.search}
  • The dialog is created automatically under the hood by the RPA.Assistant library. You can start adding inputs without creating the dialog implicitly.
  • We add a new text input field to our dialog (Add text input) setting its name and label.
  • The Run dialog keyword allows us to show the dialog to the user, and to assign the values that the user will enter into the dialog into the result object ${response}, which will contain a property with the name that we assigned to our dialog field (search). The keyword then returns the value.

Here's how our dialog will look like:

Search dialog

*** Keywords *** Search Google Images [Arguments] ${search_query} Open Available Browser https://images.google.com Close Google Sign in if shown Reject Google Cookies Accept Google Consent Input Text name:q ${search_query} Submit Form
  • This keyword accepts a search query as an argument, opens Google Images search in a browser, and executes the search by submitting the search form.
  • There are also some additional helper keywords defined to bypass cookies, consent and sign in prompts from Google
*** Keywords *** Collect the first search result image Wait Until Element Is Visible css:div[data-ri="0"] timeout=15 Screenshot css:div[data-ri="0"] ... filename=%{ROBOT_ROOT}${/}output${/}image_from_google.png

This keyword will take a screenshot of the first search result image in the Google Images search result page.

Last edit: May 17, 2023