RPA form challenge robot

This robot will solve the form challenge posted at http://rpachallenge.com.

Here's a video introduction to the challenge and this solution:

The challenge consists of downloading an Excel spreadsheet, extracting its data and filling a form on the website with the extracted data ten times.

It sounds pretty simple! However, the catch is that each time the form gets submitted, the fields' position and order is shuffled around, so you need to find a good way to identify the right fields each time the form is displayed.

When run, our robot will:

• download the test Excel file from rpachallenge.com
• collect data from the downloaded Excel file
• start the challenge by clicking on the Start button
• loop through the data and fill the form ten times
• take a screenshot of the results page
• write log files
• close the browser

Robot script

*** Settings ***
Documentation     Robot to solve the first challenge at rpachallenge.com, which consists of
...               filling a form that randomly rearranges itself for ten times, with data
...               taken from a provided Microsoft Excel file.
Library           RPA.Browser.Selenium
Library           RPA.Excel.Files
Library           RPA.HTTP


*** Keywords ***
Get The List Of People From The Excel File
    Open Workbook    challenge.xlsx
    ${table}=    Read Worksheet As Table    header=True
    Close Workbook
    [Return]    ${table}


*** Keywords ***
Fill And Submit The Form
    [Arguments]    ${person}
    Input Text    css:input[ng-reflect-name="labelFirstName"]  ${person}[First Name]
    Input Text    css:input[ng-reflect-name="labelLastName"]  ${person}[Last Name]
    Input Text    css:input[ng-reflect-name="labelCompanyName"]  ${person}[Company Name]
    Input Text    css:input[ng-reflect-name="labelRole"]  ${person}[Role in Company]
    Input Text    css:input[ng-reflect-name="labelAddress"]  ${person}[Address]
    Input Text    css:input[ng-reflect-name="labelEmail"]  ${person}[Email]
    Input Text    css:input[ng-reflect-name="labelPhone"]  ${person}[Phone Number]
    Click Button    Submit


*** Tasks ***
Start The Challenge
    Open Available Browser    http://rpachallenge.com/
    Download  http://rpachallenge.com/assets/downloadFiles/challenge.xlsx    overwrite=True
    Click Button    Start


*** Tasks ***
Fill The Forms
    ${people}=    Get The List Of People From The Excel File
    FOR  ${person}  IN  @{people}
      Fill And Submit The Form  ${person}
    END


*** Tasks ***
Collect The Results
    Capture Element Screenshot    css:div.congratulations
    [Teardown]  Close All Browsers

Robot script explained

Now let's look at what we are telling our robot to do in detail.

Settings section

In this section of the tasks.robot file, we add a description of our robot and reference the libraries that we want to use:

*** Settings ***
Documentation     Robot to solve the first challenge at rpachallenge.com, which consists of
...               filling a form that randomly rearranges itself for ten times, with data
...               taken from a provided Microsoft Excel file.
Library           RPA.Browser.Selenium
Library           RPA.Excel.Files
Library           RPA.HTTP

Our robot uses these three libraries:

• RPA.Browser.Selenium: to interact with the browser
• RPA.Excel.Files: to read the contents of our Excel file
• RPA.HTTP: to download the Excel file from the challenge website.

These libraries are provided by the rpaframework package, a set of well-documented and actively maintained core libraries for Software Robot Developers.

Keywords and Tasks

We are splitting the challenge into three separate Tasks. In our tasks, we use Keywords coming from the libraries we are importing, and also some that we define ourselves.

Let's have a look at each task individually:

Task: Start The Challenge

*** Tasks ***
Start The Challenge
    Open Available Browser    http://rpachallenge.com/
    Download  http://rpachallenge.com/assets/downloadFiles/challenge.xlsx    overwrite=True
    Click Button    Start

Using the Open Available Browser keyword from RPA.Browser.Selenium, the robot will open a new browser and navigate to the challenge website. Then, using the Download keyword from RPA.HTTP, it will download the Excel file locally, overwriting the file if it happens to exist already. Once the file is downloaded, it will start the challenge using the Click Button keyword, provided by the RPA.Browser.Selenium library.

Because there are no other buttons with a label of "Start", we can just pass the label to the Click Button keyword, and it will just work!

Task: Fill The Forms

This task is where most of the work is done. Let's have a look at it in detail:

*** Tasks ***
Fill The Forms
    ${people}=    Get The List Of People From The Excel File
    FOR  ${person}  IN  @{people}
      Fill And Submit The Form  ${person}
    END

1. ${people}= Get The List Of People From The Excel File:

   Here we are creating a variable, and we are assigning to it the Excel data by using a new keyword that we called Get The List Of People From The Excel File:

   *** Keywords ***
   Get The List Of People From The Excel File
       Open Workbook    challenge.xlsx
       ${table}=    Read Worksheet As Table    header=True
       Close Workbook
       [Return]    ${table}
   

   We are using the RPA.Excel.Files library to manipulate the Excel file (Keywords: Open Workbook, Read Worksheet As Table, Close Workbook), returning the data as a Table, so that we can iterate over it in the next step.

2. Looping over the people data:

   Here we get the table rows data @{people}, and we call the Fill And Submit The Form keyword for each row @{person}, using a FOR loop.

   FOR  ${person}  IN  @{people}
     Fill And Submit The Form  ${person}
   END
   

3) Fill And Submit The Form ${person}

Now that we have the data from one individual row, with this keyword we can fill the form on the page.

The RPA.Browser.Selenium library provides a keyword to set an input field (Input Text), but how can we identify each form item? We cannot rely on the order of the fields or their position on the page because, as part of the challenge, they will move around each time we submit the form.

We have to dig deeper to find a reliable way to identify each field. Let's have a look at the source code of the form inputs, for example the "First Name" field:

<label _ngcontent-c2="">First Name</label>
<input
  _ngcontent-c2=""
  ng-reflect-name="labelFirstName"
  id="cu1Yq"
  name="cu1Yq"
  class="ng-pristine ng-invalid ng-touched"
/>

Typically, we would use the id, or name HTML attributes to identify a form field, but the creators of the challenge did not make it easy for us because these attributes change each time the form is submitted. However, we can see that there is another attribute that does not change: ng-reflect-name. Let's use that one!

So, using CSS, our locator for the "First Name" field will become:

css:input[ng-reflect-name="labelFirstName"]

You can learn more about how to find interface elements in web applications and the concept of locators in this dedicated article.

Now with our locators and the Input Text keyword we can fill each field in the form with the corresponding data, and then click the Submit button:

*** Keywords ***
Fill And Submit The Form
     [Arguments]    ${person}
     Input Text    css:input[ng-reflect-name="labelFirstName"]  ${person}[First Name]
     Input Text    css:input[ng-reflect-name="labelLastName"]  ${person}[Last Name]
     Input Text    css:input[ng-reflect-name="labelCompanyName"]  ${person}[Company Name]
     Input Text    css:input[ng-reflect-name="labelRole"]  ${person}[Role in Company]
     Input Text    css:input[ng-reflect-name="labelAddress"]  ${person}[Address]
     Input Text    css:input[ng-reflect-name="labelEmail"]  ${person}[Email]
     Input Text    css:input[ng-reflect-name="labelPhone"]  ${person}[Phone Number]
     Click Button    Submit

Note: the keys in our ${person} dictionary are the column headers from the first row of the Excel file:

Task: Collect The Results

 *** Tasks ***
 Collect The Results
     Capture Element Screenshot    css:div.congratulations
     [Teardown]  Close All Browsers

After all the forms have been filled, the site will display a congratulations message with statistics about the accuracy and how long it took to complete the challenge. Looking at HTML source of the page, we can see that the congratulations message has a CSS class of congratulations: the robot will take a screenshot of it (Capture Element Screenshot) and save it to a file.

At the end, we tell the robot to Close All Browsers.

Job done! (...or is it?)

Right, now let's make it FAST! 🏎🔥🔥🔥

Well, we are Software Robot Developers, and there's nothing like a challenge with a timer to make us want to shave seconds and milliseconds from our programs.

So we cannot just be happy about those 6.7 seconds that our robot took to complete the challenge. It's time to optimize!

The most time-consuming part of our task is the actual filling of the forms. We are using the Input Text keyword, which makes for very readable and understandable code, but we realize it could be faster.

The RPA.Browser.Selenium library allows us to write Javascript directly by using the Execute Javascript keyword. Let's try to fill the form that way!

Our plan is to use the document.evaluate JavaScript method to find the elements, which means we need to identify our input elements using XPath instead of CSS. Using XPath, our locators become:

//input[@ng-reflect-name="labelFirstName"]

After that, we can set the value property on the returned singleNodeValue object to fill the field. Easy!

Now, instead of using the Input Text keyword, we can make our own keyword that will accept the XPath for the element, and the value we want to assign to it, and execute our simple script:

*** Keywords ***
Set Value By Xpath
    [Arguments]    ${xpath}    ${value}
    ${result}=    Execute Javascript    document.evaluate('${xpath}',document.body,null,9,null).singleNodeValue.value='${value}';
    [Return]    ${result}

So, with these changes, our robot code now becomes:

*** Settings ***
Documentation     Robot to solve the first challenge at rpachallenge.com, which consists of
...               filling a form that randomly rearranges itself for ten times, with data
...               taken from a provided Microsoft Excel file.
Library           RPA.Browser.Selenium
Library           RPA.HTTP
Library           RPA.Excel.Files

*** Keywords ***
Get The List Of People From The Excel File
    Open Workbook    challenge.xlsx
    ${table}=    Read Worksheet As Table    header=True
    Close Workbook
    [Return]    ${table}


*** Keywords ***
Set Value By Xpath
    [Arguments]    ${xpath}    ${value}
    ${result}=    Execute Javascript    document.evaluate('${xpath}',document.body,null,9,null).singleNodeValue.value='${value}';
    [Return]    ${result}


*** Keywords ***
Fill And Submit The Form
    [Arguments]    ${person}
    Set Value By Xpath    //input[@ng-reflect-name="labelFirstName"]  ${person}[First Name]
    Set Value By Xpath    //input[@ng-reflect-name="labelLastName"]  ${person}[Last Name]
    Set Value By Xpath    //input[@ng-reflect-name="labelCompanyName"]  ${person}[Company Name]
    Set Value By Xpath    //input[@ng-reflect-name="labelRole"]  ${person}[Role in Company]
    Set Value By Xpath    //input[@ng-reflect-name="labelAddress"]  ${person}[Address]
    Set Value By Xpath    //input[@ng-reflect-name="labelEmail"]  ${person}[Email]
    Set Value By Xpath    //input[@ng-reflect-name="labelPhone"]  ${person}[Phone Number]
    Click Button    Submit


*** Tasks ***
Start The Challenge
    Open Available Browser    http://rpachallenge.com/
    Download  http://rpachallenge.com/assets/downloadFiles/challenge.xlsx    overwrite=True
    Click Button    Start


*** Tasks ***
Fill The Forms
    ${people}=    Get The List Of People From The Excel File
    FOR  ${person}  IN  @{people}
      Fill And Submit The Form  ${person}
    END


*** Tasks ***
Collect The Results
    Capture Element Screenshot    css:div.congratulations
    Close All Browsers

So now, when we run it, our execution time becomes:

Now that's much better!

This fun little exercise shows that when you, as a developer, are in control, you can optimize extensively. In this case, we wanted speed, and we were willing to sacrifice a bit of code readability and slightly deviate from the standard to achieve it. It's your job to decide what's best for your specific implementation. But it's nice to have options, isn't it?

Summary

Using this example robot, you learned some concepts and features of both Robot Framework and RPA Framework:

• Downloading a remote file via the RPA.HTTP library (HTTP GET)
• Reading Excel files and working with the data (RPA.Excel.Files)
• Locating specific elements in a webpage using their label or a custom attribute with XPath
• Filling web forms and clicking buttons (Input Text, Click Button)
• Taking screenshots of elements of a webpage (Capture Element Screenshot)
• Taking alternative strategies to optimize your robot's execution times