RPA form challenge robot
This robot will solve the form challenge posted at http://rpachallenge.com.
Here's a video introduction to the challenge and this solution:
The challenge consists of downloading an Excel spreadsheet, extracting its data and filling a form on the website with the extracted data ten times.
It sounds pretty simple! However, the catch is that each time the form gets submitted, the fields' position and order is shuffled around, so you need to find a good way to identify the right fields each time the form is displayed.
When run, our robot will:
- download the test Excel file from rpachallenge.com
- collect data from the downloaded Excel file
- start the challenge by clicking on the Start button
- loop through the data and fill the form ten times
- take a screenshot of the results page
- write log files
- close the browser
Robot script
*** Settings ***
Documentation Robot to solve the first challenge at rpachallenge.com, which consists of
... filling a form that randomly rearranges itself for ten times, with data
... taken from a provided Microsoft Excel file.
Library RPA.Browser
Library RPA.Excel.Files
Library RPA.HTTP
*** Keywords ***
Get The List Of People From The Excel File
Open Workbook challenge.xlsx
${table}= Read Worksheet As Table header=True
Close Workbook
[Return] ${table}
*** Keywords ***
Fill And Submit The Form
[Arguments] ${person}
Input Text css:input[ng-reflect-name="labelFirstName"] ${person}[First Name]
Input Text css:input[ng-reflect-name="labelLastName"] ${person}[Last Name]
Input Text css:input[ng-reflect-name="labelCompanyName"] ${person}[Company Name]
Input Text css:input[ng-reflect-name="labelRole"] ${person}[Role in Company]
Input Text css:input[ng-reflect-name="labelAddress"] ${person}[Address]
Input Text css:input[ng-reflect-name="labelEmail"] ${person}[Email]
Input Text css:input[ng-reflect-name="labelPhone"] ${person}[Phone Number]
Click Button Submit
*** Tasks ***
Start The Challenge
Open Available Browser http://rpachallenge.com/
Download http://rpachallenge.com/assets/downloadFiles/challenge.xlsx overwrite=True
Click Button Start
*** Tasks ***
Fill The Forms
${people}= Get The List Of People From The Excel File
FOR ${person} IN @{people}
Fill And Submit The Form ${person}
END
*** Tasks ***
Collect The Results
Capture Element Screenshot css:div.congratulations
[Teardown] Close All Browsers
Robot script explained
Now let's look at what we are telling our robot to do in detail.
Settings section
In this section of the tasks.robot
file, we add a description of our robot and reference the libraries that we want to use:
*** Settings ***
Documentation Robot to solve the first challenge at rpachallenge.com, which consists of
... filling a form that randomly rearranges itself for ten times, with data
... taken from a provided Microsoft Excel file.
Library RPA.Browser
Library RPA.Excel.Files
Library RPA.HTTP
Our robot uses these three libraries:
RPA.Browser
: to interact with the browserRPA.Excel.Files
: to read the contents of our Excel fileRPA.HTTP
: to download the Excel file from the challenge website.
These libraries are provided by the rpaframework package, a set of well-documented and actively maintained core libraries for Software Robot Developers.
Keywords and Tasks
We are splitting the challenge into three separate Tasks
. In our tasks, we use Keywords
coming from the libraries we are importing, and also some that we define ourselves.
Let's have a look at each task individually:
Task: Start The Challenge
*** Tasks ***
Start The Challenge
Open Available Browser http://rpachallenge.com/
Download http://rpachallenge.com/assets/downloadFiles/challenge.xlsx overwrite=True
Click Button Start
Using the Open Available Browser
keyword from RPA.Browser
, the robot will open a new browser and navigate to the challenge website. Then, using the Download
keyword from RPA.HTTP
, it will download the Excel file locally, overwriting the file if it happens to exist already. Once the file is downloaded, it will start the challenge using the Click Button
keyword, provided by the RPA.Browser
library.
Because there are no other buttons with a label of "Start", we can just pass the label to the
Click Button
keyword, and it will just work!
Task: Fill The Forms
This task is where most of the work is done. Let's have a look at it in detail:
*** Tasks ***
Fill The Forms
${people}= Get The List Of People From The Excel File
FOR ${person} IN @{people}
Fill And Submit The Form ${person}
END
${people}= Get The List Of People From The Excel File
:Here we are creating a variable, and we are assigning to it the Excel data by using a new keyword that we called
Get The List Of People From The Excel File
:*** Keywords *** Get The List Of People From The Excel File Open Workbook challenge.xlsx ${table}= Read Worksheet As Table header=True Close Workbook [Return] ${table}
We are using the
RPA.Excel.Files
library to manipulate the Excel file (Keywords:Open Workbook
,Read Worksheet As Table
,Close Workbook
), returning the data as a Table, so that we can iterate over it in the next step.Looping over the people data:
Here we get the table rows data
@{people}
, and we call theFill And Submit The Form
keyword for each row@{person}
, using a FOR loop.FOR ${person} IN @{people} Fill And Submit The Form ${person} END
3) Fill And Submit The Form ${person}
Now that we have the data from one individual row, with this keyword we can fill the form on the page.
The RPA.Browser
library provides a keyword to set an input field (Input Text
), but how can we identify each form item? We cannot rely on the order of the fields or their position on the page because, as part of the challenge, they will move around each time we submit the form.
We have to dig deeper to find a reliable way to identify each field. Let's have a look at the source code of the form inputs, for example the "First Name" field:
<label _ngcontent-c2="">First Name</label>
<input
_ngcontent-c2=""
ng-reflect-name="labelFirstName"
id="cu1Yq"
name="cu1Yq"
class="ng-pristine ng-invalid ng-touched"
/>
Typically, we would use the id
, or name
HTML attributes to identify a form field, but the creators of the challenge did not make it easy for us because these attributes change each time the form is submitted. However, we can see that there is another attribute that does not change: ng-reflect-name
. Let's use that one!
So, using CSS, our locator for the "First Name" field will become:
css:input[ng-reflect-name="labelFirstName"]
You can learn more about how to find interface elements in web applications and the concept of locators in this dedicated article.
Now with our locators and the Input Text
keyword we can fill each field in the form with the corresponding data, and then click the Submit
button:
*** Keywords ***
Fill And Submit The Form
[Arguments] ${person}
Input Text css:input[ng-reflect-name="labelFirstName"] ${person}[First Name]
Input Text css:input[ng-reflect-name="labelLastName"] ${person}[Last Name]
Input Text css:input[ng-reflect-name="labelCompanyName"] ${person}[Company Name]
Input Text css:input[ng-reflect-name="labelRole"] ${person}[Role in Company]
Input Text css:input[ng-reflect-name="labelAddress"] ${person}[Address]
Input Text css:input[ng-reflect-name="labelEmail"] ${person}[Email]
Input Text css:input[ng-reflect-name="labelPhone"] ${person}[Phone Number]
Click Button Submit
Note: the keys in our
${person}
dictionary are the column headers from the first row of the Excel file:
Task: Collect The Results
*** Tasks ***
Collect The Results
Capture Element Screenshot css:div.congratulations
[Teardown] Close All Browsers
After all the forms have been filled, the site will display a congratulations message with statistics about the accuracy and how long it took to complete the challenge. Looking at HTML source of the page, we can see that the congratulations message has a CSS class of congratulations
: the robot will take a screenshot of it (Capture Element Screenshot
) and save it to a file.
At the end, we tell the robot to Close All Browsers
.
Job done! (...or is it?)
Right, now let's make it FAST! 🏎🔥🔥🔥
Well, we are Software Robot Developers, and there's nothing like a challenge with a timer to make us want to shave seconds and milliseconds from our programs.
So we cannot just be happy about those 6.7 seconds that our robot took to complete the challenge. It's time to optimize!
The most time-consuming part of our task is the actual filling of the forms. We are using the Input Text
keyword, which makes for very readable and understandable code, but we realize it could be faster.
The RPA.Browser
library allows us to write Javascript directly by using the Execute Javascript
keyword. Let's try to fill the form that way!
Our plan is to use the document.evaluate
JavaScript method to find the elements, which means we need to identify our input elements using XPath instead of CSS. Using XPath, our locators become:
//input[@ng-reflect-name="labelFirstName"]
After that, we can set the value
property on the returned singleNodeValue
object to fill the field. Easy!
Now, instead of using the Input Text
keyword, we can make our own keyword that will accept the XPath for the element, and the value we want to assign to it, and execute our simple script:
*** Keywords ***
Set Value By Xpath
[Arguments] ${xpath} ${value}
${result}= Execute Javascript document.evaluate('${xpath}',document.body,null,9,null).singleNodeValue.value='${value}';
[Return] ${result}
So, with these changes, our robot code now becomes:
*** Settings ***
Documentation Robot to solve the first challenge at rpachallenge.com, which consists of
... filling a form that randomly rearranges itself for ten times, with data
... taken from a provided Microsoft Excel file.
Library RPA.Browser
Library RPA.HTTP
Library RPA.Excel.Files
*** Keywords ***
Get The List Of People From The Excel File
Open Workbook challenge.xlsx
${table}= Read Worksheet As Table header=True
Close Workbook
[Return] ${table}
*** Keywords ***
Set Value By Xpath
[Arguments] ${xpath} ${value}
${result}= Execute Javascript document.evaluate('${xpath}',document.body,null,9,null).singleNodeValue.value='${value}';
[Return] ${result}
*** Keywords ***
Fill And Submit The Form
[Arguments] ${person}
Set Value By Xpath //input[@ng-reflect-name="labelFirstName"] ${person}[First Name]
Set Value By Xpath //input[@ng-reflect-name="labelLastName"] ${person}[Last Name]
Set Value By Xpath //input[@ng-reflect-name="labelCompanyName"] ${person}[Company Name]
Set Value By Xpath //input[@ng-reflect-name="labelRole"] ${person}[Role in Company]
Set Value By Xpath //input[@ng-reflect-name="labelAddress"] ${person}[Address]
Set Value By Xpath //input[@ng-reflect-name="labelEmail"] ${person}[Email]
Set Value By Xpath //input[@ng-reflect-name="labelPhone"] ${person}[Phone Number]
Click Button Submit
*** Tasks ***
Start The Challenge
Open Available Browser http://rpachallenge.com/
Download http://rpachallenge.com/assets/downloadFiles/challenge.xlsx overwrite=True
Click Button Start
*** Tasks ***
Fill The Forms
${people}= Get The List Of People From The Excel File
FOR ${person} IN @{people}
Fill And Submit The Form ${person}
END
*** Tasks ***
Collect The Results
Capture Element Screenshot css:div.congratulations
Close All Browsers
So now, when we run it, our execution time becomes:
Now that's much better!
This fun little exercise shows that when you, as a developer, are in control, you can optimize extensively. In this case, we wanted speed, and we were willing to sacrifice a bit of code readability and slightly deviate from the standard to achieve it. It's your job to decide what's best for your specific implementation. But it's nice to have options, isn't it?
Summary
Using this example robot, you learned some concepts and features of both Robot Framework and RPA Framework:
- Downloading a remote file via the RPA.HTTP library (
HTTP GET
) - Reading Excel files and working with the data (
RPA.Excel.Files
) - Locating specific elements in a webpage using their label or a custom attribute with XPath
- Filling web forms and clicking buttons (
Input Text
,Click Button
) - Taking screenshots of elements of a webpage (
Capture Element Screenshot
) - Taking alternative strategies to optimize your robot's execution times