module RPA.Windows

class RPA.Windows.Windows

The Windows is a library that can be used for Windows desktop automation.

Library is included in the rpaframework package by default, but as shown in the below example library can be also installed separately without rpaframework.

About terminology

ControlType is a value referred to by locator keys type: or control. Represents type of application object, which can be e.g. Window, Button or ListItem.

Element is an entity of an application structure (e.g. certain button in a window), which can be identified by a locator. (also referred as Control)

WindowsElement is an library container object for the Element. All the keywords returning elements, will in fact return WindowsElement``s. The ones accepting ``locator or root_element as arguments, will accept WindowsElement as an argument value. (locator accepts strings as well)

Structure of the WindowsElement

Example of the WindowsElement usage

Locators

Locators are based on different strategies that can used identify Control object.

Available strategies that can be used for the locators:

About root element on locators

Locators work on currently active root element. At the start root element is the whole desktop. There are different ways on changing this root element.

Keyword Control Window is the most common method of setting certain system window as a root element for further actions using locators. In the absence of a provided root_element parameter, here’s how you can control the default root element resolving:

• Set Anchor: Sets the active anchor window from which the search begins.
• Control Window: Controls and focuses on a window and marks it as the current active window, from which all the subsequent searches will start from in the absence of a set anchor.
• If there’s no set anchor nor active window, then the last resort will be the “Desktop” element itself.

Locators themselves support cascading syntax (denoted by character > in the locator string), which can denote root element in “parent (root) & child” terms.

For example.

On the above example the left side of the > character, id:controls, represents the root element (can be called as “parent element” in this case). Right side of the locator string, id:activate, represents “child” element and it will be searched under the “parent element”.

This way element search is more efficient, because search are restricted to certain section of element tree which can be quite huge especially on the desktop level and in certain applications.

Keyword examples:

some example locators, and can be omitted ie. space ` ` between locator keys means the same thing as and:

About locator restrictions

Visual locators are not supported in this library and they can’t be used in the same chain with these Windows locators. Visual locators are supported by the RPA.Desktop library. Locator chaining (image and Windows locators) support will be added in the future.

Locator syntax does not yet support OR operation (only AND operations).

About search depth

The library does element search depth by default to the level of 8. This means that locator will look into 8 levels of elements under element tree of the root element. This can lead into situation where element can’t be found. To fix this it is recommended to set root element which can be found within 8 levels OR defining depth in the locator string to a bigger value, e.g. id:deeplyNestedButton depth:16. Useful keywords for setting root element are Control Window, Set Anchor and Get Element.

About the path strategy

When automation IDs and names aren’t enough (or not reliable), then you can fallback to the positions of elements in a tree. This can be achieved using the path: strategy to specify a list of element positions which indicates how to traverse the tree from parent to child beginning with the resolved root.

Examples

An alternative way to get the whole tree to explore it yourself would be to use the Print Tree keyword.

Keyboard and mouse

Keys for the keyboard actions are given using uiautomation specification.

Special keys which are given within {} syntax.

Examples.

Using access key of the element (element property -> AccessKey ‘alt+s’). The (+s) means that previous special key is kept down until closing parenthesis is reached.

On the below example this means that ‘ALT’ key is pressed down, then ‘+’ and ‘s’ keys are pressed down before they are all released up.

Mouse clicks can be executed with keywords specific for a type of a click, e.g. Click (normal click), Double Click and Right Click.

How to inspect

Most common, and recommended by Microsoft, inspector tool for Windows is Accessibility Insights that can be installed separately. Other options are tools Inspect Object and UI Automation Verify, which can be accessed by installing Windows SDK.

A more programmatic approach is to run Print Tree log_as_warnings=${True} keyword and then observe in the logs the found elements structure starting from Desktop (or the currently set anchor / active window) as root. (refer to keyword’s documentation for more details)

Recording

The package provides some rudimentary inspecting and recording via the windows-record script, which can be started through the command line (in an environment containing the rpaframework-windows installation).

Recording inspects elements on mouse click and can be stopped by pressing the ESC key. Expected console output:

Check our Portal example in order to learn more abot the path: strategy in locators and how to record elements displaying their paths: https://robocorp.com/portal/robot/robocorp/example-windows-element-path

Video recorded demo on how to run the recorder script from VSCode: https://www.loom.com/share/2807372359f34b9cbe1bc2df9194ec68

Caveats

• Make sure your display scaling is set to 100%, otherwise you might encounter issues when clicking or interacting with elements. (since offsets and coordinates get distorted)
• Disturbing the automation (like interacting with your mouse/keyboard) or having other apps obstructing the process interacting with your app of interest will most probably affect the expected behaviour. In order to avoid this, try controlling the app’s main window right before sending clicks or keys. And keep targeting elements through string locators, as interacting with Windows element objects previously retrieved will not work as expected in a future altered state of the app (changes under the element structure).

Example: Robot Framework

The library must be imported first.

Windows Calculator automation task

Example: Python

variable ROBOT_LIBRARY_DOC_FORMAT

variable ROBOT_LIBRARY_SCOPE

variable SIMULATE_MOVE

method add_library_components

method clear_anchor

Clears control anchor set by Set Anchor

This means that all following keywords accessing elements will use active window or desktop as root element.

method click

Mouse click on element matching given locator.

Exception ActionNotPossible is raised if element does not allow Click action.

Parameters

• locator – String locator or element object.
• wait_time – time to wait after click, default is a library wait_time, see keyword Set Wait Time
• timeout – float value in seconds, see keyword Set Global Timeout

• Returns: WindowsElement object

method close_current_window

Closes current active window or logs a warning message.

• Returns: True if close was successful, False if not

method close_window

Closes identified windows or logs the problems.

Parameters

• locator – String locator or Control element.
• timeout – float value in seconds, see keyword Set Global Timeout

• Returns: How many windows were found and closed.

method control_child_window

Get control of child window of the active window by locator.

Parameters

• locator – string locator or Control element
• foreground – True to bring window to foreground
• wait_time – time to wait after activeting a window
• timeout – float value in seconds, see keyword Set Global Timeout

• Returns: WindowsElement object

method control_window

Controls the window defined by the locator.

This means that this window is used as a root element for all the following keywords using locators.

Returns WindowsElement.

Parameters

• locator – string locator or Control element
• foreground – True to bring window to foreground
• wait_time – time to wait after activating a window
• timeout – float value in seconds, see keyword Set Global Timeout
• main – on True (default) starts the search from desktop level, on False it will continue to search for child elements given the set anchor or current active window

• Returns: WindowsElement object

method disable_process_listing

Disables process iteration in List Windows keyword.

method double_click

Double mouse click on element matching given locator.

Exception ActionNotPossible is raised if element does not allow Click action.

Parameters

• locator – String locator or element object.
• wait_time – time to wait after click, default is a library wait_time, see keyword Set Wait Time
• timeout – float value in seconds, see keyword Set Global Timeout

• Returns: WindowsElement object

method drag_and_drop

Drag and drop the source element into target element.

Parameters

• source – source element for the operation
• target – target element for the operation
• speed – adjust speed of operation, bigger value means more speed
• copy – on True does copy drag and drop, defaults to move
• wait_time – time to wait after drop, default 1.0 seconds

method enable_process_listing

Enables process iteration in List Windows keyword.

method foreground_window

Bring the current active window or the window defined by the locator to the foreground.

• Parameters: locator – string locator or Control element
• Returns: WindowsElement object

method get_attribute

Get attribute value of the element defined by the locator.

Parameters

• locator – string locator or Control element
• attribute – name of the attribute to get

• Returns: value of attribute

method get_element

Get a Control Windows element defined by the locator.

The returned element can be used instead of a locator string for other keywords accepting the locator parameter. Keyword Get Attribute can be used to read element attribute values.

If locator is None, then the returned element will be in this priority:

1. root_element if provided.
2. Anchor element if that has been previously set with Set Anchor.
3. Current active window if that has been set with Control Window.
4. Last resort is the “Desktop” element.

Parameters

• locator – Locator as a string or as an element object.
• search_depth – How deep the element search will traverse. (default 8)
• root_element – Will be used as search root element object if provided.
• timeout – After how many seconds (float) to give up on search. (see keyword Set Global Timeout)

• Returns: The identified WindowsElement object.

Example: Robot Framework

Example: Python

method get_elements

Get a list of elements matching the locator.

By default, only the siblings (similar elements on the same level) are taken into account. In order to search globally, turn siblings_only off, but be aware that this will take more time to process.

Note that if the syntax parent_locator > child_locator is used in the locator, it is assumed that parent_locator returns a singular element - i.e. if the locator parent_locator returns multiple elements, only the first result is used for further processing, even if siblings_only is off.

For more details on the rest of parameters, take a look at the Get Element keyword.

Parameters

• locator – Locator as a string or as an element object.
• search_depth – How deep the element search will traverse. (default 8)
• root_element – Will be used as search root element object if provided.
• timeout – After how many seconds (float) to give up on search. (see keyword Set Global Timeout)
• siblings_only – Filter for elements on the same level as the initially found one. Turn it off for a global search. (True by default)

• Returns: A list of matching WindowsElement objects.

Example: Robot Framework

method get_keyword_arguments

method get_keyword_documentation

method get_keyword_names

method get_keyword_source

method get_keyword_tags

method get_keyword_types

method get_os_version

Returns the current Windows major version as string.

method get_text

Get text from Control element defined by the locator.

Exception ActionNotPossible is raised if element does not allow GetWindowText action.

• Parameters: locator – String locator or element object.
• Returns: value of WindowText attribute of an element

method get_value

Get the value of the element defined by the provided locator.

The ActionNotPossible exception is raised if the identified element doesn’t support value retrieval.

• Parameters: locator – String locator or element object.
• Returns: Optionally the value of the identified element.

Example: Robot Framework

Example: Python

method list_attributes

List all element attributes.

• Parameters: locator – string locator or Control element
• Returns: list of element attributes (strings)

method list_windows

List all window element on the system.

Parameters

• icons – on True dictionary will contain Base64 string of the icon, default False
• icon_save_directory – if set will save retrieved icons into this filepath, by default icon files are not saved

• Returns: list of dictionaries containing information about Window elements

method maximize_window

Maximize the current active window or the window defined by the locator.

• Parameters: locator – string locator or element
• Returns: WindowsElement object

method middle_click

Right mouse click on element matching given locator.

Exception ActionNotPossible is raised if element does not allow Click action.

Parameters

• locator – String locator or element object.
• wait_time – time to wait after click, default is a library wait_time, see keyword Set Wait Time
• timeout – float value in seconds, see keyword Set Global Timeout

• Returns: WindowsElement object

method minimize_window

Minimize the current active window or the window defined by the locator.

• Parameters: locator – string locator or element
• Returns: WindowsElement object

method print_tree

Print a tree of control elements.

A Windows application structure can contain multilevel element structure. Understanding this structure is crucial for creating locators. (based on controls’ details and their parent-child relationship)

This keyword can be used to output logs of application’s element structure, starting with the element defined by the provided locator as root. Switch the return_structure parameter to True to get a tree of elements returned as well. (off by default to save memory)

• The printed structure displays a tree prefixed with “depth” - “position” so you know how deep (0 means root) in the tree you are and on what position (1-indexed) the child you’re looking for is.
• The returned structure is a dictionary with all the children flattened per depth level. Additionally, these WindowsElement objects contain a relevant locator composed of “root > path” which will help to identify the element in the tree.

Portal example: https://robocorp.com/portal/robot/robocorp/example-windows-element-path

Parameters

• locator – The root of the tree to output.
• max_depth – Maximum depth level. (defaults to 8)
• capture_image_folder – If set, controls’ images will be captured in this path.
• log_as_warnings – Enables highlighted logs (at the beginning of the log file as warnings) and increases visibility in the output console.
• return_structure – A flattened tree with all the elements collated by level will be returned if this is enabled.

• Returns: Optionally a dictionary of children per depth level when return_structure is enabled.

Example: Robot Framework

Example: Python

method restore_window

Window restore the current active window or the window defined by the locator.

• Parameters: locator – string locator or element
• Returns: WindowsElement object

method right_click

Right mouse click on element matching given locator.

Exception ActionNotPossible is raised if element does not allow Click action.

Parameters

• locator – String locator or element object.
• wait_time – time to wait after click, default is a library wait_time, see keyword Set Wait Time
• timeout – float value in seconds, see keyword Set Global Timeout

• Returns: WindowsElement object

method run_keyword

method screenshot

Take a screenshot of the element defined by the locator.

An ActionNotPossible exception is raised if the element doesn’t allow being captured.

Parameters

• locator – String locator or element object.
• filename – Image file name/path. (can be absolute/relative)

• Raises: ActionNotPossible – When the element can’t be captured.
• Returns: Absolute file path of the taken screenshot image.

Example: Robot Framework

Example: Python

method select

Select a value on the passed element if such action is supported.

The ActionNotPossible exception is raised when the element does not allow the Select action. This is usually used with combo box elements.

Parameters

• locator – String locator or element object.
• value – String value to select on Control element

• Returns: The controlled Windows element.

Example: Robot Framework

* Settings * Library RPA.Windows

* Tasks * Set Notepad Size

Select id:FontSizeComboBox 22

Example: Python

method send_keys

Send keys to desktop, current window or to Control element defined by given locator.

If locator is None then keys are sent to desktop.

Exception ActionNotPossible is raised if element does not allow SendKeys action.

Parameters

• locator – Optional string locator or element object.
• keys – The keys to send.
• interval – Time between each sent key. (defaults to 0.01 seconds)
• wait_time – Time to wait after sending all the keys. (defaults to library’s set value, see keyword Set Wait Time)
• send_enter – If True then the {Enter} key is pressed at the end of the sent keys.

• Returns: The element identified through locator.

method set_anchor

Set anchor to an element specified by the locator.

All following keywords using locators will use this element as a root element. Specific use case could be setting anchor to TableControl element and then getting column data belonging to that TableControl element.

To release anchor call Clear Anchor keyword.

Parameters

• locator – string locator or Control element
• timeout – timeout in seconds for element lookup (default 10.0)

method set_focus

Set view focus to the element defined by the locator.

• Parameters: locator – String locator or element object.

method set_global_timeout

Set global timeout for element search. Applies also to Control Window keyword.

By default, the library has a timeout of 10 seconds.

• Parameters: timeout – float value in seconds
• Returns: previous timeout value

method set_mouse_movement

Enable or disable mouse movement simulation during clicks and other actions.

Returns the previous set value as True/False.

• Parameters: simulate – Decide whether to simulate the move. (OFF by default)
• Returns: Previous state.

Example: Robot Framework

Example: Python

method set_value

Set value of the element defined by the locator.

Note: An anchor will work only on element structures where you can rely on the stability of that root/child element tree, as remaining the same. Usually these kind of structures are tables. (but not restricted to)

Note: It is important to set append=${True} if you want to keep the current text in the element. Other option is to read the current text into a variable, then modify that value as you wish and pass it to the Set Value keyword for a complete text replacement. (without setting the append flag)

The following exceptions may be raised:

• ActionNotPossible if the element does not allow the SetValue action to be run on it nor having send_keys_fallback=${True}.
• ValueError if the new value to be set can’t be set correctly.

Parameters

• locator – String locator or element object.
• value – String value to be set.
• append – False for setting the value, True for appending it. (OFF by default)
• enter – Set it to True to press the Enter key at the end of the input. (nothing is pressed by default)
• newline – Set it to True to add a new line at the end of the value. (no EOL included by default; this won’t work with send_keys_fallback enabled)
• send_keys_fallback – Tries to set the value by sending it through keys if the main way of setting it fails. (enabled by default)
• validator – Function receiving two parameters post-setting, the expected and the current value, which returns True if the two values match. (by default, the keyword will raise if the values are different, set this to None to disable validation or pass your custom function instead)

• Returns: The element object identified through the passed locator.

Example: Robot Framework

Example: Python

method set_wait_time

Set library wait time for action keywords.

The wait_time is spent after each keyword performing mouse or keyboard action.

Library default wait_time is 0.5

Returns value of the previous wait_time value.

• Parameters: wait_time – float value (in seconds), e.g. 0.1
• Returns: previous wait value

method windows_run

Use Windows Run window to launch an application.

Activated by pressing Win + R. Then the app name is typed in and finally the “Enter” key is pressed.

Parameters

• text – Text to enter into the Run input field. (e.g. Notepad)
• wait_time – Time to sleep after the searched app is executed. (3s by default)

Example: Robot Framework

Example: Python

method windows_search

Use Windows search window to launch application.

Activated by pressing win + s.

Parameters

• text – text to enter into search input field
• wait_time – sleep time after search has been entered (default 3.0 seconds)