RPA.JavaAccessBridge
module RPA.JavaAccessBridge
class RPA.JavaAccessBridge.JavaAccessBridge
Java application UI automation library using Java Access Bridge technology.
The library utilizes java-access-bridge-wrapper package to interact with Java UI. Currently only the 64-bit Windows OS is supported.
Inspecting elements
We have built an Assistant for working with Java applicationβs element structure and Java locators. The Assistant provides copy-paste-able locators for each element and also allows testing locators against selected application.
If our tools fail to pick the locator from your target application, there is always the Access Bridge Explorer from Google that enables you to see the raw view. Please note that Access Bridge Explorer repository has been archived on July 27, 2022 and is no longer actively maintained.
The Accessibility Insights for Windows can show element properties if application framework supports Windows UI Automation (UIA), see more at using Accessibility Insights. Then the recommended library would be RPA.Windows library.
Steps to enable
- Enable the Java Access Bridge in Windows
- Set environment variable RC_JAVA_ACCESS_BRIDGE_DLL as an absolute path to WindowsAccessBridge-64.dll. It is also possible to give DLL location as library initialization parameter access_bridge_path.
C:\path\to\java\bin\jabswitch -enable set RC_JAVA_ACCESS_BRIDGE_DLL=C:\path\to\Java\bin\WindowsAccessBridge-64.dll
*** Settings *** Library RPA.JavaAccessBridge access_bridge_path=C:\path\to\Java\bin\WindowsAccessBridge-64.dll
About Java wrapper callbacks and actions
There might be a compatibility issue with callbacks and actions on target Java application. Possible reasons:
- target application is executed with 32-bit Java
- target application does not support callbacks and/or actions
Workaround for this situation is to initialize JavaAccessBridge library with parameter ignore_callbacks=True.
Then applicationβs element information is still accessible and any actions on those elements can be performed
with RPA.Desktop library. Keep in mind that you can still manuall refresh an element with Refresh Element
.
Note. There are still keywords, for example. Call Element Action
, which will cause error if used in this
situation.
Controlling the Java window
Keyword for this purpose is Select Window
. Window selection is based on the title
parameter, which can be
given as a regular expressions to match the correct window. The keyword brings the window into focus and initially
reads windowβs element structure.
Locating elements
To automate actions on the Java application, the robot needs locations to various elements using a feature called Java locators. Locator describes properties of an element.
At the moment library contains basic level support for locators.
The common locator types are name
and role
.
To identify element with more than one property and can be used, for example:
role:push button and name:Clear
To address element within parent element > can be used, for example:
name:Find Purchase Orders > name:NumberField
Some keywords accept element as an parameter in place of locator.
New locator type strict has been added in rpaframework==12.5.0. Currently property values of string type have been evaluated with startsWith which can match several property values. With strict set in the locator string, all locator on the right side of this definition will be matched using strict (equal matching), example:
# without strict, name can be 'Type', 'Type1', 'Type of'... Get Elements role:push button and name:Type # name must be equal to 'Type' Get Elements role:push button and strict:True and name:Type
Keyword Get Elements
has extra parameter strict
, which when set to
True
forces all locator value matches to be strict, example:
# without strict, name can be 'Type', 'Type1', 'Type of'... Get Elements role:push button and name:Type # name must be equal to 'Type' and role must be equal to 'text' Get Elements role:text and name:Type strict=True
About JavaElement object
The JavaElement
was added in rpaframework==12.3.0 for easy access into
ContextNode
objects which have been returned by Get Elements
keyword.
Keyword Get Elements
still returns ContextNode
objects, but with parameter
java_elements=True
the keyword returns JavaElement
objects instead (they
still contain reference to ContextNode
object via node
property, e.g.
JavaObject.node).
Properties and methods included in the JavaElement:
- name: str
- role: str
- description: str
- states: list # list presentation of states (string)
- ancestry: int # you can set the maximum depth based on this
- checked: bool
- selected: bool
- visible: bool
- enabled: bool
- showing: bool
- focusable: bool
- states_string: str
- x: int # left coordinate of the element
- y: int # top coordinate of the element
- width: int
- height: int
- node: ContextNode # original ContextNode
- row: int # table row, -1 if element is not member of table
- col: int # table column, -1 if element is not member of table
- text: str # text content of the element
- column_count: int # table column count
- visible_children: list # visible children elements of this element
- visible_children_count: int
- index_in_parent: int # position in the parent
- click() # method for clicking element center
- type_text() # method for typing text into element (if possible)
Interacting with elements
By default application elements are interacted with Actions supported by the element. Most common example is click action supported by an button element.
But because application and technology support for the actions might be limited, it is also
possible to opt for interaction elements by their coordinates by giving keyword parameter
action=False
if parameter is available.
Examples
robotframework
Python
variable ROBOT_AUTO_KEYWORDS
variable ROBOT_LIBRARY_DOC_FORMAT
variable ROBOT_LIBRARY_SCOPE
method application_refresh
Refresh application element tree
Might be required action after application element structure changes after window refresh.
method call_element_action
Call element action
Parameters
- locator β target element
- action β name of the element action to call
method click_coordinates
Keyword to mouse click at specific coordinates.
Parameters
- x β horizontal coordinate
- y β vertical coordinates
- click_type β default click, see RPA.Desktop for different click options
- delay β how much in seconds to delay after click, defaults to 0.5
method click_element
Click element
Parameters
- locator β element to click
- index β target element index if multiple are returned
- action β call click action on element (default), or use coordinates
- timeout β timeout in seconds to find element
- click_type β default click, see RPA.Desktop for different click options
method click_push_button
Click element of role push button
- Parameters: button_name β name of the button to click
method close_java_window
Close active Java window which has been accessed via ``Select Window` keyword.
method get_element_actions
Get list of possible element actions
- Parameters: locator β target element
method get_element_text
Get element text
Parameters
- locator β target element
- index β target element index if multiple are returned
method get_elements
Get matching elements
Parameters
- locator β elements to get
- java_elements β if True will return elements as
JavaElement
on False will return Java ContextNodes - strict β on True all locator matches need to match exactly, on False will be using startsWith matching on non-integer properties
- Returns: list of ContextNodes or JavaElements
Python example.
Robotframework example.
method get_locator_tree
Return Java locator tree as list of objects.
Mostly relevant object properties are:
- ancestry
- role
- name
- description
- indexInParent
- Returns: list of objects
method get_version_info
Get Java Access Bridge version information
method highlight_element
Highlight an element
Parameters
- locator β element to highlight
- index β target element index if multiple are returned
method list_java_windows
List all available Java windows.
JavaWindow object contains following properties:
- Java process id (pid)
- Java window title
- Java window handle (hwnd)
The pid and title can be used to get control of the Java process by.
- Returns: list of JavaWindow objects
Python example.
Robot Framework example.
method press_keys
Press multiple keys down simultaneously
See Desktop library documentation for supported keys
- Parameters: keys β keys to press
method print_element_tree
Print current element into log and possibly into a file
- Parameters: filename β filepath to save element tree
- Returns: element tree
method print_locator_tree
Print current Java window locator list into log and possibly into a file.
- Parameters: filename β filepath to save locator tree
- Returns: locator tree
method read_table
Return Java table as list of lists (rows containing columns).
Each cell element is represented by JavaElement
class.
Parameters
- locator β locator to match element with type of table
- visible_only β return all the children when this is False
- Returns: list of lists
Example.
method refresh_element
Refresh an element alone.
This will ensure the latest data is available in the targeted element, thus gaining speed when dealing with big apps that wonβt require an entire global refresh. The obtained Java element is returned.
Parameters
- locator β element to refresh
- index β target element index if multiple are returned
- Returns: the Java element found by the passed locator
method select_menu
Select menu by clicking menu elements
Parameters
- menu β name of the menu
- menuitem β name of the menu item
method select_window
Selects Java application window as target for the automation using Java window title.
Parameters
- title β application window title
- bring_foreground β if application is brought to foreground or not
- timeout β selection timeout
method select_window_by_pid
Selects Java application window as target for the automation using Java process ID (pid).
Parameters
- pid β application process id
- bring_foreground β if application is brought to foreground or not
- timeout β selection timeout
method select_window_by_title
Selects Java application window as target for the automation using Java window title.
Parameters
- title β application window title
- bring_foreground β if application is brought to foreground or not
- timeout β selection timeout
method set_display_scale_factor
Override library display scale factor.
Keyword returns previous value.
- Parameters: factor β value for the new display scale factor
- Returns: previous display scale factor value
method set_mouse_position
Set mouse position to element center
- Parameters: element β target element
method shutdown_jab
Call Java Access Bridge process shutdown
method toggle_drop_down
Toggle dropdown action on element
Parameters
- locator β element locator
- index β target element index if multiple are returned
method type_text
Type text into coordinates defined by locator
Parameters
- locator β target element
- text β text to write
- index β target element if multiple are returned
- clear β should element be cleared before typing
- enter β should enter key be pressed after typing
- typing β if True (default) will use Desktop().type_text() if False will use Desktop().press_keys()
method wait_until_element_exists
Wait until element(s) matching the locator are found within
given timeout or raises ElementNotFound
exception.
Parameters
- locator β locator to match element
- timeout β timeout in seconds to find element
- Returns: element(s) if found
method wait_until_element_is_focused
Wait until element is focused
Parameters
- locator β target element
- index β target element index if multiple are returned
- timeout β timeout in seconds to wait, default 0.5 seconds
method wait_until_element_text_contains
Wait until element text contains expected text
Parameters
- locator β target element
- text β element text should contain this
- index β target element index if multiple are returned
- timeout β timeout in seconds to wait, default 0.5 seconds
method wait_until_element_text_equals
Wait until element text equals expected text
Parameters
- locator β target element
- text β element text should match this
- index β target element index if multiple are returned
- timeout β timeout in seconds to wait, default 0.5 seconds