Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API's support. You can programmatically control a web page, a Windows/Linux/Mac OS X desktop application, or even an iphone or android application running in a simulator or via VNC.
How Sikuli Works
A Sikuli script (.sikuli) is a directory that consists of a Python source file (.py), and all the image files (.png) used by the source file. All images used in a Sikuli script are simply a path to the .png file in the .sikuli bundle. Therefore, the Python source file can also be edited by any text editor.
While saving a script using Sikuli IDE, an extra HTML file is also created in the .sikuli directory so that users can share the scripts on the web easily.
A Sikuli executable script (.skl) is simply a zipped file of all files in the .sikuli directory. When a script is passed to Sikuli IDE as a command line argument, Sikuli IDE recognizes its type by check its filename extension. If a .skl is seen, Sikuli IDE runs it without showing the IDE window. If a .sikuli is seen, Sikuli IDE opens it in a source code editor.
Sikuli IDE edits and runs Sikuli source scripts. Sikuli IDE integrates screen capturing and a custom text editor (SikuliPane) to optimize the usability of writing a Sikuli script. To show embedded images in the SikuliPane, all string literals that ends with ”.png” are replaced by a custom JButton object, ImageButton. If a user adjusts the image pattern’s similarity, a Pattern() is automatically constructed on top of the image.
To execute a Sikuli script, Sikuli IDE creates a org.python.util.PythonInterpreter and automatically passes a few lines of headers (e.g. to import Sikuli’s Jython modules, and to set the path to .sikuli directory) to the interpreter. Once these headers are set, the .py script is simply executed by PythonInterpreter.execfile().
Uncheck All Checkboxes
In this tutorial, we will demonstrate how to use a for loop to interact with multiple instances of a GUI component. Suppose we want to uncheck all the check boxes in a window, such as the Sharing preferences window shown below:
Unfortunately, there is no “uncheck all” function available. The solution? Write a Sikuli Script to look for ALL the checked items and uncheck them automatically. The function needed for this operation is findAll().
First, let’s capture the screenshot image of a checked item.
Then, we can insert the image into the findAll() function.
findAll() searches the entire screen for all the matching visual patterns and returns a list of locations of those similar patterns. This capability allows us to obtain all the checked items are on the screen. Then, we can simply write a for loop in standard Python syntax and call click() on each element in the list.
When this script is executed, Sikuli will find all the items that are currently checked and click on each item one by one in the loop.