Headless Sikuli with VNC

04 Jul 2014

VNC can be used to run sikuli scripts on headless machines. We'd like to launch a script on a vncserver without opening vncviewer.

Setup on Arch Linux

First install some dependencies for sikuli and vnc.

$ sudo pacman -S tigervnc gcc opencv tesseract jdk7-openjdk

Download sikuli 1.0.1 and launch the installer

$ java -jar sikuli-setup.jar

Select the 4th option. Set the SIKULIX_HOME environmental variable to the path to sikuli-java.jar.

Install rvm and jruby:

$ \curl -sSL https://get.rvm.io | bash -s stable
$ rvm install jruby

Start VNC Server

$ vncserver

Edit ~/.vnc/xstartup to modify the graphical environment. You should install a minimal stacking window manager like openbox, twm, or fluxbox.

Create a file sikuli-example.rb with the following contents

require 'watir-webdriver'
require 'rukuli'

browser = Watir::Browser.start 'www.google.com'
browser.text_field(:name => 'q').set ''
puts browser.title

screen = Rukuli::Screen.new
screen.type 'yellow'
browser.screenshot.save 'google.png'
browser.close

This will startup a firefox browser and sikuli will type 'yellow' into the search region.

We need to install watir and rukili.

$ rvm use jruby
$ gem install watir-webdriver
$ gem install rukuli

Run Script on VNC

Now we can run our script in vnc. Change the display to match the display you saw when you ran vncserver.

$ DISPLAY=:1 jruby sikuli-example.rb

The script should take a picture of google's home page after sikuli types in yellow.

You can open google.png in your favorite picture viewer.

Jason Gowan