I came across Sikuli today, a MIT User Interface Design Group project that is just awesome and allows to automate or script your computer based on screenshots.

Sikuli is a visual technology to search and automate graphical user interfaces (GUI) using images (screenshots). The first release of Sikuli contains Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API’s support. You can programmatically control a web page, a desktop application running on Windows/Linux/Mac OS X, or even an iphone application running in an emulator.

What you do ist basically tell the program which region on the GUI it should click or move or rotate bei making screenshots of the region. Sikuli even let’s you automate the fill in of forms (e.g. IP adresses in the network preferences dialog).

See the demo video they put online for a detailed inspection and give the tool a try. This is really cool…