Sending key presses to specific windows in X

For a project I wanted to send key presses to a specific X Window, so I dug out my code from "Moving mouse and pressing keys in X using python" and made some modifications.
I also included the code I wrote in "screenshots using Python under Linux" for finding a particular window id from its name.


As an example the following code takes a window name as an argument to the program when run, finds the window id and then sends the key press of F11 (the window I wanted to interact with was a browser window and therefore it will go fullscreen).

#!/usr/bin/python
"""
Issue a keypress event in X to a specific window
"""
from Xlib import XK, display, ext, X, protocol
import sys
import time

def get_window(display, name):
    root_win = display.screen().root
    window_list = [root_win]

    while len(window_list) != 0:
        win = window_list.pop(0)
        if win.get_wm_name() == name:
           return win.id
        children = win.query_tree().children
        if children != None:
            window_list += children

    print 'Unable to find window matching - %s\n' % name
    sys.exit(1)


d=display.Display()

win=get_window(d,sys.argv[1])

keysym=XK.string_to_keysym("F11")
keycode=d.keysym_to_keycode(keysym)

event = protocol.event.KeyPress(
   time = int(time.time()),
   root = d.screen().root,
   window = win,
   same_screen = 0, child = X.NONE,
   root_x = 0, root_y = 0, event_x = 0, event_y = 0,
   state = 0,
   detail = keycode
)
d.send_event(win, event, propagate=True)
d.sync()
event = protocol.event.KeyRelease(
   time = int(time.time()),
   root = d.screen().root,
   window = win,
   same_screen = 0, child = X.NONE,
   root_x = 0, root_y = 0, event_x = 0, event_y = 0,
   state = 0,
   detail = keycode
)
d.send_event(win, event, propagate=True)
d.sync()

While this seemed to work (my Firefox browser window went into fullscreen mode), I did start to see some strange behaviour, such as none of the menus would display either from the toolbar or right clicking in the window. I did notice that clicking on a menu did very briefly flash up what looked like an outline, so I wondered if this could have anything to do with focus and modified my code like so.

#!/usr/bin/python
"""
Issue a keypress event in X to a specific window
"""
from Xlib import XK, display, ext, X, protocol
import sys
import time

def get_window(display, name):
root_win = display.screen().root
window_list = [root_win]

while len(window_list) != 0:
win = window_list.pop(0)
#print win.get_wm_name()
if win.get_wm_name() == name:
return win.id
children = win.query_tree().children
if children != None:
window_list += children

print 'Unable to find window matching - %s\n' % name
sys.exit(1)

d=display.Display()

win=get_window(d,sys.argv[1])

keysym=XK.string_to_keysym("F11")
keycode=d.keysym_to_keycode(keysym)

#store current input focus
currentfocus=d.get_input_focus()

#set input focus to selected window
d.set_input_focus(win, X.RevertToParent,X.CurrentTime)

#send keypress and keyrelease
ext.xtest.fake_input(d, X.KeyPress, keycode)
ext.xtest.fake_input(d, X.KeyRelease, keycode)

#revert focus to original window
d.set_input_focus(currentfocus.focus,X.RevertToParent,X.CurrentTime)

d.sync()
d.close()

This seems to work much better and Firefox still works correctly.

screenshots using Python under Linux

I have been working on porting some code to Linux and required a method to grab a screenshot of a window using python. I could just execute a command line application such as scrot, but I wanted to see if there was an easy way in native python.
Turns out it is fairly straight forward.

First I looked at taking a basic image of the whole screen, then tried to modify it to allow me to grab particular windows.

My first attempt seemed to work but the output image was skewed, this was using pygtk (gtk.gdk).

#!/usr/bin/python
'''
Modified from example at http://ubuntuforums.org/showthread.php?t=448160&p=2681009#post2681009
'''

import gtk.gdk
import os
import time
import Image

def screenGrab():
   root_win = gtk.gdk.get_default_root_window()
   size = root_win.get_size()
   pixbuf = gtk.gdk.Pixbuf(gtk.gdk.COLORSPACE_RGB,False,8,size[0],size[1])
   pixbuf = pixbuf.get_from_drawable(root_win,root_win.get_colormap(),0,0,0,0,size[0],size[1])
   if (pixbuf == None):
     return False
   else:
     width,height=size[0],size[1]
     return Image.fromstring("RGB",(width,height),pixbuf.get_pixels())if __name__ == '__main__':

   screen=screenGrab()
   #screen.show()
   screen.save(os.getcwd()+'/'+str(int(time.time()))+'.png', 'PNG')

This appeared to be using gtk2, was there anyway to use gtk3?

While searching I found a tutorial on PyCairo and this had an example of taking a screenshot using Gdk and using a cairo Image Surface. This seemed to do what I needed.

#!/usr/bin/python

'''
Modified from
ZetCode PyCairo tutorial

This code example takes a screenshot.

Original author: Jan Bodnar
website: zetcode.com
'''

from gi.repository import Gdk
import cairo
import os
import time


def main():
   
    root_win = Gdk.get_default_root_window()

    width = root_win.get_width()
    height = root_win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(root_win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()


Now to modify this to capture a particular window. I can get information on any window using xwininfo, included in this is the xid (window id).
If I have the id of the window I want to capture can use the function GdkX11.X11Window.foreign_new_for_display() to switch to this window instead of the root window before creating the pixbuf and image surface.

#!/usr/bin/python

'''
Modified from
ZetCode PyCairo tutorial

original author: Jan Bodnar
website: zetcode.com

This code example takes a screenshot of particular window, id in hex
supplied as first argument

'''

from gi.repository import Gdk
from gi.repository import GdkX11
import cairo
import os
import time
import sys

def main():
    winid = int(sys.argv[1],16)
    root_win = GdkX11.X11Display.get_default()
    win = GdkX11.X11Window.foreign_new_for_display(root_win,winid)

    width = win.get_width()
    height = win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()


This works well, however I do not want to have to rely on an external application to get the id. Can I supply a window name and do the same thing?

GdkX11.X11Window.foreign_new_for_display() as well as taking a window id as an argument also takes a window object, so I just need to find a way to identify which window. This was not successful, however if I can identify the window by the name I can just pass the id of that window to GdkX11.X11Window.foreign_new_for_display().
This worked and now I have a script that will take a window name and dump an image of it to a png file.

#!/usr/bin/python

'''
Expanded upon code from
ZetCode PyCairo tutorial

original author: Jan Bodnar
website: zetcode.com

This code example takes a screenshot of a particular window from name
passed as argument.

'''

from gi.repository import Gdk
from gi.repository import GdkX11
import Xlib
import Xlib.display
import cairo
import os
import time
import sys

def get_window(name):
    mydisplay = Xlib.display.Display()
    root_win = mydisplay.screen().root
    window_list = [root_win]

    while len(window_list) != 0:
        win = window_list.pop(0)
        #print win.get_wm_name()
        if win.get_wm_name() == name:
           return win.id
        children = win.query_tree().children
        if children != None:
            window_list += children

    print 'Unable to find window matching - %s\n' % name
    sys.exit(1)
    return None

def main():
   
    root_win = GdkX11.X11Display.get_default()
    browser_win = get_window(sys.argv[1])
    win = GdkX11.X11Window.foreign_new_for_display(root_win,browser_win)

    width = win.get_width()
    height = win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()

Moving mouse and pressing keys in X using python

I have just seen a cool piece of code in python which simulates key presses and mouse clicks to automatically play a browser based game. This was on Windows but it got me thinking of how similar keyboard and mouse events could be simulated in Linux using pure python (I could wrap programs such as xvkbd or xdotool, or even write some extensions in C and hook into xlib).

Fortunately someone has already written a python X library (Xlib) which I can utilise. Here are a couple of simple programs to get an idea of how to simulate mouse and keypress events.

First up how to move the mouse. This snippet moves the moves to a set location and clicks the first mouse button (left click for my setup)


#!/usr/bin/python
from Xlib import X, display, ext

d = display.Display()
s = d.screen()
root = s.root
#move pointer to set location
root.warp_pointer(300,300)
d.sync()
#press button 1, for middle mouse button use 2, for opposite button use 3
ext.xtest.fake_input(d, X.ButtonPress,1)
d.sync()
#we want a click so we need to also relese the same button
ext.xtest.fake_input(d, X.ButtonRelease,1)
d.sync()


Now, how to simulate keypresses.

#!/usr/bin/python
from Xlib import XK, display, ext, X

d=display.Display()
#send F1 as in gnome this will bring up help screen, proves it works
keysym=XK.string_to_keysym("F1")
keycode=d.keysym_to_keycode(keysym)
#press key
ext.xtest.fake_input(d, X.KeyPress, keycode)
d.sync()
#remember to release it, otherwise help screen will continue to appear
ext.xtest.fake_input(d, X.KeyRelease, keycode)
d.sync()

Watching X events

I was watching a presentation on Weyland and I wondered about how to see X events as they happen, I seem to recall doing this many years ago supporting customers but have not had a reason to do it recently.

Looking into this, it should be possible with xev.

xev allows you to monitor an existing window, however I wanted to watch events when a program starts up, e.g. possibly diagnose a slow starting application which you suspect to be caused by X.

I believe this can be done by getting xev to monitor you root window (putting it in the background) and then launching the application that you wish to monitor, e.g.

xev -r &
<graphical application>