Pretty print JSON

I really need to spend some time looking at things like JSON, however I came across a quick way to pretty print JSON, using json.tool python module.

example from top of script (or from help once imported json.tool module)

$ echo '{"json":"obj"}' | python -m json.tool
{
"json":"obj"
}

Sending key presses to specific windows in X

For a project I wanted to send key presses to a specific X Window, so I dug out my code from "Moving mouse and pressing keys in X using python" and made some modifications.
I also included the code I wrote in "screenshots using Python under Linux" for finding a particular window id from its name.


As an example the following code takes a window name as an argument to the program when run, finds the window id and then sends the key press of F11 (the window I wanted to interact with was a browser window and therefore it will go fullscreen).

#!/usr/bin/python
"""
Issue a keypress event in X to a specific window
"""
from Xlib import XK, display, ext, X, protocol
import sys
import time

def get_window(display, name):
    root_win = display.screen().root
    window_list = [root_win]

    while len(window_list) != 0:
        win = window_list.pop(0)
        if win.get_wm_name() == name:
           return win.id
        children = win.query_tree().children
        if children != None:
            window_list += children

    print 'Unable to find window matching - %s\n' % name
    sys.exit(1)


d=display.Display()

win=get_window(d,sys.argv[1])

keysym=XK.string_to_keysym("F11")
keycode=d.keysym_to_keycode(keysym)

event = protocol.event.KeyPress(
   time = int(time.time()),
   root = d.screen().root,
   window = win,
   same_screen = 0, child = X.NONE,
   root_x = 0, root_y = 0, event_x = 0, event_y = 0,
   state = 0,
   detail = keycode
)
d.send_event(win, event, propagate=True)
d.sync()
event = protocol.event.KeyRelease(
   time = int(time.time()),
   root = d.screen().root,
   window = win,
   same_screen = 0, child = X.NONE,
   root_x = 0, root_y = 0, event_x = 0, event_y = 0,
   state = 0,
   detail = keycode
)
d.send_event(win, event, propagate=True)
d.sync()

While this seemed to work (my Firefox browser window went into fullscreen mode), I did start to see some strange behaviour, such as none of the menus would display either from the toolbar or right clicking in the window. I did notice that clicking on a menu did very briefly flash up what looked like an outline, so I wondered if this could have anything to do with focus and modified my code like so.

#!/usr/bin/python
"""
Issue a keypress event in X to a specific window
"""
from Xlib import XK, display, ext, X, protocol
import sys
import time

def get_window(display, name):
root_win = display.screen().root
window_list = [root_win]

while len(window_list) != 0:
win = window_list.pop(0)
#print win.get_wm_name()
if win.get_wm_name() == name:
return win.id
children = win.query_tree().children
if children != None:
window_list += children

print 'Unable to find window matching - %s\n' % name
sys.exit(1)

d=display.Display()

win=get_window(d,sys.argv[1])

keysym=XK.string_to_keysym("F11")
keycode=d.keysym_to_keycode(keysym)

#store current input focus
currentfocus=d.get_input_focus()

#set input focus to selected window
d.set_input_focus(win, X.RevertToParent,X.CurrentTime)

#send keypress and keyrelease
ext.xtest.fake_input(d, X.KeyPress, keycode)
ext.xtest.fake_input(d, X.KeyRelease, keycode)

#revert focus to original window
d.set_input_focus(currentfocus.focus,X.RevertToParent,X.CurrentTime)

d.sync()
d.close()

This seems to work much better and Firefox still works correctly.

python and numbers 08 and 09

I had noticed that the syntax highlighting in vim did not seem to pick up the numbers 08 and 09 in some code I was writing. However, it was only when running some code containing these numbers that I appreciated what was happening.

If numbers have been represented with a leading 0 then these are interpreted as octal numbers and therefore each digit after can only go up to 7, therefore 08 and 09 are invalid.

This can be seen if you try to run the following code

number = 08
print number

You get an error similar to

  File "number", line 4
    number=08
            ^
SyntaxError: invalid token

Random Programming challenges

I have found some interesting websites that set interesting problems to challenge your programming skills and as I have been messing round more with python I have decided to have a go at some of them using python.

A particularly interesting one is Project Euler where I have just achieved the award Decathlete for solving the first ten problems. I suspect the rest will be much harder.

Another challenging set of problems is at http://escape.alf.nu/ this guides you through some common pitfalls in javascript programming. The aim is to get the sample code to call alert(1).

Rookie mistake python loops and variable scope

While writing some simple python code I stumbled upon a rookie mistake which I appear to still be making so writing it down here to remind me.

Scope of variables is important in loops especially when using them to increment.

I was writing some code to work out if the product of two 3 digit numbers is a palindrome (not very elegant algorithm just brute forces it). Here is my original algorithm

a=100
b=100

numbers=[]

while a < 1000:
  while b < 1000:
    product=a*b
    print a, b, product
    if (product == int(str(product)[::-1])):
        print "palindrome"
        numbers.append(product)
    b+=1
  a+=1

From the output I noticed something strange the outer loop only went round once even though I thought I had incremented a after the inner loop.

Then it struck me I had defined b outside my while loop so it would not get reset upon next iteration of the outer loop and therefore the inner loop would not run again (and print something out). Therefore it looked like I was only looping once on the outer loop.
This could be proven by adding "print a" before incrementing a on the last line.

Moving "b=100" inside the outer loop solves the problem.

Working code looks like so

a=100
numbers=[]
while a < 1000:
  b=100
  while b < 1000:
    product=a*b
    print a, b, product
    if (product == int(str(product)[::-1])):
        print "palindrome"
        numbers.append(product)
    b+=1
  a+=1

Another alternative is to change the while loops to for loops.

python module is in which file on the filesystem

I wanted a quick way to find out which file contains certain python modules, this can by done by looking at __file__ variable for a module.

e.g. in the interpreter
>>> import pygame
>>> print pygame.__file__
/usr/lib64/python2.7/site-packages/pygame/__init__.pyc

 

screenshots using Python under Linux

I have been working on porting some code to Linux and required a method to grab a screenshot of a window using python. I could just execute a command line application such as scrot, but I wanted to see if there was an easy way in native python.
Turns out it is fairly straight forward.

First I looked at taking a basic image of the whole screen, then tried to modify it to allow me to grab particular windows.

My first attempt seemed to work but the output image was skewed, this was using pygtk (gtk.gdk).

#!/usr/bin/python
'''
Modified from example at http://ubuntuforums.org/showthread.php?t=448160&p=2681009#post2681009
'''

import gtk.gdk
import os
import time
import Image

def screenGrab():
   root_win = gtk.gdk.get_default_root_window()
   size = root_win.get_size()
   pixbuf = gtk.gdk.Pixbuf(gtk.gdk.COLORSPACE_RGB,False,8,size[0],size[1])
   pixbuf = pixbuf.get_from_drawable(root_win,root_win.get_colormap(),0,0,0,0,size[0],size[1])
   if (pixbuf == None):
     return False
   else:
     width,height=size[0],size[1]
     return Image.fromstring("RGB",(width,height),pixbuf.get_pixels())if __name__ == '__main__':

   screen=screenGrab()
   #screen.show()
   screen.save(os.getcwd()+'/'+str(int(time.time()))+'.png', 'PNG')

This appeared to be using gtk2, was there anyway to use gtk3?

While searching I found a tutorial on PyCairo and this had an example of taking a screenshot using Gdk and using a cairo Image Surface. This seemed to do what I needed.

#!/usr/bin/python

'''
Modified from
ZetCode PyCairo tutorial

This code example takes a screenshot.

Original author: Jan Bodnar
website: zetcode.com
'''

from gi.repository import Gdk
import cairo
import os
import time


def main():
   
    root_win = Gdk.get_default_root_window()

    width = root_win.get_width()
    height = root_win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(root_win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()


Now to modify this to capture a particular window. I can get information on any window using xwininfo, included in this is the xid (window id).
If I have the id of the window I want to capture can use the function GdkX11.X11Window.foreign_new_for_display() to switch to this window instead of the root window before creating the pixbuf and image surface.

#!/usr/bin/python

'''
Modified from
ZetCode PyCairo tutorial

original author: Jan Bodnar
website: zetcode.com

This code example takes a screenshot of particular window, id in hex
supplied as first argument

'''

from gi.repository import Gdk
from gi.repository import GdkX11
import cairo
import os
import time
import sys

def main():
    winid = int(sys.argv[1],16)
    root_win = GdkX11.X11Display.get_default()
    win = GdkX11.X11Window.foreign_new_for_display(root_win,winid)

    width = win.get_width()
    height = win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()


This works well, however I do not want to have to rely on an external application to get the id. Can I supply a window name and do the same thing?

GdkX11.X11Window.foreign_new_for_display() as well as taking a window id as an argument also takes a window object, so I just need to find a way to identify which window. This was not successful, however if I can identify the window by the name I can just pass the id of that window to GdkX11.X11Window.foreign_new_for_display().
This worked and now I have a script that will take a window name and dump an image of it to a png file.

#!/usr/bin/python

'''
Expanded upon code from
ZetCode PyCairo tutorial

original author: Jan Bodnar
website: zetcode.com

This code example takes a screenshot of a particular window from name
passed as argument.

'''

from gi.repository import Gdk
from gi.repository import GdkX11
import Xlib
import Xlib.display
import cairo
import os
import time
import sys

def get_window(name):
    mydisplay = Xlib.display.Display()
    root_win = mydisplay.screen().root
    window_list = [root_win]

    while len(window_list) != 0:
        win = window_list.pop(0)
        #print win.get_wm_name()
        if win.get_wm_name() == name:
           return win.id
        children = win.query_tree().children
        if children != None:
            window_list += children

    print 'Unable to find window matching - %s\n' % name
    sys.exit(1)
    return None

def main():
   
    root_win = GdkX11.X11Display.get_default()
    browser_win = get_window(sys.argv[1])
    win = GdkX11.X11Window.foreign_new_for_display(root_win,browser_win)

    width = win.get_width()
    height = win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()

Moving mouse and pressing keys in X using python

I have just seen a cool piece of code in python which simulates key presses and mouse clicks to automatically play a browser based game. This was on Windows but it got me thinking of how similar keyboard and mouse events could be simulated in Linux using pure python (I could wrap programs such as xvkbd or xdotool, or even write some extensions in C and hook into xlib).

Fortunately someone has already written a python X library (Xlib) which I can utilise. Here are a couple of simple programs to get an idea of how to simulate mouse and keypress events.

First up how to move the mouse. This snippet moves the moves to a set location and clicks the first mouse button (left click for my setup)


#!/usr/bin/python
from Xlib import X, display, ext

d = display.Display()
s = d.screen()
root = s.root
#move pointer to set location
root.warp_pointer(300,300)
d.sync()
#press button 1, for middle mouse button use 2, for opposite button use 3
ext.xtest.fake_input(d, X.ButtonPress,1)
d.sync()
#we want a click so we need to also relese the same button
ext.xtest.fake_input(d, X.ButtonRelease,1)
d.sync()


Now, how to simulate keypresses.

#!/usr/bin/python
from Xlib import XK, display, ext, X

d=display.Display()
#send F1 as in gnome this will bring up help screen, proves it works
keysym=XK.string_to_keysym("F1")
keycode=d.keysym_to_keycode(keysym)
#press key
ext.xtest.fake_input(d, X.KeyPress, keycode)
d.sync()
#remember to release it, otherwise help screen will continue to appear
ext.xtest.fake_input(d, X.KeyRelease, keycode)
d.sync()

Monkey Patching

This was mentioned in passing in a python book I read and I ignored it until I saw it being used to discover the internals of frozen/obfuscated python code.

It is a great term, basically you can overload defined functions with your own code, this is particularly useful for code testing. For example I have seen someone discuss testing an emailing function and rather than send out lots of emails they monkey patch parts of smtplib to add the messages to a dictionary.

A simple example
class orig:
    def method1(self):
         print "Original method1()"

    def method2(self):
        print "Original method2()"

if __name__ == "__main__":

    obj = orig()

    def monkeypatch():
        print "Monkey code!"

    print "method1()"
    obj.method1()
    obj.method1 = monkeypatch
    print "method1()"
    obj.method1()
    print "method2()"
    obj.method2()

Simple http server

In need of a quick simple http server on your local system to do some testing?

I have just come across this useful tip.

In a directory that you want to make available via http to do some testing on your local machine run the following command where <port> is a port to listen on

python -m SimpleHTTPServer <port>

The -m option means run a module as a script.

C and Python

Not really sure why I would ever want to do this but here is how to run python code from within C Code. Here is a test program that prints a string using python being called from C.

/*
 * Test program to run python code from C
 * Compile with gcc -Wall -I /usr/include/python2.7/ -lpython2.7 -o runpython runpython.c
 *
 */

#include <Python.h>

int main() {
  Py_Initialize();
  PyRun_SimpleString("print('print using python')");
  return 0;
}

Python Easter Eggs

There are some amusing Easter Eggs in Python.

There is the "Zen of Python" obtained by
import this

This prints out Python programming principles that have been rot13 encoded in the module.

While checking out pypy, I came across this one
import antigravity

This opens a web browser and opens http://xkcd.com/353/

Compiling Python code

I know that compiling python code to bytecode speeds loading of code, however I never really thought about how to do this until now.
There appears to be multiple ways to do this according to the documentation.

The first method runs the code and generate an .pyc file.
In the python interpreter just import your .py file, e.g.
>>> import myfile.py


The next method I have come across is to use the py_compile module, e.g
import py_compile

py_compile.compile("myfile.py")


The next method is to use the compiler package, note this is marked as deprecated since version 2.6 and has been removed from version 3, e.g.
import compiler

compiler.compileFile(myfile.py")

With multiple files this could become time consuming, therefore there is another module compileall, which will compile all files in a directory, e.g.

import compileall 

compileall.compile_dir("mylib")

In the above example it will not recompile exisiting .pyc files if the corresponding .py timestamp has not been updated (timestamp is stored in .pyc file). To force compilation add argument force=1 when calling compile_dir().

See documentation for more useful options.

Fun with Fractals

I saw an article on drawing fractals (Julia sets) using the Scratch programming language so I thought I would have a go at doing this in python using the pygame libraries. Program at the bottom of the page.

While playing around with the code and testing different values I came up with some pretty cool images. The first few attempts had it painting black pixels when they should have been coloured.

first test image (incorrect code)

first image zoomed x axis

experimenting with colours

different colours

Favourite colour selection

random

 

looks like a chinese dragon

Filled Julia set for c=-1+0.1*i

#!/usr/bin/python

import os, pygame, sys;

from pygame.locals import *

def main():

    pygame.init()

    #set environment variable for SDL to position screen
    os.environ['SDL_VIDEO_WINDOW_POS'] = 'center'

    pygame.display.set_caption(os.path.split(os.path.abspath(__file__))[01])

    #windowed mode, 500x500, automatic colour depth
    screen = pygame.display.set_mode([500,500],0,0)

    #equation c=-0.7467+0.3515i
    #some default values
    #c_real = -0.1
    #c_im = 0.651
    c_real = -0.7467
    c_im = 0.3515
    max_It = 20
    col_offset= 5
    x_zoom = 3
    y_zoom = 1
    normalise=255.0/max_It

    #filename to store generated image
    filename='fractal-'+str(os.getpid())+'.png'

    screen.fill((255,255,255))
    pygame.display.flip()

    x = 0
    while x < 200:
        y= 0
        while y < 180:
           z_re = 1.5*(x - 200)/ (0.5 *x_zoom*200)
           z_im = (y - 180 / 2)/(0.5 * y_zoom *180)
           zi2 = z_im*z_im
           zr2 = z_re*z_re
           iterations = 0
           for iterations in range (0, max_It):
           zi = z_im*z_re
                z_re=(zr2-zi2)+c_real
                z_im=(2*zi)+c_im
                zi2 = z_im*z_im
                zr2=z_re*z_re
                iterations+=1
                if (zi2+zr2 >4):
                    break
           print('iterations=%d'%iterations)
           #convert iterations to a colour value [0-255]
           colour=iterations*normalise
           #if zi*zi + zr2*zr2 stays below 4 uptill max iterations, it has
           #'escaped' and pixel is painted black, otherwise paint with 'colour'
           if iterations >= max_It:
                print('colour=black')
                pygame.draw.line(screen, (0,0,0),(x,y),(x,y))
           else:
                print('colour=%d'%colour)
                pygame.draw.line(screen, (255-colour,colour,colour),(x,y),(x,y))
           pygame.display.flip()
           y+=1
        x+=1
    print('fractal drawn')
    while True:
        #exit for loop, e.g. close window
        for event in pygame.event.get():
           if event.type == pygame.QUIT:
                pygame.quit()
                sys.exit()
        pygame.image.save(screen, filename)
        pygame.time.wait(10000)

if __name__ == '__main__':
    main()

Simple animated graphic using pygame

Re-implementation of an exercise that was given to some university students while I worked there, the original was implemented as a Java applet using 2D graphics. I was looking at the pygame libraries and decided to test this out on the RaspberryPi, there are plenty of improvements that could be made, but the libraries seem easy to pick up.

#!/usr/bin/python
#
# draw castle shape using pygame library
# animate castle around the screen, including collision detction of borders
#

import os,pygame,sys

from pygame.locals import *


def main():

    pygame.init()

    #set environment variable for SDL to position screen
    os.environ['SDL_VIDEO_WINDOW_POS'] = 'center'

    pygame.display.set_caption(os.path.split(os.path.abspath(__file__))[01])

    #windowed mode, 400x200, automatic colour depth
    screen = pygame.display.set_mode([400,200],0,0)
   #fullscreen mode, same size as screen resolution, auto colour depth
   #screen = pygame.display.set_mode((0,0),pygame.FULLSCREEN,0)

    size = screen.get_size()

    points = [[0,50],[60,50],[60,20],[50,20],[50,30],[40,30],[40,20],[30,20],[30,30],[20,30],[20,10],[10,0],[0,10]]
    points2 = [[50,100],[110,100],[100,70],[100,70],[100,80],[90,80],[90,70],[80,70],[80,80],[70,80],[70,60],[60,50],[50,60]]

    bgcolour = [255,255,255]

    castleSurf = pygame.Surface((60,50))
    castleSurf.set_colorkey((0,0,0))
    pygame.draw.polygon(castleSurf, (0,255,0), points)

    x = 0
    y = 0
    positivex = 1
    positivey = 1

    while True:
       #exit for loop, e.g. close window
        for event in pygame.event.get():
            if event.type == pygame.KEYDOWN:
               if event.key == pygame.K_q:
                  pygame.quit()
                  sys.exit()
            if event.type == pygame.QUIT:
               pygame.quit()
               sys.exit()

        screen.fill(bgcolour)

        #debug
        #print x,y,positivex,positivey

        screen.blit(castleSurf,(x,y))

        pygame.display.flip()

        pygame.time.wait(60)

        #move bounding rectancle and collision detection
        #print size[0], size[1]
        if positivex:
           if (x+60)>=size[0]:
                positivex=0
                x-=5
           else:
                x+=5
        else:
           if x<=0:
                positivex=1
                x+=5
           else:
                x-=5  

        if positivey:
           if (y+50)>=size[1]:
                positivey=0
                y-=5
           else:
                y+=5
        else:
           if y<=0:
                positivey=1

if __name__ == '__main__':
    main()

Python Command history

I was using the python command interpreter interactively to test out some snippets of python and rather than either typing it out again, copy/paste and removing the prompt markers at the beginning of the line or loosing it entirely, I wondered if there was a way to save the command history like in bash.

Turns out there is an easy way to do this and it is referenced in the python documentation (http://docs.python.org/3/tutorial/interactive.html)

Place the startup script below in a file in your home directory such as .pystartup
and export the environment variable PYTHONSTARTUP pointing to this file

import atexit
import os
import readline
import rlcompleter

historyPath = os.path.expanduser("~/.pyhistory")

def save_history(historyPath=historyPath):
    import readline
    readline.write_history_file(historyPath)

if os.path.exists(historyPath):
    readline.read_history_file(historyPath)

atexit.register(save_history)
del os, atexit, readline, rlcompleter, save_history, historyPath