bash history tricks

I came across an interesting blog entry about maintaining a persistent history of commands across multiple bash sessions using some features of bash I was unaware of.

First there was a variable PROMPT_COMMAND which will run the command stored in the variable before presenting the new prompt.

Then there was the BASH_REMATCH variables, which store substring patterns when used with [[ ... ]] e.g.

test_string="Random string of numbers : 1234"
[[
   $test_string =~ (.*)\:\ +([0-9]+)
]]

echo ${BASH_REMATCH[0]}
echo ${BASH_REMATCH[1]}
echo ${BASH_REMATCH[2]}

This snippet will output the following
Random string of numbers : 1234
Random string of numbers
1234

I have simplified the example from Eli's blog as I am not using the same format for history output , so my code in .bashrc looks like

log_bash_history()
{
  [[
    $(history 1) =~ ^\ *[0-9]+\ +(.*)$
  ]]
  local command="${BASH_REMATCH[1]}"
  if [ "$command" != "$HISTORY_LAST" ]
  then
    echo "$command" >> ~/.persistent_history
    export HISTORY_LAST="$command"
  fi
}
export PROMPT_COMMAND=log_bash_history

gdb not stepping into break point set on strcpy

I was messing around with gdb and some test programs and I set a breakpoint on a call to strcpy, when I ran the program it stepped right over this breakpoint.

Initially I had assumed that glibc had been stripped but this was not the case. Further investigation lead me onto this discussion, where it was suggested to compile with the gcc option -fno-builtin.

This appeared to solve my issue and now when running this code in gdb it does step into the breakpoint set for strcpy.

launching applications upon login to Gnome3

I wanted to run a simple program after logging into a graphical session (Gnome3).

There is a simple graphical utility startup applications (gnome-session-properties), this can be run directly but does not by default show up in search.
To have it show up in all applications or search edit the file /usr/share/applications/gnome-session-properties.desktop and change the line NoDisplay to false e.g.

NoDisplay=false

Adding a new startup application will create a new desktop file in ~/.config/autostart. Therefore to create a new startup application a new file could be created in this directory, for format of the file just copy an existing file and modify as necessary.
A simple entry is below

[Desktop Entry]
Type=Application
Exec=<command>
Hidden=false
X-GNOME-Autostart-enabled=true
Name[en_US]=<name>
Name=<name>
Comment[en_US]=<comment>
Comment=<comment>

python module is in which file on the filesystem

I wanted a quick way to find out which file contains certain python modules, this can by done by looking at __file__ variable for a module.

e.g. in the interpreter
>>> import pygame
>>> print pygame.__file__
/usr/lib64/python2.7/site-packages/pygame/__init__.pyc

 

screenshots using Python under Linux

I have been working on porting some code to Linux and required a method to grab a screenshot of a window using python. I could just execute a command line application such as scrot, but I wanted to see if there was an easy way in native python.
Turns out it is fairly straight forward.

First I looked at taking a basic image of the whole screen, then tried to modify it to allow me to grab particular windows.

My first attempt seemed to work but the output image was skewed, this was using pygtk (gtk.gdk).

#!/usr/bin/python
'''
Modified from example at http://ubuntuforums.org/showthread.php?t=448160&p=2681009#post2681009
'''

import gtk.gdk
import os
import time
import Image

def screenGrab():
   root_win = gtk.gdk.get_default_root_window()
   size = root_win.get_size()
   pixbuf = gtk.gdk.Pixbuf(gtk.gdk.COLORSPACE_RGB,False,8,size[0],size[1])
   pixbuf = pixbuf.get_from_drawable(root_win,root_win.get_colormap(),0,0,0,0,size[0],size[1])
   if (pixbuf == None):
     return False
   else:
     width,height=size[0],size[1]
     return Image.fromstring("RGB",(width,height),pixbuf.get_pixels())if __name__ == '__main__':

   screen=screenGrab()
   #screen.show()
   screen.save(os.getcwd()+'/'+str(int(time.time()))+'.png', 'PNG')

This appeared to be using gtk2, was there anyway to use gtk3?

While searching I found a tutorial on PyCairo and this had an example of taking a screenshot using Gdk and using a cairo Image Surface. This seemed to do what I needed.

#!/usr/bin/python

'''
Modified from
ZetCode PyCairo tutorial

This code example takes a screenshot.

Original author: Jan Bodnar
website: zetcode.com
'''

from gi.repository import Gdk
import cairo
import os
import time


def main():
   
    root_win = Gdk.get_default_root_window()

    width = root_win.get_width()
    height = root_win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(root_win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()


Now to modify this to capture a particular window. I can get information on any window using xwininfo, included in this is the xid (window id).
If I have the id of the window I want to capture can use the function GdkX11.X11Window.foreign_new_for_display() to switch to this window instead of the root window before creating the pixbuf and image surface.

#!/usr/bin/python

'''
Modified from
ZetCode PyCairo tutorial

original author: Jan Bodnar
website: zetcode.com

This code example takes a screenshot of particular window, id in hex
supplied as first argument

'''

from gi.repository import Gdk
from gi.repository import GdkX11
import cairo
import os
import time
import sys

def main():
    winid = int(sys.argv[1],16)
    root_win = GdkX11.X11Display.get_default()
    win = GdkX11.X11Window.foreign_new_for_display(root_win,winid)

    width = win.get_width()
    height = win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()


This works well, however I do not want to have to rely on an external application to get the id. Can I supply a window name and do the same thing?

GdkX11.X11Window.foreign_new_for_display() as well as taking a window id as an argument also takes a window object, so I just need to find a way to identify which window. This was not successful, however if I can identify the window by the name I can just pass the id of that window to GdkX11.X11Window.foreign_new_for_display().
This worked and now I have a script that will take a window name and dump an image of it to a png file.

#!/usr/bin/python

'''
Expanded upon code from
ZetCode PyCairo tutorial

original author: Jan Bodnar
website: zetcode.com

This code example takes a screenshot of a particular window from name
passed as argument.

'''

from gi.repository import Gdk
from gi.repository import GdkX11
import Xlib
import Xlib.display
import cairo
import os
import time
import sys

def get_window(name):
    mydisplay = Xlib.display.Display()
    root_win = mydisplay.screen().root
    window_list = [root_win]

    while len(window_list) != 0:
        win = window_list.pop(0)
        #print win.get_wm_name()
        if win.get_wm_name() == name:
           return win.id
        children = win.query_tree().children
        if children != None:
            window_list += children

    print 'Unable to find window matching - %s\n' % name
    sys.exit(1)
    return None

def main():
   
    root_win = GdkX11.X11Display.get_default()
    browser_win = get_window(sys.argv[1])
    win = GdkX11.X11Window.foreign_new_for_display(root_win,browser_win)

    width = win.get_width()
    height = win.get_height()   
   
    image_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)               
    pixbuf = Gdk.pixbuf_get_from_window(win, 0, 0, width, height)
       
    cr = cairo.Context(image_surface)   
    Gdk.cairo_set_source_pixbuf(cr, pixbuf, 0, 0)    
    cr.paint()

    image_surface.write_to_png("screenshot-"+str(int(time.time()))+".png")
       
       
if __name__ == "__main__":   
    main()

Moving mouse and pressing keys in X using python

I have just seen a cool piece of code in python which simulates key presses and mouse clicks to automatically play a browser based game. This was on Windows but it got me thinking of how similar keyboard and mouse events could be simulated in Linux using pure python (I could wrap programs such as xvkbd or xdotool, or even write some extensions in C and hook into xlib).

Fortunately someone has already written a python X library (Xlib) which I can utilise. Here are a couple of simple programs to get an idea of how to simulate mouse and keypress events.

First up how to move the mouse. This snippet moves the moves to a set location and clicks the first mouse button (left click for my setup)


#!/usr/bin/python
from Xlib import X, display, ext

d = display.Display()
s = d.screen()
root = s.root
#move pointer to set location
root.warp_pointer(300,300)
d.sync()
#press button 1, for middle mouse button use 2, for opposite button use 3
ext.xtest.fake_input(d, X.ButtonPress,1)
d.sync()
#we want a click so we need to also relese the same button
ext.xtest.fake_input(d, X.ButtonRelease,1)
d.sync()


Now, how to simulate keypresses.

#!/usr/bin/python
from Xlib import XK, display, ext, X

d=display.Display()
#send F1 as in gnome this will bring up help screen, proves it works
keysym=XK.string_to_keysym("F1")
keycode=d.keysym_to_keycode(keysym)
#press key
ext.xtest.fake_input(d, X.KeyPress, keycode)
d.sync()
#remember to release it, otherwise help screen will continue to appear
ext.xtest.fake_input(d, X.KeyRelease, keycode)
d.sync()

OpenIndiana

I have not used Solaris in some time so I thought it would be interesting to get OpenSolaris installed under kvm to test out. However, this has stopped being produced, but there is a fork called OpenIndiana.

I downloaded the Live dvd for desktops and booted that up under kvm, I have given the vm a qcow2 format disk to install to.

Once the OS boots up there is an option to install to disk, this will take you through a few install questions and then install to disk.

Qemu monitor

If your virtual system is running under qmeu then you can access the qemu monitor to interact with that os, various things can be achieved such as sending keystrokes, sending nmi, changing state of the vm, snapshotting, add/remove devices etc.

To gain access to the monitor
ctrl-alt-2

There is a good description of the various options in QEMU-Buch / QEMU-Book, the book is in German but there is an English translation.

Running Raspbian image via qemu on x86_64

While testing some virtualisation configurations I came across mention that it is possible for qemu to run code compiled for different architectures. I was particularly interested in arm, as I could then run some quick testing of Raspberry Pi stuff from my laptop.

So here is how to get Raspbian OS running under qemu.

I needed to install the package containing qemu-system-arm to allow me to run software compiled for arm. I also needed a suitable kernel compiled for arm as well, there are some good instructions located at http://xecdesign.com/compiling-a-kernel/ .

The particular arm cpu I am going to use is arm1176, first double check that this is provided by qemu, it should show up in the list of cpus given by the command
qemu-system-arm -cpu ?

The particular arm machine I am going to use is versatilepb, for a full list of machines run
qemu-system-arm -M ?

Next I need a copy of Raspbian OS image from http://www.raspberrypi.org/downloads, extract from the archive.

To boot the OS I use the following command
qemu-system-arm -kernel <location of kernel for arm> -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1" -hda 2013-07-26-wheezy-raspbian.img

This did not work upon first attempt, this is mentioned on the Raspberry Pi forums. I changed my command to include an additional kernel boot parameter init=/bin/sh, and once booted up I remounted the root filesystem rw and commented out the line in /etc/ld.so.preload.

I restarted the OS using the unmodified command above and it booted.

Monkey Patching

This was mentioned in passing in a python book I read and I ignored it until I saw it being used to discover the internals of frozen/obfuscated python code.

It is a great term, basically you can overload defined functions with your own code, this is particularly useful for code testing. For example I have seen someone discuss testing an emailing function and rather than send out lots of emails they monkey patch parts of smtplib to add the messages to a dictionary.

A simple example
class orig:
    def method1(self):
         print "Original method1()"

    def method2(self):
        print "Original method2()"

if __name__ == "__main__":

    obj = orig()

    def monkeypatch():
        print "Monkey code!"

    print "method1()"
    obj.method1()
    obj.method1 = monkeypatch
    print "method1()"
    obj.method1()
    print "method2()"
    obj.method2()