Wednesday 21 August 2013

Developing Android apps

I thought I would have a quick run through some of the tutorial info on getting a basic Android application built and then run it on the Android x86 port I have running in a VM.

I downloaded the adt bundle for linux which includes the sdk, a version of eclipse and the adt plugin. First time through I forgot that I did not have a jdk environment installed so had to grab that as well.
I also added the platform-tools and tools directories to my PATH.

Following the instructions I tested out a basic layout on the emulator that comes as part of the bundle and then exported my project as an Android application.

Here is where I believed the tricky part would be, to get the apk file onto my android instance running in a VM. I could have placed it on a webserver and then downloaded it (but where is the fun in that). If this was runnign on a real device/handset I could connect it up via usb and use adb.

Actually I can still use adb, as android is running in a VM and has networkig configured I just need to configure adb to connect to the VM.
I do this by the command

adb connect <ip address of VM>:5555

Then I can load my custom application using
adb install <apk file>

When I did this I needed to dismiss a warning that popped up on android, but the application installed fine. 

Once the application has been installed, disconnect adb using
adb disconnect

Both before connecting adb and after it is worth checking that an existing adb server instance is not already running, it can be killed with
adb kill-server

Now to figure out a useful app to develop.

Tuesday 20 August 2013

Haiku

I am not about to break into poetry so don't panic.

A number of years back when I first started looking at Unix/Linux I came across several other Operating Systems which I would like to try out, one of these was BeOs (time permitting I may explore some more).

However, BeOs has all but died and was a proprietary OS anyway, but I did come across some people developing an opensource implementation called Haiku

I have downloaded it and installed it in a virtual image under KVM.
I created a 3Gb raw image and have booted from the ISO image using the following command to bring up a 32bit vm and given it 512Mb

qemu-system-i386 -m 512 -boot d -cdrom <image file> -hda haiku.img

I just passed it a raw image, so using the disk utility in the installer I need to create a partition and a location to create as Be File system. Follow the instructions in the installer and it is installed.
Next time when booting up the following command is used

qemu-system-i386 -m 512 -boot c -hda haiku.img



Friday 16 August 2013

emscripten

There was a cool demo using the Unreal Engine that had been ported to javascript using emscripten I saw a while ago.
I had some time I thought I would test this out and see what it can do, I will come back to it later if I have any nifty code that I want to reproduce on the web.

Following the instructions, I downloaded and compiled up llvm (& clang) version 3.2, as per the instructions 3.3 (which was provided in my distribution) does not work.
I needed to grab node (version 10.15 was the one I tested) as well.

Once I had all these I ran the emcc utility which generated a default config file in my home directory (.emscripten), the paths in this needed modifying as I compiled/installed the utilities in my own directory and not in directories for system wide use.

I ran emcc again and it ran with out errors, just the fact that I had not provided any input files, so looks good.

Following the brief tutorial on the wiki I was able to run some of the test programs and some very simple C code that I had written (including some basic openGL using simple lines).

When I have some good examples I will attach them to this page.

Wednesday 14 August 2013

Simple http server

In need of a quick simple http server on your local system to do some testing?

I have just come across this useful tip.

In a directory that you want to make available via http to do some testing on your local machine run the following command where <port> is a port to listen on

python -m SimpleHTTPServer <port>

The -m option means run a module as a script.

Tuesday 13 August 2013

Compiling old versions of gcc

I wanted to compile up an old version of gcc (2.95.3) for another project which I might blog about in the future, I assumed this would not be too difficult, however it turned out more troublesome than I had thought.

I have on my Linux system gcc4 compiler and to use this to compile the source of version 2.95.3 a few tweaks have to be made. I should also mention as a remindeer to read and understand the output/errors they can save some time. I would have got to the solution sooner if I had matched up the error with some information I already had.

I unpacked gcc 2.95.3 source to a directory and then created a separate directory to build into (as recommended), I have also created a separate directory to install finished binaries into so as not to mess up my current system (also doing this as a non-root user).

First attempt to compile (from within my build directory) using the following options
../gcc-2.95.3/configure --prefix=<new bin directory> --enable-languages=c,c++ --enable-threads=posix --enable-shared --disable-nls --host=i386-pc-linux-gnu --target=i386-pc-linux-gnu

I only want support for c and c++, posix threads are enabled, I have disable NLS as I do not need it and I am specifying I want to create a compiler for 32bit.

This attempt fails to compile (due to two issues, but originally I concentrated on one and wasted a lot of time).
The first issue I noted was the errors from the assembler such as
Error: invalid instruction suffix for `push'
Error: invalid instruction suffix for `pop'

This suggested I had an architecture issue, so I spent a lot of time looking for the solution to this and ignored the following problem (which later on I realsied needed fixing too).

/usr/include/bits/pthreadtypes.h:99: warning: unnamed struct/union that defines no instances

This version of gcc does not support the unnamed union definition that appears in one of the header files on my system, more about how to solve this one later.

While trying to track down solutions for the architecture problem I believed I was facing above, I came across a blog entry here, which seemed to document a similar issue about invalid instructions.
The fix given was to add "--32" to the gcc assembler spec options in the header file gcc/config/svr4.h 

Second attempt to build having added the above entry, again compiling in my build directory with the following options

../gcc-2.95.3/configure --prefix=<new bin directory> --enable-languages=c,c++ --enable-threads=posix --enable-shared --disable-nls --host=i386-pc-linux-gnu --target=i386-pc-linux-gnu

This also failed to compile and returned

In file included from /usr/include/sys/types.h:271,
                 from /usr/include/stdlib.h:320,
                 from ../../gcc-2.95.3/gcc/frame.c:42:
/usr/include/bits/pthreadtypes.h:99: warning: unnamed struct/union that defines no instances
../../gcc-2.95.3/gcc/frame.c:55: extra brace group at end of initializer
../../gcc-2.95.3/gcc/frame.c:55: (near initialization for `object_mutex.__data')
../../gcc-2.95.3/gcc/frame.c:55: warning: excess elements in struct initializer
../../gcc-2.95.3/gcc/frame.c:55: warning: (near initialization for `object_mutex.__data')

This is what I noted earlier and spent ages on other possible problems ignoring this one and again this I had already found on the blog entry here, and is due to an unnamed union definition. The solution to this is just to make a copy of the header file, add a name to the unnamed union and then get gcc to include it.
The makefile that needs modifying is gcc/Makefile.in, the directory containing the modified header file is included before any others.

This now successfully compiled on the next attempt and I was able to use this to compile 32bit binaries.

In between each attempt I issued "make distclean" to ensure a clean directory in which to do the builds.

Friday 9 August 2013

Nested hypervisors

I wanted to try out some different hypervisors and rather than shelling out for additional hardware I thought I would see how well my current system copes with nested hypervisors.

Initial test with VMWare under KVM failed, it did not seem to like the CPU I had provided to the guest and no matter what options I chose it did not want to run, therefore that is on hold for now.

However, Citrix Xenserver appeared to install fine.

I want to add a guest to Xenserver, just to prove this is working, but when I created my original disk I gave it only the bare minimum for Xenserver which was 16Gb. So I have created a new raw disk image using qemu-img and attached this to the Xenserver VM.
This image is attached as a new SCSI disk (so it will be /dev/sdb).

I now need to create a new storage repository (SR) which Xenserver will use to store my guests.

First off I need to fine the id of my new disk (/dev/sdb) this can be done by looking at the symlinks in /dev/disk/by-id and finding the one pointing to /dev/sdb.

For my Xenserver vm
ls -l /dev/disk/by-id















I also need the uuid of the Xenserver host, to get this use
xe host-list

Now I can create the new SR using
xe sr-create content-type=user device-config:device=<disk id> host-uuid=<host-uuid> name-label=”Second Disk” shared=false type=lvm


Creating new SR
















Tip:- When entering the console from the xsconsole application, the font was pretty unreadable, I just ran setfont  with no options to load the default and this resolved the issue. To set the font permanently modify /etc/sysconfig/i18n.

Now I should be able to create a new VM on this new SR, but first I need to get the uid of this new SR using
xe sr-list

To create the new VM using an available template

xe vm-install template=<template-name> new-name-label=<vm-name> sr-uuid=<sr-uuid>
To list the available templates use
xe template-list


Tip:- With the xe command you can use tab completion to finish commands and uuids.

The new VM now shows up in the list of VMs
xe vm-list

I have a basic VM (with nothing in it), which I now need to start up and boot from some install media. As I want to boot from the install media I need to set the disk for my VM to not be bootable as I cannot have two boot devices.
First off I need to find out the uuid for virtual block device (VBD) which will contain my VM, this will be the device xvda in my OS and is therefore the first userdevice (0).


xe vbd-list vm-uuid=<vm_uuid> userdevice=0 params=uuid --minimal

Set this to not be bootable
xe vbd-param-set uuid=<root_disk_uuid> bootable=false
I have attached an iso of the install media for a linux distro as a cdrom/dvd drive to my Xenserver VM, I now need to attach the cdrom/dvd drive to the new VM which I will boot up on Xenserver.
To find out what cdrom/dvd devices are available use
xe cd-list
In my case this is labelled "SCSI 2:0:0:0", and to add this device as a virtual cdrom/dvd drive to my VM is use the following
xe vm-cd-add vm=<vm_name> cd-name="SCSI 2:0:0:0" device=3
This will attach the cdrom/dvd drive of the Xenserver host (in my case the iso I have attached to the cdrom/dvd drive of the Xenserver VM) as device 3 which in Linux will be xvdd.
I need to make the VM boot from the cdrom/dvd drive so I need to find the VBD and set it to be bootable
xe vbd-list vm-uuid=<vm_uuid> device=xvdd params=uuid --minimal
xe vbd-param-set uuid=<cddrive_uuid> bootable=true
Finally, before I start the VM I need to set the install-repository to be the cdrom
xe vm-param-set uuid=<vm_uuid> other-config:install-repository=cdrom
Once all the above has been completed successfully it is time to start the VM
xe vm-start uuid=<vm_uuid>
This should start the VM, to check this look at the power-state line in the output of
xe vm-list uuid=<vm_uuid>
I want to be able to interact with the installation program to install my OS onto my VM, normally if Xenserver was on physical server I could connect to it using Xencenter. However, it is on a virtual machine running under KVM, also that would require me to run Windows.
I could create another VM under KVM and connect to it from there but as this is runnign on my laptop it sounds like overkill.
Xenserver exports the display of the VM via VNC, again not very useful as I do not have a VNC client on the Xenserver host and I would have to forward ssh connections to it from outside the Xenserver VM. So I need a console connection to the console of my newly created VM, there are two commands on Xenserver that allow you to connect to the console of a VM, xenconsole and xl console.
For some reason the location of xenconsole is not by default in the PATH for root, it can be found in /usr/lib/xen/bin.
To use either of these tools I need to know the domain-id of the running VM I want to connect to, all running domains can be gotten using the command list_domains.
list_domains output

My VM is listed as having a domain-id of 4.
The domain-id is also available in the dom-id parameter for the VM and can bo obtained using
xe vm-list uuid=<vm_uuid> params=dom-id
I tried both of these tools and I was not able to get a usable connection to the console for my new VM.

**Update**
I later discovered that this could be due to xapi setting up vncterm which takes the text console and draws them as graphics to make them available to vnc clients.
There is an option which can be set before the VM is start which will then allow me to use xenconsole or xl console, this can be done with the following
xe vm-param-set uuid=<vm_uuid> other-config:disable_pv_vnc=true
This may impact tools that connect via vnc
To remove this option use
xe vm-param-remove uuid=<vm_uuid> param-name=other-config param-key=disable_pv_vnc
*****

I found out that the text consoles are also made available on certain ports. To find out which port I need to look up the console entry for that domain-id (as mentioned above) in xenstore.
xenstore-ls /local/domain/<domain-id>/console

console entry in xenstore

For some guests (depending on whether they HVM or PV) this entry might be /local/domain/<domain-id>/serial.
If I telnet to the "tc-port" I get access to my text console.
A few notes on this, I changed the tty settings using "stty sane" running telnet. I also set character mode within telnet. e.g.
stty sane
telnet localhost <tc-port>
ctrl-] mode character
Once the install has finished the cdrom/dvd should be ejected from the VM
xe vm-cd-eject vm=<name or uuid>


Wednesday 7 August 2013

Internals of Android applications .apk files

Ever since I saw an article on creating a basic android application I have been curious about how some android applications have been created and since I have a working android installation it is time to take a look at how they work.

My android install is on a qcow2 disk image so I can mount it on my system via ndb, just like I did when building my LFS system.

I copied off an apk file to test.

The apk file is effectively just a zip file, so I could just unzip it to get the files, however when I do this most of the files are in a binary format (serialized objects?) and unreadable. Since the discussion a few months back on android bug 8219321 I was aware of the apktool utility to unpack an apk file.

So used apktool to unpack the apk file, like so

apktool d <apkfile> <dir to unpack to>

I now have manifest xml files which are readable and code. I was expecting to get java sourcecode files from this process, but all the files were smali (Dalvik byte code). This does not look too hard to understand but I wanted something I was a little more familiar with, i.e. java.

Therefore I needed to extract files from within the classes.dex file (Dalvik executables) and convert them to java classes.
So I just extracted the classes.dex file from the original apk file using unzip (jar would work as well if a jdk was installed).

I now need to convert the Dalvik executables to java class files, to do this I used the utility dex2jar e.g.
d2j-dex2jar.sh classes.dex

This has converted the dex file into a jar file, again I could use the jar utility to extract the class files but as it is basically a zip file I will use unzip.

I now have a bunch of java class files which I need to disassemble and look at. In the past I have not had much luck with java dissassemblers, however, I have seen one recommended called jd-gui.

This is a 32bit application and as I am running a 64bit OS I needed to install the 32bit libXxf86vm runtime library.

OCR

I was wondering if there were any easy to use OCR tools for Linux, turns out there is tesseract, originally developed by HP and now maintained by Google (I guess they are using it for their digitisation projects).

To use it simply pass it an image file and the file you want the text to be written to e.g.
tesseract <image file> <output>

I first tried with a png image but that did not produce anything, looks like tiff seems to work so I converted the png image using gimp.

This could be useful to script up something with some scanning software (like scanimage) to automaticlly produce a text file of a scanned document.

I will post more on this once I have played around with it some more.

Tuesday 6 August 2013

Dynamic favicon

I saw a post the other day, that Google had changed the favicon to display a play symbol when audio is playing in that tab/window, therefore I decided to see if I could knock up a page that dynamically changes the favicon.

My idea was to create icons that would count to 10 just so it would be easy to tell when it is working correctly. Therefore I created 16x16 pixel png images containing my numbers 1-10 called favicon1.png - favicon10.png. I also created a start image called favicon0.png this is the same image as posted on the banner of my blog (one of the fractal images I created).

To add a favicon is easy just add the following to your html in the head section

<link id="favicon" rel="shortcut icon" type="image/png" href="location of your image">

To change this dynamically it should be easy create a loop where the loop counter goes from 0 to 10 and modify the href in this line.
First problem, how to modify this line. I gave the link an id so I should be able to reference this via the DOM. This can be referenced using the following piece of javascript

document.getElementById('favicon');

Now I just need to create a loop in javascript and modify the href, but wait there is a problem with this. To display each individual favicon image I need my script to pause for a short amount of time. In other languages we would just pause by calling some kind of sleep or wait function, however as this will be running in a browser we do not want it too sleep as this would impact the loading of the page. It also turns out that there is no sleep function in javascript for precisely this reason.
So how to achieve a pause to display the images. Looking around, there are some Timing Events that can be used in javascript, the one that looked to fit the bill is setInterval.
To use this I had to modify the way I was writing the script, rather than having a loop and pausing within the loop, I created a function that I would call at a particular interval.

I also had to remove the old favicon link before adding a new one.

The following sample code is what I came up with and seems to do what I was after, note I was testing on my local machine so the href entries all point to local files.


function changefav(i) {
   //only have 10 numbers to display
   if (i==11) {
         //remove Timeout
        clearTimeout(changeInterval);
        return;
   }
   var filename = "file:///tmp/favicon" + i +".png";
   myfav=document.getElementById('favicon');
   header.removeChild(myfav);
   lnk.href=filename;
   header.appendChild(lnk);
}

header=document.getElementsByTagName("head")[0];
//create new link element, only href will change
var lnk=document.createElement('link');
lnk.id="favicon";
lnk.type="image/png";
lnk.rel="shortcut icon";

var i=0;

//call changefav function every 100 milliseconds
changeInterval=setInterval('changefav(i++)', 100);



I placed this into a simple html document and added a favicon link as mentioned above.

This worked well in firefox.

Instead of creating separate images I could have taken a base image and then modified it using a canvas element dynamically in javascript.

While researching favicons I found Defender of the Favicon, a Defender clone implemented as dynamic favicons.

Thursday 1 August 2013

Uploading images to Blogger

I have been struggling for a while to get images uploaded to blogger successfully every time.

My usual browser is Firefox, less and less frequently am I able to upload images using the Insert Image button in the composer without it throwing an error. I have seen other blogger's comments that say it works much better in html mode than in the wysiwyg mode, however most of the time this failed for me in the same way.

There is also some suggestion by other bloggers that this works much better in Chrome, however, I also see a number of comments from people that this is unpredictable as well.

I came across another web browser, which seems to work every time (fingers crossed), this is a lightweight webkit based browser called Arora (http://code.google.com/p/arora/).

While this continues to work, I will be using this to update my blog.

**Update Sept 2013**
An update to firefox worked for a while, however I am back with Arora as now I am getting a different issue.
Every time I login I get a message telling me I have logged out from somewhere else and do I want to login again. I see this was reported as a known issue a year ago, however it seems to have reappeared.