Your choice, your reputation

Posted on Monday, 14th May 2012 by david

Earlier in the year we ordered made to measure curtains from Laura Ashley. They quoted 7 weeks delivery but arrived 5 weeks after our order and have been exactly what we wanted. Following this experience, deciding to place a subsequent order wasn’t a hard decision. We placed the order and waited. And waited. And waited. Returning home today we had an answering machine message from Laura Ashley saying their courier had failed to deliver them after leaving several cards. As we’ve haven’t had a card through the door for the last 8 weeks or so and certainly haven’t had multiple cards, this has left us a little puzzled.

Tomorrow we’ll call Laura Ashley and try and rearrange delivery, but sadly it’s yet another issue with City Link. Yes, the courier company that seems to try harder than any other courier to ruin the reputation of any company using them strikes again :-(

Just to clarify the situation

  • City Link have delivered to this address before
  • Laura Ashley have the correct address as the previous order arrived here
  • We were here during the period the deliveries allegedly took place
  • Our neighbours were also in continuously during the alleged deliveries
  • City Link routinely leave parcels with them if we’re not in (this is what happened to the previous delivery)

So, why no delivery???

Posted in General | Leave a comment

Banking on Trust

Posted on Monday, 14th May 2012 by david

I last wrote about our experiences with HBOS in This is not the bank you want, but tonight has proved that things really could get worse.

Towards the end of April we opened a new account with Halifax to try and get a better savings rate for our money. The account was, like so many these days, an online only account. To get the better rates of interest this seems like a reasonable compromise and given we already had internet banking setup didn’t pose any problems. We setup the account and arranged the transfer of some money from our older savings account that was paying 1/30th the amount of interest! All seemed to go OK and the account said it was setup.

When I checked our accounts online earlier this evening I was surprised to find no mention of the new account. I had received no confirmation or details of the account, but as it was an internet only account that wasn’t unexpected, but for it to not be listed was a surprise. When I tried calling their support telephone number I was greeted by a message telling me that it could be 20 minutes before they answered. Deciding to persevere I waited and after 30 minutes I was finally connected to an operator. After explaining the problem his initial reaction was
"I think I know what the problem is. We had a technical glitch last month."
Not being filled with confidence I went through security and after providing the details of the new account I had gleaned from the transfer details (having no other information the fact the details were listed was a relief) he confirmed that the account had suffered from “the glitch“.

As he explained it, “the glitch” basically removed all of my details from the account. The account was still there but was no longer associated with me! PARDON. So basically the bank had suffered an issue that meant a large amount of my money was no longer my money. They knew about it and hadn’t bothered to try and trace those affected. Had I not contacted them the account would have just sat there – with no apparent owner. I did check and it’s not April 1st, but surely this had to be a joke? One of the largest banks in the UK had managed to loose my money? In 2012?

To say that I’m astonished and very concerned about this is a mild understatement. Every time there is an issue with HBOS I say I should move my money, but after this it really is time to move.

The operator took my details and said the account would be reattached to my “profile” which should mean that within 48 hours I have access to and control of my money again. Time will tell.

Update
The account has been linked and is now available.

Posted in General | Leave a comment

Rosie Gets a Blog

Posted on Sunday, 6th May 2012 by david

It would have been a busy weekend for us without the problems a server upgrade threw my way, but one nice change is that Rosie now has her own blog! She’s been on a journey of discovery getting wordpress installed and themed etc, but she’s now starting to add content. It can be found at http://www.rosie-reid.com/blog and is currently entitled Shades of Grey.

Posted in General | Leave a comment

Adding a RAID array

Posted on Wednesday, 25th April 2012 by david

The home server (a Tranquil PC BBS2 as I wrote about before) started life with a 2 disk 250G RAID array and soon saw an additional 2 disk 1Tb array added. It’s done us well and just sits there purring away under the stairs serving files to the rest of the house. It’s done the job excellently but over the last few years we’ve been eating through the storage space available. I’ve been conscious that the amounts of data involved are non-trivial and so while there was enough space on the 1Tb array to move all the data from the smaller 250Gb onto it I decided it was time to think about an upgrade.

I bought 2 2Tb drives and copied the data from the small array onto the larger one (there was only around 100G of data). Removing the old drives and inserting the new ones proved to be exceptionally straightforward thanks to the design of the BBS2. The screwless drive enclosures are great.

When the machine was rebooted I found the disks available as /dev/sdd and /dev/sde. What follows is what I *should* have done, but it took me a few attempts to get it done in this order thanks to missing a vital step :-( Thankfully until the drive was happy following reboots I didn’t copy any data to it :-)

Before starting I should say that the BBS is running Ubuntu 10.04 Server edition.

Prepare for Disk Change
Before shutting down to replace the disks I needed to prevent the system trying to use the 250G array when starting. (If you don’t then it generates errors and stops the boot requiring console access.) I did this by

  1. commenting out the /dev/md0 entry in /etc/mdadm/mdadm.conf
  2. commenting out the entry for it’s mount point in /etc/fstab

Once these changes had been made, I shutdown the system

sudo shutdown -p now

Disk Setup

  1. sudo fdisk /dev/sdd
    Create a new primary partition using the entire disk
  2. sudo fdisk /dev/sde
    Create a new primary partition using the entire disk
  3. sudo fdisk -l
    Check the 2 disks (sdd and sde) show the same partition details

At this point a check of /dev shows the expected sdd, sdd1, sde and sde1 listings.

Create the Array
I use /dev/md0 and /dev/md1 for the first 2 arrays, so decided to use /dev/md2 for this array.
mdadm --create --verbose /dev/md2 --level=1 --raid-devices=2 /dev/sdd1 /dev/sde1

This resulted in

mdadm: size set to 1953513472K
mdadm: array /dev/md2 started

To follow progress I watched the output from /proc/mdstat using

watch cat /proc/mdstat

The output looked like this

md2 : active raid1 sde1[1] sdd1[0]
1953513472 blocks [2/2] [UU]
[>....................] resync = 0.0% (875136/1953513472) finish=855.2min speed=38049K/sec

It took slightly less time than initial predictions, but I left it running overnight!

Once it has finished /dev/md2 will exist and be available for formatting, but as I had experienced problems I decided to be cautious and ensure the array was usable through a reboot before formatting it, which meant I needed to add the details to /etc/mdadm/mdadm.conf (yes this was what I forgot to do when I experienced issues).

Add the array details to mdadm.conf
To create the lines needed there is a nice shortcut command
sudo mdadm --detail --scan
This will create output similar to this

ARRAY /dev/md1 level=raid1 num-devices=2 metadata=00.90 UUID=xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx
ARRAY /dev/md2 level=raid1 num-devices=2 metadata=00.90 UUID=xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx

To add the lines to mdadm.conf I simply did
sudo mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Of course nothing in life is as simple as it first appears, so the file needs to be edited :-(

  1. I removed the duplicate line for /dev/md1
  2. The metadata value doesn’t seem to be recognised as ‘00.90‘ but is happy with ‘0.90‘ so I removed a leading zero.

After saving the file it was time to reboot the machine and make sure that all worked as expected.
sudo shutdown -r now

After restart a quick look at /proc/mdstat showed all was well

cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md2 : active raid1 sdd[0] sde[1]
1953513472 blocks [2/2] [UU]

md1 : active raid1 sdc1[1] sdb1[0]
976759936 blocks [2/2] [UU]

unused devices:

Formatting the new disk
I formatted the new disk as ext4 using

sudo mkfs.ext4 /dev/md2

After this had completed I needed to add the mountpoint and an entry to /etc/fstab

sudo mkdir /storage/2

To add the entry to fstab I generally use UUID’s, so I needed to find the UUID for the new disk.
ls -l /dev/disk/by-uuid

lrwxrwxrwx 1 root root 9 2012-04-25 15:05 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -> ../../md2

The fstab entry then looked like

UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /storage/2 ext4 relatime,errors=remount-ro 1 2

To test that all was working as expected, I simply did

sudo mount -a

No errors were shown and a quick check showed the disk mounted and the sizes as expected.

df -H

Filesystem Size Used Avail Use% Mounted on
…
/dev/md2 2.0T 205M 1.9T 1% /storage/2

Before copying any data across I decided to make sure that a reboot wasn’t going to cause trouble and so once more I restarted the machine

sudo shutdown -r now

Everything worked and all expected devices were available, mounted where I expected them to be.

Summary
It’s taken me a little longer than I expected to get things fully setup, but as last time I did it I didn’t bother noting what I did that’s not a huge surprise. The links below were helpful in figuring out what was needed :-)

Posted in General | Tagged | Leave a comment

This is not the bank you want

Posted on Monday, 23rd April 2012 by david

It’s been one of those days again. We had a lot of small tasks to do while in town this afternoon and the 2 hours we had was always going to be a short time, but with some fast walking a spot of luck we had an incredibly successful afternoon, completing every task except 2 – both of which involved the Halifax/Bank of Scotland.

The problems started when we tried to arrange an appointment to check some information for the mother in law. It was all pretty simple stuff but she likes the face to face contact that going into a branch gives. I rang and spoke with someone who seemed to be having a hard time understanding what it was we wanted, then telling me there were appointments available, but if you want that, then, oh, no, sorry we’re really busy and there’s no appointments available. This seemed like a poor result after 3 phone calls (first 2 rang out) and a 5 minutes phone call involving me sitting waiting on hold for 4 minutes. Still, we decided just to ignore the stuff that involved asking questions and just pop in and do the simpler stuff.

When we arrived at the branch we were asked to wait for an “adviser”. Rosie had a question about a letter that had arrived at lunchtime so she showed the advisor the letter and asked for an explanation. The head scratching began. Several possible options were offered – but as Rosie had discussed these in her explanation of the problem we weren’t impressed. listening was not one of his strengths. He went off to talk to a colleague who gave him a form to do something that we had already done and, indeed, something that Rosie had explained we had already done in her explanation. His grasp of the issue appeared very weak and seemed to be getting weaker. Eventually he talked himself into a corner and decided there was nothing he could do. Rosie got her letter back and will simply do what seems right.

The mother in law then started trying to get her small task completed. As it turned out the account we were trying to transfer into didn’t allow transfers, but when it was setup this wasn’t explained! Oh. Can we say poorly advised?

Returning home we decided to simply use the online service. Resetting the access details was required and went OK. For the first time in 6 months the online access worked and we were able to do some of the things we needed to. Buoyed by our success, we tried to open a new account for her online. Oh boy.

Suddenly there was a cryptic error message being displayed

Unfortunately there has been a technical problem. This transaction has not been completed.

whenever we tried to login to open a new account. Sometimes we could login to the accounts page, but often we couldn’t. Removing the cookies sometimes worked, sometimes it didn’t. To say this was frustrating was an understatement.

Apparently the Halifax don’t want our money.

This isn’t the first time that they’ve displayed poor service, but the level of failure shown today has been beyond anything I could have imagined. Our decision to move our day to day banking to First Direct was one of the best we’ve made and in the last 3 years their customer service has not failed once. If they only offered better savings product we wouldn’t even consider using another bank, but there are better places for savings :-(

The mother in law is of a generation that remembers their bank managers and has fond memories of banks and building society branches. The weekly visit to pay in wages stick in their minds and are recounted fondly whenever we visit, but even she is starting to dread having to deal with them. Online banks such as First Direct may offer a great solution for us, but for her the thought of not having a branch is just a step too far.

Surely there must be another bank in the UK that still knows what service is and can deliver it consistently?

UPDATE: It’s now impossible to login to the accounts with every attempt resulting in the same cryptic error message. :-(

Posted in General | 1 Comment

Photos

Posted on Saturday, 14th April 2012 by david

Earlier this year my pro account at flickr expired. I had the usual notifications prior to this and it casued me to start wondering what I wanted to do with my pictures. This wasn’t an easy question to answer as there are a lot of possible answers and an almost infinite range of answers already in existance. A few weeks ago I made my answer public and so far I’ve been very happy with the results.

So what did I finally decide to do?

One a Day

The site is very simple. One picture per day, given it’s day on the home page and then an after life on a url defined by the date it was displayed. I came to realise that when I added a number of pictures to flickr only the final few were ever looked at and even fewer were really ever “looked” at.

Older images are displayed in date order. Images may have a title, but don’t need to have one.

It’s minimalist. But that was what I wanted. I wanted the site to be about the pictures. Not how I took it. Not where I took it. Not when I took it. It’s the picture stupid.

The machanics are even more straightforward. Upload an image (optionally with a title) and it finds the next available date to display it and makes the smaller versions (according to a defined list). Every day the front page finds the image for that day and displays it. A tweet gets sent every day with the title and a link to the permanent url for the picture. Simple.

I added support for pingbacks and added an RSS and an Atom feed. Twitter and Google+ buttons followed (but are never used).

The results are interesting. I’m getting as much feedback as I ever did from posting at flickr. The number of views of images is higher (only by a small margin) than the better pictures on flickr.

It’s a work in progress, but so far I’m happy with my choice and have found the self imposed limtation to be liberating rather than restrictive.

Posted in Photography | Leave a comment

When the boot hangs…

Posted on Saturday, 24th March 2012 by david

In my attempts to get a home built kernel running on my phone I’ve had quite a few times where things haven’t quite worked and I end up stuck at the HTC screen. As there isn’t any output via USB at that point it’s frustrating to know what the problem is, but there is a solution…

1. Power off the phone (remove the battery)
2. Start the phone in recovery
3. Use adb to view the dmesg

This gives you the previous boot dmesg and hopefully more information about what went wrong :-)

Posted in General | Leave a comment

One year on…

Posted on Friday, 23rd March 2012 by david

Just over a year ago I wrote This is not the Android you want and as I’ve been spending time with android again I find it a little depressing how little progress has been made. Google have made their publishing process for AOSP slicker and quicker, the interfaces have gotten better and generally Android is much improved, but still things feel closed.

The recent release of the Ice Cream Sandwich (ICS) code has led to the usual raft of releases of third party ROM’s based on the ICS code, but for many devices the changes have been too large to allow easy transitions. The complete change of the camera interface has meant few older phones have decent camera support and the utter lack of information from the manafacturers means this isn’t likely to change anytime soon. The kernels need to be updated, the AOSP source needs to be tweaked and lots of patches and work arounds need to be found – all of which delays the adoption of ICS for older phones. In fact the adoption rate on more recent phones via updates from the phone manafacturers hasn’t been great either. Change is good, but the cost appears to be great.

I have some sympathy with the hardware manafacturers as we, the end user, are not their customers. We didn’t pay them money or sign contracts with them, we bought phones that had been developed and built using their components. Companies like Qualcomm guard their IP closely and whenever they are asked for details of their devices they defer saying that they can’t release details as they are “commercially sensitive”. They release binary files that require specific versions or patches of kernels to work and claim to be “doing their bit”. HTC are no better and in many cases even worse. At least Texas Instruments have actually released source for many of their components and even have it included in the AOSP tree as it’s needed for the Google phones.

A simple eearch on Github will reveal a huge number of forks of forks of forks of the AOSP code and even more of the same for kernels to support AOSP builds. There are so many to choose from it’s a little bewildering. Some are very old and abandoned, some are kept up to date, most are somewhere in the middle. If one of the strengths of git is that it fosters a culture of easy forking and branching, that’s also one of it’s real weaknesses when it comes to developing a consistent codebase. I can understand why AOSP is focussed in the way it is, but surely it would be in the interest of Google and Android to have a central repository of all things Android? Take the AOSP tree and accept patches and improvements from the open source world. There are a lot of people spending a lot of energy to get older devices working, often with very little attention and with their work repeated or forgotten as it’s easy overlooked in the noise that is the AOSP ROM universe. A single, definitive repository with a well explained path to submit patches would allow everyone to participate.

Like many people, I won’t submit a patch unless I can test it, a requirement that makes any direct involvement in AOSP less likely as the source tree I have is actually a collection from CyanogenMod. The Gerrit instance for patches isn’t AOSP, it’s CyanogenMod’s. While AOSP is a good thing, it’s more fo a half way house and really could be so much better.

In todays world people expect updates. One of the attractions of an Android phone is the promise of updates and improvements. Each release gets a lot of attention and hype, but it’s always accompanied by the knowledge that by the time it arrives on your device the next version will be arriving. It’s not what people expect and damages the Android brand.

My last 2 phones have run Android and been made by HTC, but my next one probably won’t be from HTC and may not even run Android. The idea behind Android is amazing, but the vision is not being matched by reality. Until the manafacturers start to be more open and Android itself becomes more coherent it will fail to be what it should be. Qualcomm may take pride in their hardware, but their attitude towards openness and community means that my next phone will not feature any of their products. I’ll vote with my money and would encourage others to do the same. In some enchanted land, far, far away, if enough of us did it they would change their ways :-)

Posted in development, mobile | Leave a comment

Camera HAL Module

Posted on Wednesday, 14th March 2012 by david

I’ve spent a bit of time looking at a module that would allow the camera sensor on my phone to work with the new ICS interface. I’ve not made as much progress as I’d have liked, but have the start of code that at least doesn’t crash and should allow me to continue building…

Today I took the big step and started a github repository with the code. Hopefully others will look at it and fork the repo to help out, but that might be expecting too much. Time will tell. There are significant steps to overcome and so far all I’ve done is the easy parts :-( If you can help or know someone who can, point them at the code and lets see if we can’t overcome the reluctance of Qualcomm and HTC to release the information that’s needed.

https://github.com/zathras777/msm7x30_camera_hal

Posted in General | Leave a comment

Android Kernel Debug

Posted on Friday, 9th March 2012 by david

Recently I’ve been trying to trace some kernel debugging being generated on an android phone and the relative shortness of dmesg has been proving an issue. I figured there would be another way and of course there is…

On a computer attached via a USB cable with USB debugging enabled, simply do

adb shell cat /proc/kmsg

Simple when you know how :-)

Posted in development, mobile | Leave a comment

Building for HTC Desire HD

Posted on Friday, 2nd March 2012 by david

This is for a build in ~/work/android/system rather than the usually quoted ~/android/system.

Start off in the usual way…

sudo apt-get install gcc-multilib
sudo apt-get install bison flex schedtool
cd work
mkdir bin
curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > bin/repo
chmod a+x bin/repo
mkdir android/system
cd android/system/
repo init -u git://github.com/CyanogenMod/android.git -b ics

Before starting the long first sync, we need to add some extra repositories to the build as presently the Desire HD isn’t included “out of the box”.

Create a file, .repo/local_manifest.xml with the following contents

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project name="CyanogenMod/android_device_htc_ace"
path="device/htc/ace" remote="github" />
<project name="CyanogenMod/android_device_htc_common"
path="device/htc/common"
revision="refs/heads/gingerbread"/>
<project name="CyanogenMod/android_device_htc_msm7x30-common"
path="device/htc/msm7x30-common" remote="github" />
<project name="koush/proprietary_vendor_htc"
path="vendor/htc" />
</manifest>

Next step is to start the initial sync, then go do something else for a few hours while it grabs a lot of code for you…

repo sync -j16

Now we need to get some prebuilt files from the CyanogenMod Project.

cd vendor/cm/
./get-prebuilts
cd ../..

At this point the codebase will build but will not boot. To fix this, you need to edit the file device/htc/ace/media_profiles.xml as follows. [Patch submitted here http://review.cyanogenmod.com/13069)].

Insert @ line 77:
< !ATTLIST VideoEditorCap maxPrefetchYUVFrames CDATA #REQUIRED>
Change line 250 from
maxOutputFrameHeight="720"/>
to
maxOutputFrameHeight="720" maxPrefetchYUVFrames="10"/>

Once all that has been done you can start building!

. build/envsetup.sh
brunch ace

It will take a while, so sit back and wait…

NB
The build results in a ROM that runs and is stable enough for everyday use, but there is no camera support! Neither picture nor video work in the current build :-(

Posted in development, mobile | Leave a comment

Goodbye Ubuntu

Posted on Monday, 30th January 2012 by david

It’s time. I’ve finally had enough of the workarounds and issues thrown up by Ubuntu’s change to Unity.

So, Linux Mint looks like it will tick most of my boxes and appears to have a growing community around it (quite a few of whom appear to be disgruntled Ubuntu users – odd, eh?). I’m downloading the ISO and after getting a suitable backup done (I need to buy a new external disk) I’ll do a full install to start with a clean slate.

I did try installing the Mate desktop into Ubuntu 11.10, but it failed

There is a problem with the configuration server
(/usr/lib/MateConf/mateconf-sanity-check-2 exited with status 256)

After backing up my existing installation I used the ISO to clear the partition that Ubuntu had been installed in and install Mint. The process was easy enough and worked as well as I expected it would. When it started things looked more familiar than I expected, but after some checking that was because the desktop was Gnome 3. Changing to Mate brought things back in line with my expectations and I could start configuring things to be the way I wanted them. As Mint is based on Ubuntu, using it with the Mate desktop results in a very familiar look/feel – a little like an older Ubuntu release.

However, it’s not perfect and has left me wondering if I’ll be a Linux user for long. For example,

  • After starting Eclipse I was a little alarmed that the panels I have at the top and bottom of my screen vanished! Further investigation shows they are still there, just invisible for some reason :-(
  • To edit the menu you need to install an additional app (alacarte) that isn’t included by default. Surely this something that should either just work or the app should be available from a clean install?

Such issues really shouldn’t be an issue in 2012. I realise that Mint 12 is a new version and still in development, but these problems on top of all the issues with Ubuntu have really led me to wonder about my computer usage.

Posted in General | Leave a comment

Timelapse Processing

Posted on Friday, 13th January 2012 by david

I’ve always enjoyed creating timelapse movies and with newer digital cameras it’s easier than ever. I bought Rosie a GoPro Hero2 camera at christmas and this has a dedicated interval shooting mode that works very well. My D300 also has an interval mode, but is limited to 999 frames – a limitation that the GoPro doesn’t have. In fact the biggest limitation of the GoPro appears to be the battery! As an experiment this afternoon I stuck the camera to the outside of one of our windows (the suction attachment works as well as reports say) and set it off on 5 second intervals. A little over 3 hours later the battery finally gave up after 2,297 frames had been captured. At around 5.1M an image, that’s a whopping 11.44G of images to process!

First step is to reduce the image sizes. Maybe I should actually shoot at a reduced size in the first, but in this instance it was at full quality. Having copied them all to the hard drive, it was time for a bulk resize

for i in $(ls *.JPG); do convert -resize x1080 $i re_$i; done

After a suitable period of whirring I ended up with an additional 2,297 images sized at 1440×1080 (the originals were 3840×2880). It’s the first time I’ve done it with the GoPro so in future I may well adjust the sizes.

Next step is to actually try and create a movie form them. For this I’ve been using avidemux. It’s not 100% apparent when you first start using it how to do this, but a bit of digging and it turns out all you do is open the first file in the sequence and it imports the rest! Simple and straightforward. Again, more whirring and waiting and the preview appeared. The reduced image size meant the worked well and the movie was working well.

Avidemux is a great tool, but doesn’t allow for adding titles and an audio track (as far as I can tell) so the resulting movie needs to be edited in a different editor. I’ve yet to find an editor I really like for this, but have been using OpenShot and Pitivi.

This afternoon was just an experiment and much of the resulting footage isn’t worth uploading, but I have uploaded a small piece at the larger resolution and xvid video encoding, which is on flickr.

I’m still finding my way on the processing of timelapses, so comments, suggestions & help always appreciated :-)

Update
The output generated a file of 1.7G. Hmm, this seemed a bit large and after reviewing the video options I saw it was set to ‘Copy’. Changing this to MPEG-4 ASP (xvid) reduced the file size dramatically (to 35M!!).

Posted in Photography | Leave a comment

Building OpenWRT for Bifferboard

Posted on Sunday, 8th January 2012 by david

Since getting the bifferboard booting and discovering that the opkg python segfaults, I’ve been looking at getting a custom build of OpenWRT (that includes python) built. On the face of it this should be straightforward, but sadly it hasn’t quite turned out that way.

The bifferboard is supported by a small community and there is a lot of information available onine, but some of it is out of date and the board has seen enough development to make some of it irrelevant. Picking the right pieces isn’t as easy as it could be, but after some false starts I’m starting to get there. This is roughly what I’d suggest and is what I’ll do next time…

I started out looking at the code in https://github.com/bifferos/openwrt but this had issues even building whereas the repository listed below seemed to work better. YMMV

1. Checkout the code from github. https://github.com/bifferos/bb
2. Get the MAC address from the label on the bifferboard. This will be printed as 00b3f600xxxx.
3. Go to the ‘bb’ directory where you checked out the repository in step #1.
4. I’m not interested in QEMU, so won’t bother using the supplied Makefile. Hence cd openwrt
5. The kernel you build needs to have a kernel command line set or it’ll hang (this was something that took me a while to figure out), so

make kernel_menuconfig
Find the line, Processor type and features --->
Press Enter key to enter the menu
Fine the line [ ] Built-in kernel command line (at the bottom)
Press space so it is marked as [*] (if it’s already got [*] then just leave it)
Go to the line Built-in kernel command string (you may need to press down arrow to find it)
Press Enter key and enter ‘console=uart,io,0x3f8 rootfstype=squashfs,jffs2‘
Use the right arrow key to highlight Exit and press Enter key to return to main menu
Highlight Exit and press enter again, setting the changed settings as default.

6. At this point simply build the kernel to make sure all is well.
make
7. If it completes OK, you should find the images in the bin/rdc directory. The one I used was
named openwrt-rdc-jffs2-64k-bifferboard.img

With that done, it’s time to try and get the image installed onto the bifferboard. I’ve installed via the serial cable and network, but the network method seems easiest and is certainly the fastest, so that’s what I’m describing below.

1. Have the board connected to power and network, but switched off.
2. Assuming you haven’t changed directory from the above instructions, run this command
sudo ../tools/bb_eth_upload8.py < if> < MAC address> bin/rdc/openwrt-rdc-jffs2-64k-bifferboard.img

< if> is the network interface on your computer, so for me it was wlan0 but could well be eth0 or something totally different if you’re not on linux.
< MAC address> is the MAC address of the board that you recorded in #2 above. (I added the ‘:’ but looking at the code you should be OK just using the number.)
You’ll need to be sudo as the script opens a raw socket and most systems don’t allow regular users to do such things.

3. Switch on the bifferboard. This needs to be done within 10 seconds of starting the script or things won’t happen. This time limit seemed sensible and wasn’t too hard, but it makes sense to be aware of it before trying for the first time :-)
4. If all is well the number will count up and the image will be flashed to the bifferboard!
5. The board will automatically boot into your new image, so no need to power off/on. (Nice touch).
If you have a serial cable plugging it it and monitoring the boot is useful, but if you don’t wait around 5 minutes to give it time to complete.

telnet biffer

If all is well you’ll be in!

Some things I’ve found useful along the way,

  • to record a serial session via minicom, run minicom -C
  • from time to time I had problems with bb_eth_upload8.py seeming to work but not sending any data. Not sure what it was, but waiting a few minutes then trying again fixed it
  • after a few different configurations of OpenWRT a make clean && make was needed to get an image that would install

I’m still playing with configurations to get everything working as I want and will try and post more details when I have it working :-)

Posted in General | Leave a comment

Bifferboard

Posted on Saturday, 7th January 2012 by david

This post is another of those “this will help me more than you” ones, but you never know, it may help someone other than me!

Today marked the first time I had plugged the Bifferboard into the network and after finding the first wall socket wasn’t connected and moving to the one next to it, the network led showed life. All was good. Next step was to connect to it.

I knew it had DHCP but didn’t realise that they now set a hostname of ‘biffer’ so I went and looked at the DHCP table of our HomeHub to find the IP address. D’oh! All that was needed was

telnet biffer

Et voila, the response came back

=== IMPORTANT ============================
Use ‘passwd’ to set your login password
this will disable telnet and enable SSH
——————————————

BusyBox v1.17.2 (2011-02-06 12:53:15 GMT) built-in shell (ash)
Enter ‘help’ for a list of built-in commands.

_______ ________ __
| |.—–.—–.—–.| | | |.—-.| |_
| – || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
Bifferboard port (forked from r23226) ————-
—————————————————

Things were looking up!
Hmm, so how much space do I have?

root@OpenWrt:/# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 6.9M 2.8M 4.2M 40% /

Things were looking up!

I’d like to run a small data collection on the box which is in python, so python is a requirement not presently available. Time to figure out opkg.

opkg update
opkg install python-mini

The install went OK and completed after installing zlib (a dependancy). Next, check if all is well

root@OpenWrt:/# python
Segmentation fault

D’oh! That wasn’t really what I wanted to see :-(

Looking at dmesg didn’t show much of help. I’d already looked at Graham Jones blog posts about the bifferboard, so decided to follow his lead and build a new OpenWRT image. Of course as it’s a custom image I can specify what should be built and so hopefully avoid having to install additional packages. Thankfully the good people in the Bifferboard community have made it easy to do this.

https://sites.google.com/site/bifferboard/Home/openwrt-git

As you can imagine, my laptop has a full set of build tools already installed, so after following the instructions, selecting a few additional packages the build is wandering along. Having done this sort of build before the pace isn’t a surprise, but if you’ve not done one before it can seem glacial. Hopefully it’ll be done later tonight and I can try flashing it onto the box.

Posted in General | Leave a comment

Virtual Mail Aliases

Posted on Wednesday, 13th April 2011 by david

When setting up my current servers email system, I managed to get almost everything working via SQL tables that allowed me to use a web interface for the day to day administration. It also allowed users to control most aspects of their accounts, resulting in very low administrative load. The one thing that I never quite managed to get working was setting up more complex aliases – until today.

My configuration provides redirections and aliases as seperate concepts and entities. A redirection is simply redirecting mail from one email address to another with anything more complex being an alias. Users of the system can easily create redirections for addresses they control, but aliases are only created by administrators or those given permission. The distinction has worked well.

Getting the redirections working proved as simple as adding a mysql query to the virtual_alias_maps setting in the main.cf file for postfix. The mysql file looks like

user = ???
password = ???
dbname = ???
query = SELECT destination FROM mbox_redirection WHERE address='%s'
hosts = unix:/var/run/mysqld/mysqld.sock

However, getting the full aliases to work took a bit more effort. In essence I want to replicate the functionality of the /etc/aliases file using mysql. The initial problem is that with a pure virtual setup, as we have, the contents of /etc/aliases are meant to be used by the postfix local daemon, not the virtual daemon that we rely on for the majority of our requirements.

Adding entries to just the local maps will result in an unknown recipient error as they won’t appear in the virtual maps. This means we need to have 2 entries for each alias – one for the virtual map and one for the local map. Adding every entry twice seems like a lot of work, but there are ways round it when we think about how the maps get their data.

The next poblem is that we need to bridge the divide between virtual and local map. This means we need some way of configuring a virtual address to point at a local address. The solution I settled upon was to use localhost for local addresses, so that blah@localhost would point to a ‘blah’ in the local map and would be accepted as a valid redirect internally. To accomplish this I needed to add localhost to the mydestination setting in main.cf, 

mydestination = $myhostname, mail.$mydomain, localhost

Creating a table for the aliases in mysql resulted in something that looked like this

CREATE TABLE `mbox_alias` (
    `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
    `name` varchar(50) NOT NULL,
    `address` varchar(75) NOT NULL,
    `alias` varchar(1024) NOT NULL,
);

In the vitual_alias_maps I now need to map from an incoming address (address in the above table) to a local address that will then be mapped to the alias (called alias in the above table) I have provided. This I do by using the name field, using the mysql map file shown

user = ???
password = ???
dbname = ???
query = SELECT concat(name, '@localhost') as destination FROM
                             mbox_alias WHERE address='%s'
hosts = unix:/var/run/mysqld/mysqld.sock

This will map address to name@localhost, which will cause postfix to look for name@localhost in it’s local maps. To provide a valid mapping I added the following map file to alias_maps in main.cf

user = ???
password = ???
dbname = ???
query = SELECT alias as destination FROM mbox_alias WHERE name='%s'
hosts = unix:/var/run/mysqld/mysqld.sock
alias_maps = hash:/etc/aliases proxy:mysql:/etc/postfix/mysql/aliases.cf

As sometimes an example makes things simpler (my simple mind often needs one),

  1. Create an entry for an address of myscript@example.com, with a name of myscript and an alias of |/home/me/scripts/hello
  2. Email for myscript@example.com arrives
  3. Virtual lookup resolves myscript@example.com into myscript@localhost
  4. Local lookup for myscript find the alias |/home/me/scripts/hello
  5. Local daemon pipes message to /home/me/scripts/hello

With this in place, I’m able to add aliases via the web interface and have them available immediately. If I need more complex redirections, then I can add a regex file with patterns and redirect those to @localhost addresses in the same way.

There are likely better ways of doing this, but this took me a while to figure out and so maybe this will help someone else.

Posted in General | Leave a comment

NoSQL

Posted on Saturday, 26th March 2011 by david

NoSQL databases have been around for a while and they offer a lot of attractive features. I’ve been meaning to have a look at CouchDB for a while and having some time I’ve spent some time over the last few days trying to get a basic webapp running using CouchDb and CouchApp. I’d like to say that I’ve been impressed by how easy it is as that would fit with the hype, but sadly I’ve found it annoyingly difficult to get a handle on.

The issue isn’t with CouchDb which really has been as simple to install and setup as I thought it would be. It took me a very short time to install, configure and then write a small script to import the dataset I wanted to use for my test webapp. The Futon interface works as well as others say it does and overall I was very impressed. The CouchDb guys deserve a large amount of praise for their efforts.

The next step is to figure out some way of using the data for a small webapp. CouchApp is the one that gets the most attention and praise, so this was where I went next. Installing it was no problem. Starting a new app was easy and after a simple push I had the sample webapp on my screen. Of course the next step is to start modifying it to do what i wanted and this was where it all went wrong. CouchApp seems to have undergone radical development in the last year and so much of what you find on the web is out of date and no longer works. The sample apps that you get pointed to have little documentation and aren’t clear.

It’s been annoying as CouchDb looks really useful, but CouchApp just doesn’t seem to tick any of the boxes I need and none of the alternatives look as though they do either. If anyone has suggestions, drop me a line :-)

Posted in ASF, development | Leave a comment

This is not the Android you want…

Posted on Saturday, 19th March 2011 by david

Compared to Apples “one size fits all” iOS, Googles Android comes in a myriad of different guises. By choosing to partner with different firms they have ceded absolute control over how their operating system will look to those how manufacture phones. I’ll leave it to others to debate the relative advantages and disadvantages of this approach, but the result is that my current phone ships with a customised version of Android. If the customisations were only skin deep then they would be easily removable, but the effort put into the Sense interface by HTC means it has its fingers in all of the pies. If this meant my phone worked better I wouldn’t mind, but I find the changes to be obstructive and unwelcome.

One of the features that makes Android stand out from iOS is that new releases and updates can be delivered Over The Air (OTA) thus allowing the mobile devices to remain mobile and untethered from computers – freedom their iOS cousins can only dream of. The theory is great and when a new update is available the process works exactly as advertised, but the updates need to be created by the phone manufacturer to ensure that all their proprietary drivers and software is compatible. Linux kernel modules need to be rebuilt when the kernel changes and changes in Android itself must be incorporated. This all takes time and effort which means that older or less popular phones tend not to receive updates while current models wait months after the official release.

Android has an open source relative, the Android Open Source Project (AOSP) which appears to be a slightly backdated copy of code used for actual releases. The project pages give instructions to download the source and even build your own version, but after waiting for the large download (2.6 Gb) to complete and following the instructions you’ll likely feel a little let down for unless you have the most recent Google phone you won’t really be able to do much with the results of your endeavours. (With a slightly older Google phone you can extract the binary files you need, but only for the release you have installed on your phone.)

I find it hard to fathom the purpose behind the existence of the AOSP, as it currently exists, is. Releasing the amount of source they have is amazing. The efforts of
Jean-Baptiste Queru in updating the source and making available the source of each version within days of the official release are incredible and he deserves a lot of praise. One of the main beneficiaries of his travails has been the community that has grown up to supply custom versions of Android for virtually every device released. Using a mixture of cunning and determination people have found ways of extracting the proprietary files and merging them with custom built kernels and builds of AOSP to deliver new versions of Android to old and new devices. Given that these projects are based on AOSP, employ the same methodology as AOSP and do their development in an open source style (usually using forked versions of the AOSP source form github), why are their efforts not being incorporated into AOSP? The source trees of these projects contain various fixes and improvements that have been contributed by a large number of people keen to improve their phones. Surely such contributions would be better directed to the AOSP?

I started looking at AOSP as it appeared to offer a way of installing a version of Android on my phone that wasn’t infected by the Sense UI. After spending much of the last decade involved in sections of the open source world, I have a lot of respect the effort it takes to keep projects going and look for opportunities to contribute to those projects I use – an approach that isn’t dissimilar to many people I know. AOSP doesn’t encourage such activity because of the way it’s setup, which probably explains why there the community that exists providing custom versions is so large and seemingly uninvolved in the AOSP itself.

While I might keep an eye on AOSP, I think that directing my effort towards a custom project will be far more rewarding. Maybe they will create the Android I want?

Posted in mobile, Phone | 1 Comment

Kernel Building

Posted on Thursday, 10th March 2011 by david

First attempt at getting a new kernel for my phone, an HTC Desire HD.

  1. Download the source from http://developer.htc.com/
  2. Expand it into a directory
  3. As we’ll be building for ARM, we need the cross compilation toolchain, so install relevant packages.
  4. Get the config from the phone. 
    $ adb pull /proc/config.gz .
$ gunzip config.gz && mv config .config
  5. Now try and build the kernel. 
    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
  6. Answer questions for the config we’re using
  7. Wait…

This all looks good with the usual progression of messages indicating all is well, until

make[3]: *** No rule to make target `drivers/input/touchscreen/ntrig.o',
    needed by `drivers/input/touchscreen/built-in.o'. Stop.
make[4]: *** No rule to make target `drivers/media/video/msm/s5k6aafx.o',
    needed by `drivers/media/video/msm/built-in.o'. Stop.

The appropriate lines from the Makefile are

drivers/input/touchscreen/Makefile
obj-$(CONFIG_TOUCHSCREEN_NTRIG) += ntrig.o
drivers/media/video/msm/Makefile
obj-$(CONFIG_S5K6AAFX) += s5k6aafx.o s5k6aafx_reg.o

Looks like HTC missed some files from their kernel source archive :-( I’m going to send them an email asking for the files, but suspect there will be no reply as HTC have a long history of obstruction around their source releases.
e.g. http://users.livejournal.com/joshua_/41833.html

Update

HTC have replied to my email. Their initial response was

Thank you for contacting HTC.

In regards to your ROM query, I can help you with that.

We cannot support you on unofficial development of the ROM in any
case. 

I trust that this resolves your query. Please do not hesitate to
contact us again if required.

I replied explaining that all I wanted was 3 additional files that they had omitted from their source release, the following response was received,

My apologies, I misread you original email.

I will forward the details for the missing files to our developers.
Once they release the required files they will be found at 

http://developer.htc.com

Hopefully this means they will release the files, but we’ll wait and see.

Posted in mobile, Phone | 1 Comment

Android

Posted on Thursday, 3rd March 2011 by david

I’ve been using an Android phone since late 2009 and am now on my second phone. I’ve found Android to be easy to use and the over the air updates I’ve experienced from 1.5 to 1.6 and 1.6 to 2.2 have been smooth and effortless. In fact the only times my phone gets connected to a computer is for a charge when I’m travelling :-) Yes, my phone really is a phone.

Of course that’s not to say that all is prefect. My second phone is an HTC Desire HD, which works well but arrived with HTC Sense installed. Having been using “stock android” for a long time I find HTC Sense to be intrusive and would rather revert back to the “stock android”. Sadly, HTC have seemingly made every effort to make this harder than it should be. All of the advertising for current HTC phones features “HTC Sense” and they promote it as an “experience” (as opposed to merely a different interface) so I’m guessing that HTC aren’t about to easily relinquish it from their devices. Samsung have their own interface for their phones, but don’t spend quite as much time focussing on it during their adverts. A simple web search shows just how many people want to know how to remove HTC Sense, so I suspect I’m not the only one who feels this way.

I feel that all of these proprietary interfaces detract from Android as a whole.

There has been much written about the “fragmentation of android” – a problem that is likely to get worse as time goes on with the expansion in app stores for the platform – but the hardware vendors seem to be oblivious to the concerns as their policy of layering their own software on top of Android shows. It’s common sense that every custom layer requires time, and therefore money, to convert and update for a new version of Android – slowing the update cycle or even stopping it if the conversion is too hard (as was witnessed by some of the earlier phones being stuck on 1.5 or 1.6).

Google are justifiably proud of each release of Android. The collection of statues in Mountain View keeps growing and every release brings more functionality, speed and better integration. Sadly though, the gap between announcing a new release and updates being rolled out to phones has remained stubbornly long for most users – primarily due to the vendors custom interfaces. In an ideal world the vendors would have unlimited resources and enormous teams of engineers who could spend their time updating all their phones, but we live in the real world where money doesn’t grow on trees :-)

Hardware vendors want to sell phones and so having as many “differentiators” as possible is a good thing for them. Most phones in a given price bracket have virtually identical specification which means that these “differentiators” are becoming crucial as the market becomes more and more crowded. Choice is a good thing for the users and keeps prices down – the ability to make your product stand out is essential for sales and keeping the prices up. The two desires are opposite and at present the balance is a little too far towards the vendors.

I’ve taken the time and have rooted my phone and thus have the ability to flash new ROM’s, but given the complexities of merging the various pieces of software to create a ROM this also takes time and falls to groups of volunteers. The fact these groups can create a ROM quicker than the vendors speaks volumes for me about the complexity they have added.

Please HTC, allow me the option of having a “stock android” instead of your “HTC Sense android”.

Posted in Phone | Tagged , , | Leave a comment