May 2011


I spent most of this week updating the Crown “Colour Match” iPhone app that I’ve previously made a few small changes to (I had nothing to do with the original codebase on this app). There was a few false starts while I waited for the graphics for the app, there was a few issues getting them through in just the right formats for the low and high res iPhone screens, but once I’d got them I did manage to make some headway.

The other thing I did was to launch the new version of Bubblino & Friends. This new version offers some great new graphics, new animations and audio that has been created just for the app. I’m really happy with the app and have enjoyed using it to follow some hashtag searches myself but I have to say that sales have actually been quite low since it launched, lower than I’d like anyway. I think that it’s not clear to people why they need an iPhone app to help them use twitter’s search features. As I say I’ve enjoyed using it but, as ever, people are just never sure about putting down that 59p/99c for something they don’t know they need. Hashtags are in use so much though, from the Eurovision Song Contest Final today (#eurovision) to episodes of The Apprentice (#apprentice OR #theapprentice) people are hashtagging tweets all the time and I do hope this app will become a great way to consume these. I can think of a few things that could improve the experience including finding a way to see more of the tweet text (without, of course, hiding Bubblino away!) with one obvious thing being an iPad version. I’ll be listening carefully to any feedback I get and pushing updates out as soon as I can.

Finally, this week we held two of our Maker Night hacking evenings in the Art and Design Academy of Liverpool John Moores. With a 7-9pm session on Wednesday and a 6-10pm session on Friday we all made lots of progress on the various projects we’ve been working on. On Wednesday I got more involved in the Cupcake 3D printer we’ve been building so that I could spend Friday looking after it (as Ross Jones who has led the build so far couldn’t make it). Friday night’s event was actually part of a wider event going on in Liverpool called Light Night with events occurring all over the city. This was great as it meant we had lots of visitors coming down and finding out about what we were doing and looking at our mini-exhibition of hardware hacks (we had my location clock versions 1 & 2, Mycroft’s Radio, Bubblino and an Internet connected temperature guage). While it was great to tell the visitors about what we were up to it did delay my work on the 3D printer! We eventually got the final few bits done and everything wired up and managed to test the plastic extruder and the platform stepper motors that move everything around. While it was great to have some success I’m afraid the most we managed to “print” was things like this. Next month’s Maker Night should be great as finally the first real objects should be printed, just a shame I won’t be there to see them! If you’re interested in going, head over to the Maker Night website.

Bit of a delay as I’ve been busy with other things but in this post I’ll be completing the set of YAHMS hardware by discussing the base station hardware and software. The base station has a few jobs to do in my YAHMS setup:

  • Physically connect to the relays via digital output pins.
  • Download the config for digital output pins and then control them.
  • Receive the XBee signals from the temperature probes.
  • Take samples from an on-board temperature sensor.
  • Submit samples received locally and via XBee to the server.

The circuit for this is fairly simple though, just connecting up some inputs and outputs. I mentioned connecting an XBee to an Arduino in the temperature probes post, we simply need to connect the DIN and DOUT pins up to the Arduino so that it can receive the information (and of course the power and ground wires). Because I like to use the main serial interface on the Arduino for outputting debug information back to the computer I’ve gone with connecting to pins 2 & 3 and using the NewSoftSerial which essentially means the serial interface will be provided by software. This isn’t ideal as in theory it means you’re more likely to miss data as it comes in, but the latest software serial drivers largely get around that issue by being interrupt driven. So I end up with the XBee power and ground going directly to one of the grounds on the Arduino board and the regulated 3.3V output, and then pin 2 (DOUT) on the Xbee is plugged into pin 2 on the Arduino and pin 3 (DIN) goes to pin 3 on the Arduino.

I have two NPN transistors turning on the relays (using transistors so that the magnet in a coil relay won’t cause a burst of current draw to the Arduino digital pin) which are plugged into two 3.5mm audio jacks. I have 5VDC power going onto the tip of the headphone jack, the sleeve connection of the jack then goes to the collector on the transistor and the emitter of the transistor completes the circuit by connecting to ground. The base of the two transistors go to pins 4 and 5 respectively on the Arduino.

I also decided to add a temperature sensor onto the board just because I had lots handy and to make sure that I had something to sample locally. The TMP36 is wired to the 5V power supply and then the VOUT goes to A5 on the Arduino to sample the temperature. See my temperature probe post for more details on TMP36s.

Here’s a picture of my original circuit diagram:

And here’s a fritzing version of that diagram:

And here’s a photo of the finished product:

The board I’ve used here is a prototyping shield that I got from @oomlout at MakerFaire. Being a shield it’s really handy and has two sets of connected track on the ends allowing power distribution. I couldn’t find a part for it in Fritzing so I’ve just used the breadboard piece and tried to arrange the parts in the diagram in a similar way to how they ended up, but with more useful spacing.

I had a few small physical problems with the board when it was completed. The pins on the headers on the prototyping shield are note long enough for it to properly stand above the RJ45 jack on an ethernet shield. You can kind-of push it down so that all the contacts are made but it’s not quite ideal. This also means that the RJ45 shielding will short circuit any connections it’s butting against on the prototyping board. That caused a bit of a debugging nightmare for me at first until I realised and applied some insulation tape! Things fit well if you put the ethernet shield on top of the prototyping shield but unfortunately the ethernet shield then didn’t work, I assume because it needs the ICSP connection.

The hardware here is pretty simple but I’ll do a parts list as usual for completeness. Thought I’d also include this cheatsheet which I drew on the second whiteboard in our office:

Part Quantity
Arduino Uno 1
Ethernet Shield 1
XBee (series 1) with Chip Antenna 1 Data-sheet
XBee breakout board 1
2mm 10 way headers 2 Sparkfun link
TMP36 (temperature sensor) 1 Data sheet
3.5mm Jack (PCB mounted) 2
P2N2222AG – NPN Transistor 2 Data Sheet

It’s worth noting that as YAHMS is completely configurable you can actually have whatever circuit you want using any of the digital output pins or Analog input pins, and choosing whether to use the XBee or not, my circuit is only really shown as a guide. In the future I intend to add support for digital inputs too.

Once that was all done it was time to write some software. As I mentioned in the first post I’ve actually open sourced the software for this so you can grab the source code for YAHMS from github and extend and fork it as much as you want. Unfortunately at the moment it has a few non-standard dependencies. The main thing is the new version of the Ethernet libraries that Adrian McEwen (@amcewen) has been working on. These will be part of an Arduino build in the near future but for now you can try getting them from his github fork of the Arduino environment. They’re really useful though as they provide DHCP and DNS support meaning no more fixed IP addresses! I’m also using a HttpClient library that Adrian has also written. This isn’t actually available properly from anywhere yet (but keep an eye on his github repositories!) so I’ve linked to a zip file of the version I’ve used below, that also contains a few other libraries that it uses. The final custom library is a version of this XBee¬†Arduino library that I’ve hacked to support NewSoftSerial instead of just the standard Serial interface, see the links below for that too. You’ll also need NewSoftSerial of course and the Flash library which I’ve used to decrease memory usage. Follow the instructions in “patching_print.txt” to patch the system Print library to support the Flash objects.

If you manage to get through the rather complicated compilation process for YAHMS you’ll find that you just need to edit the MAC address in YAHMS_Local.h and you’re ready to go. In theory you should not have to configure anything else locally once I have yahms.net working fully. Once running on an Arduino that sketch will retrieve an IP address via DHCP, synchronise the time using NTP and will then attempt to retrieve the latest config for the MAC address from yahms.net yahms.johnmckerrell.com (yahms.net does work but seems I’ve forgotten to update the source code). Currently there’s no way for you to put your config into yahms.net but hopefully I’ll get that up soon enough. Until that point you can edit YAHMS_SERVER in YAHMS_defines.h and use something on your own system.

Config is requested by a HTTP GET request to a URL like the following:

http://yahms.net/api/c/a1b2c3d4e5f6/1/00000000

Which breaks down as follows:

/api/c/ a1b2c3d4e5f6 / 1 / 00000000
MAC address of your ethernet shield, as registered with yahms.net Protocol version, currently ignored Time last update was retrieved, idea being that the server will return nothing if the config has not updated, also ignored and incorrectly sent as zeroes by the YAHMS code right now

And should return something like this:

YAHMS
A:5
O:4,5
X:2,3
C:30 6 * * * 60 4
C:0 7 * * 8 120 4
C:30 19 * * * 60 4
C:40 16 13 4 * 60 4
C:2 8 6 5 * 60 4
S:60

Which breaks down as:

YAHMS header, identifies this as YAHMS config
A: 5 comma separated list of analog pins to take samples from
O: 4,5 comma separated list of digital pins to use for output, controlled by control blocks given later
X: 2,3 Pins being used for Xbee as RX then TX
S: 60 comma separated list of settings, the first number is a number of minutes that this system is offset from GMT – the server is expected to handle daylight savings time and change this accordingly
and a set of control blocks
C: 30 6 * * * 60 4
pin 4 turned on at 6:30am every morning for 60 minutes
C: 0 7 * * 8 120 4
pin 4 turned on at 7am at weekends for 120 minutes
C: 30 19 * * * 60 4
pin 4 turned on at 7:30pm every evening for 60 minutes
C: 0 * * * 2 10 5
pin 5 turned on for 10 minutes every hour, every Monday
C: 0 14 * * 2 10 5 0
pin 5 turned off for 10 minutes at 2:00pm on Mondays, i.e. cancelling out the block on the previous line

The syntax for the control blocks is inspired by the syntax of crontab although YAHMS only supports either a number or an asterisk (*) for the numbers, and instead of a command the time is followed by a number of minutes, a pin that you want updating and the state you want it to have. An asterisk in a numeric column means that this column always matches, a number means “only use this block when this field has this value”. Here’s a breakdown of the fields:

Field Example Description
C: Identifies this as a control block
m 30 number of minutes past the hour (0-59)
h 6 hours since midnight (0-23)
dom 3 day of the month (1-31)
mon 5 month of the year (1-12)
dow 2 day of the week (Sunday is 1, 8 is weekend, 9 is weekday)
len 45 length of time the block is active, in minutes (stored as an int so -32,768 to 32,767 though it would be unlikely you’d use values that big)
pin 4 the arduino digital pin that you would like to control
state 0 This field is optional and is “on” by default, 1 is on, 0 (zero) is off.

So this would give us a very specific control block that turns pin 4 off for 45 minutes from 6:30am on the 3rd of May, but only if that day happened to be a Monday.

At any point in time it would be possible for a number of control blocks to be active for a pin. If any of these blocks specify that the pin should be off then this gets priority and the pin is turned off. The default state for a pin when no control blocks are active is also off. Generally you would only need to have “on” control blocks but the “off” blocks allow you to override, so you can have a block that turns the heating on every evening, but add a temporary “off” block to turn it off on a particularly warm evening.

For sampling, the YAHMS system will take a sample from any analog pins that you have requested on every cycle of the loop method. It will store the last 10 samples and then submit a smoothed value back to the server. For the XBee values it will check every cycle for an XBee packet (waiting for 5 seconds). If it receives a packet it will record a smoothed value from the samples that have been sent. Every minute the system will submit any samples that it has taken, and will then blank the XBee records. This way you won’t get continuous records from the XBee if it stops transmitting. The samples will be sent to the following URL as a HTTP POST:

http://yahms.net/api/s/a1b2c3d4e5f6/1

(Again that’s the MAC address and a version number in the URL)

The POST data is sent as Content-Type: application/x-www-form-urlencoded and will have a format similar to the following:

A5=0b8&X1P0=226&X1P1=039&X1P2=05f&X2P0=233&X2P1=18f&X2P2=18f&X2P4=0de&

Each sample is a three digit hex number (three digits is required to send to 10 bit accuracy). The samples from Arduino analog pins will have keys of A0-A5 and samples from an XBee are given as X, then the ID of the unit (the Arduino code will only support single digit IDs at the moment, i.e. 0001-0009 are supported), then the letter P then the XBee Analog/Digital pin that was sampled. So in the above we have a value of 184 from Arduino analog pin 5, then 550 from pin AD0 (pin 20 on the module) of the XBee with ID 0001, we have some more values and end with 222 from pin AD4 (pin 11 on the module) of the XBee with ID 0002.

The code for yahms.net currently requires Rails 2 but I’m thinking that I should update it to work with Rails 3 considering that’s been out for ages. I’m intending to do this update before releasing the code. If anyone particularly wants a copy then let me know and if lots of people do I’ll just release it anyway. For now I’ve linked to a small PHP script below which you can use to send config to your YAHMS system and store the submitted data in a text file. Below are the links to the other custom libraries too:

Do let me know if you find this interesting or useful! I really hope someone does and the more feedback I get the quicker I’ll release the yahms.net code and improve the whole system!

Quite a mixed week this week as I was expecting some materials from a client to come through to let me work on some updates for them but they didn’t come through until Friday. In theory it was also a short week due to the May Day bank holiday in the UK. As it happened I ended up spending Monday making some final changes to Bubblino, including the addition of bubbles! Such an important thing for Bubblino but I kept forgetting to add them. Really happy with how they came out, they’re randomly generated and individually controlled. At the moment the bubbles all float up which is a bit different to the real Bubblino, but they look good for now. I finally got the app submitted early on Tuesday morning so hopefully it’ll get accepted in the next few days and we can get it up for sale. The new version is going to be on sale for 99c/59p so it’ll be interesting to see if it generates any useful revenue.

I also spent some time this week trying to improve my App Store Positions service. I’ve noticed recently that the graphs it generates of positions have been fluctuating quite wildly. I’d previously checked that it was successfully completing all the downloads it does so I couldn’t really see what might be the issue and was assuming it was a problem on Apple’s end. As it turns out the problem is definitely on Apple’s end but I have been able to work around it. Basically Apple’s servers are sometimes returning the wrong result for the request I send, i.e. I might request UK Navigation Paid apps and get Venezuelan Navigation Paid apps back. While I would notice a response coming back with incorrect XML because it didn’t have positions data in, I wasn’t checking that what I got back was what I asked for. Fortunately I’ve been able to add a check in and am now retrying all requests that get bad responses. Unfortunately this means that my downloading is now taking about 1 hour 15 minutes when previously it was only taking 30 minutes, but at least I’m getting the right data now! I also improved the style of the emails it sends so that high performing apps are towards the top and the highest app store rankings show up first so I’m glad to be getting some nicer data to present there.

Apart from this most of the week was spent on little bitty things, some (hopefully final) changes to the Chess Viewer and answering various emails. Next week should be more active working on a client app, though that hasn’t started well as I’ve found that the image resources I’ve been sent through are all arbitrarily sized. I’m sure I’ve got a fun week to look forward to.