• Week Notes Week 95

    May 22, 2010

    Again this week I’d been intending to work on a new client iPhone project but unfortunately it hasn’t been signed off by my client’s client, but that did leave me free to put more time into the chess app, or that was the idea (again, see last week). This week I managed two pretty solid days on the chess app and made some real progress. I’ve now got an area at the bottom of the screen showing the moves for the game and a set of buttons on the side for moving forwards and backwards in the game and for taking any alternative branches. I’ve created an internal wrapper for the game which models the game and you can send messages to it telling it to go forwards or backwards in the game, when it does this it also alerts any other objects observing it that the state has changed meaning I can automatically update the moves view and chess board view when the game is progressed.

    Something I’ve spent a fair amount of time on is making sure that all the rotation code works. It can be quite fiddly making sure that when you rotate the phone all the UI components end up in the right place. The SDK does provide you with various hooks that are supposed to make things easier but in the end I’ve largely had to manually set positions of elements to make sure they go into the right places.

    There’s still plenty to do on the app. To complete the elements I’ve done so far I need to add the ability to read comments that might be interspersed in the game, and to indicate better when these comments or alternative branches are available. When that’s done I really need to start working on something for managing the games that you have stored on your phone and allow downloading more, neither particularly simple jobs. I then also need to offer more functionality for reading through the game so there’s still a good way to go.

    On Monday, after having my old server’s disk fill up for the umpteenth time, I decided I might as well get the server move done and dusted. I’d previously copied most of the files from my old server to my new one so all that was required was configure all the services - the web server, database server, mail server, etc. - perform another sync, copy the databases across and then switch the DNS. It actually went fairly smoothly in the end and I think I managed to keep the downtime to a minimum. The final rsync only took about 5 minutes but then the database dump and import took about 10-15 minutes each. Wasn’t helped that every time I imported the database my connection to the server seemed to hang so the import would finish (the magic of GNU Screen) but I wouldn’t actually see until I tried to do something in the window and SSH finally told me that the server had disconnected.

    I don’t think I had any other real problems though and I definitely managed to get everything transferred before my app store positions script started running, so definitely within about 4 hours everything was ready. I’ve actually switched to running directly on the bare metal so I’ve gone from a virtual machine on an older server to running directly on a quite modern server. I’m glad to report that the app store positions downloads are now taking 50 minutes so basically half the time they took on the old server. Queries of the database also seem to be much faster which is really good, maybe that’ll help me to get around to writing some externally accessible query mechanisms, like a website.

    The rest of the time was spent on my revision apps. On Tuesday I spent some time preparing a press release which you can find here. It’s the first press release I’ve put out about the apps and is largely intended to get a bit more attention to them just as the exam period is coming around. The apps are doing quite well and seeing increases as we get to the exam period but I’d still like to see some bigger increases to be completely happy. On Friday I prepared a few more apps for submission, it’s probably getting a bit late now but I had the material ready so seemed a shame not to use it. One was for “Human Biology” for AS-level students - another Biology app but this time focused on the human side of things. The others were for the American market. I’m actually quite excited about this, it’s a huge market and has taken relatively little effort to prepare new apps tailored to US students. The questions aren’t too different so it just required some tweaking of the grading algorithms and then repackaging. Whenever Apple get around to reviewing them we should see Grade 10 and Grade 12 Biology, Chemistry and Physics appearing in the store. Again it’s probably a bit late in the school year but it will be good to see some healthy sales in another market if this works out.

    So that’s about it for the past week. Not sure what I’ll spend most time on in the next week, I do need to get version 1 of the chess app “finished” but I now have 3 other iPhone apps basically confirmed and needing completing by the end of June. Think I’ll be keeping busy for a good while yet!

  • Weeknotes Week 94

    May 17, 2010

    Well, this past week was supposed to be devoted to a new client iPhone app. “Unfortunately” I didn’t get sign-off for it but that did mean I could keep going with my chess app. With various other bits and pieces to cover I’ve managed to spend about a day, maybe two on that in the last week. Last Monday I also ended up finishing off the wordpress blog project for Clear Digital and then taking a trip over to Manchester. I had an interesting meeting with a potential client who want to do a really big iPad project. They initially wanted me for 12 weeks full time which actually spooked me a bit as I don’t usually do full time. I’ve yet to hear what’s happening with that but could be an interesting project to work on if they do want me to go ahead with it. After that meeting I met up with Dave Verwer and went to NSManchester in the evening. While at NSManchester I gave a very hastily put together presentation on the iPhone app store positions app that I worked on the previous week. Talk seemed to go well and I had time to chat with some interesting people in the pub afterwards too.

    I got home from Manchester at about 11:30 and began my planned server migration. My current server is hosted by Hetzner and is a “DS3000”, AMD Athlon 64 3700+ with 2GB memory and 2x160GB drive (probably, I bought mine over a year ago so specs may have changed). They now have an “EQ4” which offers Intel i7-920 quadcore with 8GB memory and 2x750GB drive for exactly the same price, though with a setup fee. I’m currently hosting mapme.at on one of these and it’s running really well so I decided to upgrade my other server too. On the older server I’ve been using VMware to host most of my stuff in a virtual machine. The idea for this was that when it did come to moving servers in the future (i.e. now) I could do it by simply copying the VM across and starting it up. By about 3:30am on Tuesday morning after wrestling with VMware and networking for many hours I was getting pretty tired. I got a few hours sleep (as much as my cat would let me before she decided she needed feeding) and then tried again in the morning. After another few hours I decided that VMs were not the way to go :-/ Considering I host everything on ubuntu and that’s super easy to set up anyway the ease of setup isn’t really that big a deal, also having to copy massive virtual disk images wasn’t proving to be fun anyway. I’m going to host my services on the bare metal which means I can switch from one server to the other by doing a simple rsync to get up to date, re-syncing databases and then switching DNS. Unfortunately I haven’t yet had time to do this, ideally I’d do it overnight like I attempted last week but considering the app store positions app keeps filling up the disk and knocking out services I’m worrying less about the downtime, it should be minimal anyway now it’ll just take some resyncing.

    I ended the week by attending the Liver & Mash event organised by Mandy Phillips. The event was in the spirit of previous “mashed libraries” events which have tried to introduce librarians to the concept of mashups and the many ways in which they can be useful. I wasn’t really sure what to expect from the event but had agreed to talk on “Maps” so prepared some slides and went along. In the end the event was really useful, it wasn’t really too dissimilar from other web/technology events I’ve been to. Everybody was really interested in mashups using various web APIs, hardware hacking with Arduino and other techologies, and pretty much anything that interests geeks. The libraries side of it gave it some focus but was easy to get to grips with for someone like me who has had minimal experience of libraries recently.

    Most of the talks were given in three tracks and the rooms were assigned by order of popularity. As it happened my talk on maps was voted most popular and I was asked to give it in the main room to everybody! The talk seemed to be well received, looking at the twitter back channel, with most people finding the various examples I gave interesting. I only had 15 minutes so gave some basic background on maps in general and where my experience of online mapping started. I gave some examples of using mapping APIs and OpenStreetMap then finished off with a quick overview of mapme.at and my experiences of tracking my location. As usual slides can be found on slideshare (usual problems with videos, though these can be found on my user page on vimeo).

    In the afternoon I also ran an hour and a half workshop on mapping. I’m not sure how well this went as we were in a fairly small room and I hadn’t particularly prepared any tasks for attendees to try out. I tried to go through some of the best ways to use mapping APIs (use mapstraction!) How to get involved with OpenStreetMap, how to edit the map using GPS, Aerial imagery or even the new Ordnance Survey data. I also covered the various ways to use OSM data including loading it into mapstraction or using the Cloudmade APIs to generate custom map styles and retrieve data through the web services. I got plenty of questions from my audience though and gave answers for all of them so hopefully they enjoyed it.

    All-in-all I think the day went really well. Unfortunately when my talk finished at 3:30 I had to rush out straight away and didn’t get to enjoy the evening revelry. Instead I hopped into the car to rush to London for a leaving party!

  • Week Notes Week 93

    May 8, 2010

    So, a much quicker update after last week’s delayed entry. This past week I’ve been working on a project for Clear Digital. A relatively simple project that required setting up a Wordpress blog and re-skinning it to match the client’s requirements. I hadn’t played with Wordpress so much in quite a while so it was an interesting experience. Turned out not to be too difficult, making use of plenty of existing plugins to extend functionality. On a recommendation from Dave Coveney I used the Thematic theme. This is more of a tool than an actual theme itself. The theme you get is very simple but it allows lots of hooks to extend the theme and customise it how you like. I think there’s lots of themes that are based off this but I chose to create a new theme building on top of the very basics that Thematic provides, the better to match the client’s requirements. Wordpress provides “widgets” which are small UI elements that you can drop onto the page in various places. Things like tag clouds, a calendar of your blog posts, a list of Categories, and lots of others. Thematic provides quite a few different places that you can drop Widgets making it even easier to customise your blog.

    In case it helps someone else, here’s a complete list of the plugins I used:

    As well as this project, I also started out on a new personal project last Sunday. I intended it to be just a quick thing to try something out but it’s started taking a lot more time and resources than I expected. As you may know I have quite a few iPhone apps in the app store. Right now I’ve got 22 live on my own account and another that I did for a client under their account. Though Apple provide perfectly good sales statistics they don’t give any indication of how well you’re doing in their “Top 100” charts. Though much of the desire to know your position is due to vanity there are some uses to knowing, you can use it for marketing and if you reach the top 20 it’s a good reassurance that you’re going to make a reasonable sum of money from sales.

    Apple don’t provide this information but a number of other people do. APPlyzer offer access to some of the data for free and require you to pay for more. An iPhone app called “PositionApp” also gives you some information and allows you to select favourite apps but still didn’t give me the information in a way that I liked, so I decided to write my own.

    I had already found a perl script that would download the information for the Top 100s and would give me information for a specific app, category and country if I wanted. I was originally running this twice a day but unfortunately I hadn’t updated it to list some of my latest apps so when I found that two of my apps were in the Top 100 in the UK Education category I decided I needed a better option. If I was going to download the Top 100s I really ought to be putting them in a database them so that I could do more with them in the future.

    I started by writing a script that would do the basic download of the XML and for some reason decided to throw the XML in the database for later parsing. Actually a large part of my reasoning behind this was having minimal time but wanting to leave something downloading data as I went off to OGGCamp. As it turned out storing the XML in the database was an incredibly bad idea, after a short while I had thousands of entries with 600KB of data in each meaning that an SQL query to request the latest download to check if it had changed took 15 seconds!

    So, version two, parse the data straight away. The data was in XML so obviously the safest way to parse it was to use a proper XML parser. Because the file was pretty big I decided to go with a SAX style parser. After spending a while doing this and getting a completed parser going, I found that my XML parsing was taking over a minute! I’d already noticed that sometimes the HTTP request from Apple could take up 15 seconds and doing that 5000 times (for all the categories and countries) was going to take a long time, so an additional minute was terrible news!

    Next day I decided to skip the “proper XML parsing” and go with a regex. After half an hour of coding I had something that would parse the entire 600KB file in less than a second, much better.

    I’ve now been running this script four times a day for nearly a week. I’ve downloaded approximately 20 batches of data in that time. Each batch is pretty big as I’m querying 40 categories in 62 countries for two types of app (free and paid), which comes to 4960 requests four times a day! Each of those requests then generates 100 positions entries meaning I now have over 10 million position entries in my database. This quantity of data has been causing its own problems but so far I’m keeping on top of them. Yesterday I added a few more indexes to the tables and converted the tables from InnoDB to MyISAM. This gave much better results. The 6pm batch yesterday took 5 hours to run whereas the midnight batch took 1 hour 45 minutes and the 6am batch took just an hour to run. I’m also coincidentally hoping to move to a more powerful server this weekend so that should help too.

    So, future plans for this data? Well basically I’m not sure how much effort I’m willing to put into it. The main thing that I want to get out of it is positions for all of my apps on some sort of regular basis, and the ability to query history for apps even if I haven’t specifically remembered to add them to my list. Other people might have other ideas of things to do though so I’m intending to dump the data out into some basic form, CSV most likely, and make it available to download. Hopefully I’ll get around to putting a web interface on this to allow people to look for information on their own apps or even register to get emailed position updates but any of that will be time permitting, and I’ve got lots of work to keep me busy!

    If you’re interested to know though, Basic Sat Nav is continuing to do well in the UK Navigation category, hovering around the number 10 mark and hovering around the 60 mark in Ireland. My GCSE and A-Level revision apps are doing nicely in the run up to the exams, none of them getting particularly high in the Top 100 but most of them making appearances in various positions. Even iFreeThePostcode is sitting at number 60 in the UK Free Navigation category.

    I’ll be talking about this project a little at NSManchester on Monday night so go along to that if you’re interested to know more.

  • Week Notes Week 92

    May 7, 2010

    I’m a bit worried things are getting far too samey here. This past week I spent a day fixing up a client’s buggy iPhone app and then another day working on another client’s PHP+JS property database. I also managed to get time to make some progress on an interesting client app that I’m not going to talk about in detail.

    Well, that’s how I started writing this blog post a week ago, and then other activities took over and dragged me away. I attended OGGCamp over the weekend which was really interesting but did take up plenty of time. I’ll continue below and hopefully get week 93’s post out a lot sooner!

    During the last week I did tweet a few times about chess moves so I’ll now admit that the interesting client iPhone app is based around chess. Obviously the rules of chess lead to some very interesting challenges while coding. For instance I had expected that validating moves was going to be very difficult to do. I think this was because I imagined taking a piece and a starting square and having to figure out all the places that it can go to. While this isn’t overly difficult it could be fiddly and time consuming.

    In actuality when it came down to it I found it wasn’t so hard. In my situation I have a starting square and a destination square and need to figure out if it’s valid for a given piece. This is actually much easier, or at least was much easier for my brain to work out.

    For instance with a knight, even though they have a funny way of travelling all you have to check is that they’ve travelled 2 squares sideways and 1 up or down, or vice versa, anything else is wrong. I also made sure that before doing custom moves per-piece I also did some basic sanity tests like making sure the starting square and destination were within the board and that the destination square didn’t contain a piece of the same colour.

    I’ve included a screenshot above to liven this post up but it doesn’t really show what the app does, in fact the app doesn’t do very much at the moment. I’ve been very careful to keep my code very modular so I’m actually hoping to use it in a number of apps, or at least do an iPhone and iPad version of the main app. For a lot of the code I’ve even been trying to stick to pure C rather than Objective-C as I believe that will help me if I try porting this to Android.

    Unfortunately I’ve had to put this app down again while working on another project this week which I’ll mention in the next post but I’m hoping to get back to the chess app again soon so that I can get it finished off and delivered to the client. I promise my next post will come a lot sooner, I’m already writing it in my head!

  • Week Notes Week 91

    Apr 24, 2010

    Another busy week but I finally made some progress on my interesting iPhone app. I got the first version of my parser working so that I can parse most standard versions of the file. One thing that has made this parser more “interesting” is something said to me at WhereCamp by Philipp Kandal: “If you’re aiming at Android and iPhone platforms, code in C”. Even though my client has made no particular reference to Android, a growing interest by other clients and by myself led me to decide that the parser at least should be written in basic C. Using no more interesting libraries than , and I've got my text database file reading in a character at a time and building up a (hopefully) elegant data structure based on linked lists. I've had trouble getting my head around memory management in C in the past but I'm hoping that my experience with Objective C is going to help me after my many years of scripting languages and garbage collection.

    For that project I also got going on some of the Objective C iPhone code too creating my own custom View class to generate the User Interface and a model to hold the current state of the app (currently in Objective C though I’m starting to consider moving this to C/C++ too). The view’s looking good already but I’ll be switching my current basic code out to use CALayers to enable some nifty animations later.

    I finally got Foursquare “push” support working too, so if you update your location on mapme.at and map yourself to a “place” we’ll try to find a similar venue on Foursquare and check you in on their site too. Currently matching is quite strict requiring that the location be within 200m and have exactly the same name but I’m looking to add some functionality to improve this by allowing people to manually match up Foursquare venues to OpenStreetMap places. I also need to add something to allow people to consolidate their old “favourites” so that they match the new OpenStreetMap places too. Not sure when I’ll do that but hopefully I’ll grab a few hours to do it in the next week or so.

    Monday was spent working more on the problematic client iPhone app again. A list of specific things that needed fixing, including a problem with API sessions expiring mid, er.. session. All of these were fixed only for this to reveal that the app works really badly on anything but a perfect internet connection leading me to book another day in for next week to try to sure up the resilience a little more.

    Friday involved a variety of small tasks but did result in a good Cathedral Valley lunch with local entrepreneurs, one of which was a designer eager to find a freelance developer to work with on future projects. I also had a meeting with a local agency which looks like it’ll turn into a good iPhone project so, though I didn’t get very much work done that day it may lead to some interesting future developments.

    Next week is looking like more of the same with a day for my client’s PHP+JS property database app to tidy up some bugs on that. I’m hoping that I’ll get my iPhone project into a more completed state where it’s fully usable for a specific purpose if not particularly good looking. Once that milestone is hit I can start looking at future directions for it with the client.

  • Week Notes Week 90

    Apr 18, 2010

    Well, this week was busy even though in some ways I feel I got nothing done! The week started on Sunday as I had agreed to work on a client’s iPhone app. It had been developed by someone else but they wanted some new functionality added. I’d suggested a day should cover it but did mention that things could take longer, I’d need to see the code before I was sure. As it happened of course things did take longer and once I’d started over the initial day on it the client had more features I could add so I ended up spending two days on that one. Unfortunately even with those two days there wasn’t enough time to track down some annoying bugs so I ended up spending hours here and there trying to improve things and will likely spend more time on it next week.

    Something else that happened on Monday was that a press release went out from Clear Digital about my new non-exec director post with them. I’ve obviously known about this for a while now so when I tweeted about it and got lots of congratulations back it was a really nice surprise. It’ll be interesting to see how things go with Clear Digital. I’ll be working with them to define the technical direction they should take on the various projects they do. I’ll also work to connect them with the developer community to help them find the best people for the job on those projects. Can’t really go into any of those projects right now but should be some interesting things to talk about in the future.

    Tuesday was my second planned day working on another client’s map based property database, went pretty well, got through most bugs and they want me to allocate more time later in the month to finish the rest off.

    I agreed to talk at the second Ignite Liverpool event on Thursday so spent Wednesday morning preparing my slides. The format of an Ignite presentation is fixed, you must have 20 slides that appear for 15 seconds at a time leading to an interesting experience for both the presenter and the audience! A (possibly defacto) tagline for Ignite is “Inspire us, just make it quick”, people also say that you’re supposed to talk about something which you’re passionate about. I couldn’t think of anything too inspirational though I did have a few quirky ideas. In the end time got the best of me and I ended up repackaging the Why I Track My Location and You Should Too talk that I gave at Where 2.0. Though I did cheat a little on timings at the end as the laptop we used for presentations couldn’t embed my videos, I feel I got the full experience of the strict timings. At times having to pause and wait for the next slide (“as you can.. will.. can… see!”), at other times running out of time and having to skip on. I feel it went well and as ever the video of my Weasley Clock seemed to impress people more than the rest of the presentation! Take a look at the review of the event on LDP Creative.

    I finally got around to submitting another three revision apps to the app store. A2 Chemistry Unit 1, AS Biology Unit 2 and AS Chemistry Unit 2 will all be appearing once apple have reviewed them.

    Friday morning I spent joining Nat Severs, aka Nomad’s Land on a walk from Liverpool to Crosby. He’s attempting to walk around the entire coastline of Britain for charity. He started around Portsmouth on the 10th of January and is now somewhere around Blackpool! He’s making great progress so I wish him luck for the rest of the trip and suggest you donate some money to his charities. Our walk took about 2.5 hours and seemed to cover around 7 miles. It was good to get out for a walk as I haven’t been out for about 6 months. It was a little tempting to join him all the way up to Southport but I had things to do and had to get home!

    Rest of the time this week was spent on more odds and sods. I imagine next week will be similar, I’ve already had to reshuffle my calendar to take a look at my client’s buggy iPhone app on Monday. After that I really need to get my head down and make some progress on the other app. Need to get a parser complete before I can do anything pretty.

  • Week Notes Week 89

    Apr 10, 2010

    This past week was a pretty short one as I was essentially travelling for two days (ok, maybe Monday was spent enjoying the sun (finally) in California a bit).

    View from Twin Peaks, San Francisco

    Most of the past three days was spent catching up on client work that I’ve been rather letting hang while I’ve been away. I’ve started working on a really interesting iPhone app that I’ve been looking forward to for about 4 months now, not going to say too much about it for now but I’m sure I’ll mention something when it progresses. It’s probably not all that interesting to the majority of people but has some interesting challenges which should be fun to work through.

    Apart from that I was just making some changes to an existing PHP map based property database and little changes here and there for other clients. For the next couple of weeks or so I’m hoping to spend most of the time getting some client work done and working on some iPhone apps. I might even work on some iPad apps as I picked one up in Palo Alto while I was in the US last week!


    OMG iPad!

    Not much else to report so I’ll finish off with some photos from my trip last week:

    Apple Store Palo Alto on night of iPad Launch
    Apple Store Palo Alto on night of iPad Launch

    Apple Headquarters
    Quick stop at Apple Headquarters

    Waves at Half Moon Bay
    Waves at Half Moon Bay

  • Week Notes Week 88

    Apr 8, 2010

    Hm.. doing week 88 a day before I should be doing week 89. Oh well, I’ll try to make this a quick one just to get it out.

    Last week was really busy with Where 2.0 and WhereCamp in San Francisco. The conferences went really well, met up with lots of old friends and made some really great new friendships. I got my talk finished and gave it to a good sized audience. The people I spoke to seemed to think it went well and especially liked the video of the clock (as usual) and the new visualisations I got ITO World to produce. I put a write-up of the talk over on the mapme.at blog.

    All trips taken in the past 3 years from John McKerrell on Vimeo.

    At WhereCamp I also got the opportunity to show my visualisations again, including the clock video and the graphs as well as the videos. This was during an “open mic” style session on geo-visualisation which was fun. Various people got up and showed what they’d been doing.

    I could probably have done more to get push mapme.at and make connections while I was out there, unfortunately I didn’t get any meetings arranged or anything like that, but I still think the experience was valuable. Hopefully I’ll get to go next year, I’ve already thought of something I can show at the WhereFair!

  • Coming Soon in iPhone OS 4.0?

    Apr 3, 2010

    … or “How do you solve a problem like Background Location Tracking on the iPhone?”

    This post has ended up pretty long and I realize that people of the Internet don’t like to read too much (TL;DR), especially when magical new Apple hardware is being released so I’ve highlighted the preamble below to allow you to jump straight to the amazing insightful ideas.

    <preamble>

    As you probably know if you read my blog, or are aware of my website mapme.at (and it’s corresponding blog) I like to track my location. I think it’s fun and useful. I use it to show friends and family where I am and what I’ve been up to. I use it as a personal record of where I’ve been allowing me to go back months and even years later and find out things like “yes, I was in Company B’s office on that date, they should have paid me”. Lots of other people like to track their location too using it to help with socializing, with fitness and lots of other uses. Hey even the government thinks it’s good to track people… but maybe we shouldn’t go there.

    I use lots of methods to track my location depending on how much accuracy I want to store. At the very least I try to “check in” to every “place” that I visit using the mapme.at mobile page which works well on my iPhone. I also like to store more accurate high resolution traces like the one below of a ferry ride through Sydney, Australia. These are great, especially when I’m taking photos or sending lots of tweets, allowing me to look back later and see exactly where I was when I did whatever I did. I tend to use an iPhone app called GPS Tracker from Instamapper.com to do this. It’s a very simple app and simply stores up locations for a minute before sending them to a server. They have an API which allows me to pull these into mapme.at and show my trace alongside the rest of my location history. The app’s great but having to launch it and keep it active to get a full trace isn’t ideal. Poking the screen every few minutes while driving to stop the phone locking isn’t the safest thing to do! It took me a good few months before I found out that if you were playing music in the background then the foreground app could stay active but even this isn’t ideal. Perhaps you want to listen to the radio instead. It also means that the only thing you can use the phone for is the tracking app, so if I’m walking down the street I can’t track and use twitter at the same time.

    Apple have been quite insistent on not allowing apps to run in the background (well, when I say insistent, there’s simply no way to do it in the SDK). The reasons they tend to cite (sorry, a cursory google didn’t find any official responses) center around the fact that running multiple apps at once causes unnecessary load on the CPU which causes the battery to drain too fast. As an iPhone 3G (not 3GS) user I know that battery life can be an issue on the iPhone, my aging phone sometimes has issues getting through the night!

    Not being able to run multiple apps has never been a huge deal for me, apart from this background tracking aspect. My previous phone, a Sony Ericsson K750i, couldn’t run multiple third party J2ME apps at once. It could run the current app in the background though and so has often ended up being a better option for tracking though does require me to carry two phones, this one and my iPhone (I can run 2 apps at once now, 1 per phone!)

    So you might ask why I’m writing a blog post about this, none of this is particularly new information. Well, a few months ago I came up with an idea that I was convinced solved the problem and, I thought, would be the direction Apple would take. More recent rumours about Apple simply “turning on” background/multiple apps in iPhone OS 4.0 made me a little less confident that Apple would go with the idea but I’ve decided that even if Apple do allow background apps, “we” in the iPhone SDK Geo developers community need to be a clever about how we handle these apps. As such I’m going to outline what I think(/thought/hope) Apple will do, and then I’ll discuss what we can do if they don’t.

    </preamble>

    The direction that I think Apple will take is inspired by their solution to the “How are we going to write Instant Messaging apps??” question previously asked of Apple. Their solution to that was to create a single central channel by which developers could send information back to the iPhone even when their app is not running. Developers use the “Push Notifications API” to do this. On the phone, when their app is first run it will request the ability to send push notifications, the OS will show a message to the user confirming that they want to allow this and the app will then be given a “device token” which uniquely identifies the phone. The app then sends this token to a server and that server now has the ability to send push notifications to the phone, using the token, whenever it wants. The Push Notifications API only allows for simple small messages but is an efficient way to allow apps to update their users without running in the background.

    So to do the same thing for location updates Apple should offer a single central API for requesting the location when the app is not running. I think Apple will call it something along the lines of “Mobile My Core Location Remote Notifications Me API” but for now I’ll just refer to it as “the API”. As far as users and iPhone (client) developers are concerned the API interaction would be very similar to push notifications. The app would request access to location updates in the background, the user would see a message such as:

    **"Ace Tracker" would like to use your location now and in the future** This will allow the app to know where you are even when it isn't running.

    If the user accepts, the app receives a token which it sends back to its server. On the server side I’m not sure how Apple would handle things, currently push notifications are done as a very simple SSL encoded binary connection. For background location updating the best solution would most likely be something based on XMPP so that the server could connect to Apple once and just sit and wait for updates. I’m guessing Apple wouldn’t want a polling option with multiple servers continually asking “where are my users?!”, and having Apple pushing to various different servers that may or may not be turned on and active is probably even less likely.

    So that’s the crux of it, Apple should offer an API that gives apps the ability to get location updates in the background. But still you might be thinking there’s a problem with this. Way up in my preamble I did mention that one of the problems with background apps is battery usage. Using Core Location continuously on the iPhone is one of the best ways to drain your battery so surely Apple aren’t going to make it easy for developers to do this? Well Apple engineers tend to be pretty smart and I reckon they’ll be able to find interesting and novel ways to reduce the battery usage. Ways that - if Apple choose not to go with an API and just to allow background apps universally - we should use ourselfes.

    Using Wi-Fi Positioning it’s possible to get a relatively good location fix just by doing a quick scan of the Wi-Fi airwaves. This might not work so well as you’re walking down a street as your position would keep jumping around by 50-150m or so, but if you’re sat in a bar or an office should give good results. How to know you’re sitting still? Well, one way is just to notice that the nearby access points haven’t changed for the past 5 minutes and decide based on that, another approach would be to use the accelerometer in the phone: if the iPhone hasn’t moved for hours then there’s no point trying to get a location fix at all. Again it might be possible to do similar things using cell based positioning, in cities the cells tend to be smaller which helps to get the accuracy higher. Doing some low power cell positioning to check for movement of more than a few hundred yards before deciding to turn on the GPS could be one way to save power. I’m sure there’s lots more options too.

    Obviously some of these techniques aren’t going to work if you want a 1-second high accuracy trace of your location, but if this is what you’re looking for then there’s not much you can do about battery life. A lot of interesting applications could be made using just simple background “checking in” giving a good profile of the businesses you visit on a regular basis for instance. While developers using the iPhone SDK don’t have full access to the hardware and so wouldn’t necessarily be able to optimise for switching between the different locating technologies, Apple do, and this is why I think they should create this API even if they do allow background apps. Even without full access to the hardware I think we as iPhone SDK developers should be able to create something much better than a hundred apps that all simply fire up Core Location, ask for best accuracy, and then constantly push that back to our servers and I think that we should start looking at ways to do this.

    Obviously decisions would need to be made, for instance for this to work optimally and not require multiple background processes for different apps there would need to be a single central service to send the location updates to. A central brokering service would be best. While I’d be ecstatic if mapme.at was used other choices would be Yahoo’s Fireeagle or potentially the new location services Facebook are rumoured to be introducing. Also while I talk about a “single centralised solution” I do realise that competition can be good so the idea of multiple implementations isn’t, in essence, bad I’m just trying to avoid everyone in the location space coming up with a solution which just results in multiple background updating processes running and defeating the purpose of this whole exercise!

    So that’s my idea. Might be a crazy one, who knows, might even be what Apple have planned anyway. A very knowledgeable friend seemed impressed though and pushed me to write up this blog post (and reminded me about the usefulness of the accelerometer) so maybe there’s something here. Let me know any comments you have, and I’ll be at WhereCamp in a few hours so would love to discuss it with you then. In fact I’ll be in the bay area until late Monday afternoon if anyone at Apple wants to get in touch ;-) If I don’t hear anything I might just write some code and stick it on github!

    Addendum

    I noticed looking back on my notes for this article that I listed to myself two things that make background tracking difficult: power and connectivity. Connectivity becomes a big issue as soon as you start roaming and would certainly be something worth looking at, having a way to store a trace for a long period of time and then report it back later would be great. I decided not to cover it in this article though as it was just getting far too big, maybe if there’s interest I’ll write something else up.

    Also, yes, Android already allows background apps, as far as I know there’s nothing similar to what I’m proposing for that platform, maybe there should be.

  • Week Notes Week 87

    Mar 28, 2010

    This week was spent getting everything prepared for my where2.0 talk next week. One of the things I want to do in my talk is show some interesting visualisation based on all the location data I’ve stored in the past three years. I started this on Tuesday by creating a page on MapMe.At that anyone can use to graph the amount of time they’ve spent in places over the course of a year. The graphs give really interesting results and can give some genuine insights. For instance they show that in 2009 I tended to go to the same places more regularly whereas in 2008 I went to a greater variety of places. I’ve also seen that I went to London more often than I thought last year.

    Once I had my graphs done it was time to work on my slides. Though I was originally a little nervous about whether I’d have enough material and spent most of Wednesday procrastinating I have ended up with nearly 40 slides which take me about 20 minutes to deliver. This is 5 minutes more than I have but I’m sure I can get it down, I’m just happy I have enough material!

    The final thing I spent most of yesterday on was adding in support for foursquare synchronisation. MapMe.At is all about offering as many ways as possible to store your location and currently foursquare is one of the most popular location tracking services around. They have an API so it was obvious to me that I should support it. That’s gone well and I’ve got full support for pulling your check-ins and “shouts” from foursquare into MapMe.At. I hope to get support for pushing your location into Foursquare done before my talk but I can definitely add another logo to my slides now! That support should be rolling out onto the site pretty soon.

    This week is the big week then. I give my talk at Where 2.0 actually quite late in the conference so I get to stress about it all the way through. Hopefully it should go ok but wish me luck!

subscribe via RSS or via JSON Feed