Another week spent on the Chess Viewer app. One of the features that I’m adding is support for searching within the chess games, specifically of the filenames and the headers that are generally attached to each game. In the original app I was reading all of this information into memory and storing it in a big in-memory data structure of arrays and dictionaries. This made the searching fast but also made it complicated and could result in having very big data structures in memory. To fix this I decided to use Core Data.
Core Data is the “Model” part of the Model View Controller paradigm in iOS and is the recommended way to store your data. It’s generally backed by an sqlite database so can be quite quick but as it’s an Object Model can be simple to work with from code without having to use SQL. I ported the code over to use it pretty quickly and then sent a build over to the client to try out. Unfortunately the app crashed for them soon after launching, in fact they couldn’t get it to launch at all. It turned out that as the code was parsing the chess files and building up the Core Data data model it was using excessive amounts of memory. It could also be really slow, some files (specifically The Week in Chess) can hold around a thousand games, each of those games will then have an average of ten headers each meaning you end up with around ten thousand headers to store. This alone causes the parsing to be slow and use memory, but if you have multiple of these files (and TWIC is updated weekly) it could cause real problems. This was mostly a problem on the iPhone 3G but it would be silly to refuse to support it.
In the end I decided to use sqlite without going via Apple’s Core Data model. Perhaps there was a way to optimise the existing code but I couldn’t find it in the days I spent on it. Considering the search function was supposed to be a half day task I really needed to move on! Previously I’ve tried using the sqlite functions directly, these are a real pain to work with so it’s fortunate that I came across FMDB quite recently. This is a library that wraps the C functions of sqlite with a nice Objective-C interface making things much easier to work with. Again I managed to port the code over to use this without too much difficulty… I still had to make various optimisations to make sure that the big imports don’t take too long and that the searching would be as fast as possible. It’s “funny” when spending so much time on something like this, it’s only really going to affect people upgrading to the latest version that have big files, as this is when the database will be built initially. When downloading files in the future it shouldn’t be so bad considering there’ll only generally be one file being processed at a time.
In case anyone else has issues I thought I’d paste in some snippets (uninteresting bits of code replaced with comments) to show how I ended up doing things:
So what’s happening there is every 5000 inserts for a file we’re committing the transaction to the database and clearing the auto release pool. This way we should hopefully not run out of memory but will run as fast as possible. You’ll notice there’s a memoryWarning flag too, this gets set separately by a notification, so if we get a memory warning before we’ve done 5000 transactions we’ll still commit the transaction and clear the pool to clear out as much as possible.
And, finally, more progress on my Arduino control system, I’ve got all the bits working now so that I can control my central heating and hall light from a website. The UI still needs a lot of work but it essentially matches the abilities of the original central heating controller and that’s the minimum I need in the short term. I’ll be putting some blog posts up with build logs soon.
Spent this week making improvements to the Chess Viewer that I released for Everyman Chess a few weeks ago. It was good to spend some time re-factoring an app that I actually started building about a year ago.
Monday was largely spent redoing the code for the “ChessBoardView” to use Core Animation Layers. I had to make some quite fundamental changes so that I could track the movement of the pieces but at the end of it I had a board that shows nice animations as pieces move around and are taken, and also added the ability to flip the board around to see if it from the point of view of the black player.
On Tuesday I added a feature to prompt the user of the app to review the app on the app store. Lots of apps have started doing this but I couldn’t find a simple small library to implement this, so I did it myself. The code for MKEReviewRequester is now available on Github, there’s some instructions in the README file but it’s fairly simple. Set some variables when the app launches, call a method every time a significant app action occurs (an ebook is read or a game is played) and then call another method at the point you want to display the alert to users. The code was all based around some code I found on this blog post, I just wrapped it together into a nice simple class.
Got lots more done on the app too but have run into a problem with preparing Ad-Hoc builds on my Xcode4 install so annoyingly I can’t show the updates to the client. Need to check whether I can distribute a different app and also try my old laptop, hopefully one of those will work.
Today (Friday) I’ve finally managed to get the memory usage on one of my Arduino projects down so that it fits within the 2K of RAM provided. I should hopefully be writing up some build notes on that with more details shortly.
I’ve had a busy week this week attending the various Social Media in Liverpool Week events that I wrote about last week. I just wanted to post an update to talk about the Social Media Cafe Liverpool iPhone app that I released last night.
Unfortunately I wasn’t able to declare the app live last night but the review went through and I can finally do it, head here to download the app and get lots of information about the great talks we’ve had at Social Media Cafe over the last year. You can find the slides for my Life Cycle of an iPhone App talk here.
As I mentioned in my talk, I have made the app available as Open Source Software under the Artistic License 2.0. What this means is that anyone can go away and create apps based on my code. The main requirement is that if you do so then you can’t modify the source code to hide the fact that I was the original creator, but read the license for confirmation on the details.
The source code is actually available in two parts. The Social Media Cafe app is quite simple and small and provides an app that will download a feed of information and then pass it on to a “Hierarchy View Controller”. This is part of the “HierarchyApp” which is a separate code base which I developed last year. I’ve also released this as open source under the same license so with both of these available anyone should be able to develop some interesting apps with a minimum of effort. Click the following links for more information:
While the data within the app can be updated by modifying a simple text file on a server there’s still plenty of features that the app could do with, which will require modifying the source code. It might be nice to have a page of information about the next event, or perhaps a page showing recent twitter traffic. If you have any ideas for features, or you come across bugs, then add them to the smcliv Issues page on GitHub and hopefully I, or anyone else who feels like delving into the code, will add them.
If you do use the code then I’d love to hear about it and see what you’ve done, and if you want me to help you develop an app I’d still be happy to get involved.
So, what have I been doing since November 27th?
- Secret Location Startup
- New version of Basic Sat Nav/Basic GPS launched 19th December with audio support and Spanish translations!
- “HTML5” and Android version of Examstutor.com apps almost finished!
- Bubblino & Friends iPhone app went from idea to submission in less than 24 hours, and is live on the App Store!
- CamViewer for Foscam Webcams goes live after a short dev time and gets hundreds of downloads and rave reviews on the App Store!
- After over a year “in development” Chess Viewer has gone live offering an advanced viewer for chess e-books popular amongst chess fanatics. Further development is planned including in-app purchase of e-books from Everyman Chess.
And all of this has taken me to the point where I’ve had 50 apps live on the app store! Not all of these are live right now as some of the Revise apps were replaced by Examstutor.com apps but I currently have 18 Examstutor.com, 15 Reviseapps.com, Basic Sat Nav, Bubblino & Friends, CamViewer, Chess Viewer, Credit Cards, iFreeThePostcode = 39 apps live right now and another one should be coming this week. Even discounting the carbon copies that’s still 9 separate code-bases, and again doesn’t include the app I completed for 7digital which Apple are unfortunately barring from the store. I’ve been busy and hopefully will continue to be with both client work and my own projects as this year progresses.
If you want me to help with a mobile web, or native mobile app, then get in touch by emailing me at john “At” mckerrell “dot” net
I’ll be posting an update of what I’ve been up to recently soon but I also wanted to write something about what’s happening in the coming week. We’re celebrating “Social Media in Liverpool Week” (we got told off for calling it Social Media Week™(C)®) and also Global Ignite Week with a week of activities in Liverpool. Getting into the spirit of things I’ve agreed to speak at two of them.
The first event of the week is Ignite Liverpool tomorrow in the LJMU Art & Design Academy. I was quite disappointed not to be able to speak last year (though I must admit I did enjoy my time on a yacht in the Whitsundays ;-)) so I jumped at the chance to speak this year. I’m going to be giving a talk on “Open Source Software Projects I Have Known”. I’m intending it to be a fairly lighthearted overview of various applications and projects that people should really be aware of and using. There’s lots of other talks lined up so if you’re in the area you should definitely come along. There’s not many tickets left so be quick!
On Wednesday there will be a meeting of the Society of Swandeliers, or the Friends of Swan Pedalo. In Autumn John O’Shea of Re-Dock procured a Swan pedalo that was being sold off cheap by Liverpool Biennial organisation. This meeting is of people who have shown interest in working with the pedalo to try to determine what to do with it. We’ve all got lots of ideas but a 6ft fibreglass swan isn’t so easy to move around so it can be quite difficult to arrange activities for it. I’ll be going along and hopefully we’ll all be able to work something out.
Thursday is time for another Social Media Cafe Liverpool event, also in the Art & Design Academy. I’ll be giving a talk on the lifecycle of an iPhone app. I’m intending to give an idea of the various stages you go through when building an app and getting it live on the app store. I’ll also have a few surprises during my talk too. There’s also going to be a live skype interview and more besides so head over to the website to sign up for that.
Finally on Friday there’s going to be the Social Media Social or “Oh, I follow you on Twitter”. More details on that can be found on the Social Media Cafe Liverpool website but essentially it’s a free party in the Leaf bar on Bold St. Entertainment will be provided (but you’ll need to buy your own drinks!) I’m going to a friend’s birthday party first but should be there later on in the evening.
So a busy week ahead, should be fun!
I’ve been spending a few days this week trying to get my Examstutor.com apps working on Android. I had already decided to do this using HTML and spent the past two weeks creating a “HTML5” version of the app. I was surprised but pleased about how quickly I managed to complete this, partly due to it always being easier doing something the second time. This HTML5 version uses localStorage a lot to store the test modules and various other bits of data. It’s probably not something that I would actually release as currently it stores too much data in localStorage, but I built it in such a way that I can build plug-ins that work on different devices meaning that I only need to recreate the device specific code for storing and retrieving data and the rest of the code that handles the interface should be the same.
ExamsTutorobject and then the plugin would be another singleton called
ExamsTutorDevicePlugin. To save having to decide which object’s methods to call,
ExamsTutorwould copy all the functions from the plugin into itself, as follows:
In that example I’ve already assigned
objis going to be the
After that code I would expect a call like
You might notice in that snippet that I’ve used
$.parseJSON. This was due to the final issue I’m going to describe here. For some reason the string objects returned by the Android interface don’t seem to react to the
typeofoperator in the way I might expect, and in the way jQuery’s
parseJSONmethod was expecting. The first thing that
$.parseJSONdoes before parsing is some sanity checking to make sure it has a string:
For some reason, calling
typeofon the strings returned by Android was giving
"object"and so this check was failing and jQuery was giving up on the parse. Fortunately this was simple enough to handle. I added a local parseJSON method to coerce the json to a string and also to handle the exception that might be fired by jQuery:
With that I managed to get the app to the point where I can select a Revision Pathway and have it download the information for that pathway and download the test modules from the Internet. There’s still plenty more to be done but as you’ll see from the screenshots it’s not looking bad already.
Last week I mentioned that we’d got the examstutor.com apps finished and submitted. This week they went live. Apple approved them all on Tuesday evening at which point they moved into the “Processing for App Store” state. Unfortunately it then took about 24 hours for all the apps to get through this stage as 6 of them ended up sitting there for ages. We finally got everything sorted out and put the apps live on Wednesday night. The apps have started selling straight away which is a great relief though not a huge surprise as the old apps were still selling ok. This is the first time in a while that I’ve had a free app on the store that I really care about the sales of so it’s interesting to see how the “Login” versions are doing. Currently they’re “outselling” the paid-for apps by way over 10 times but this isn’t a huge surprise. It’s good to see that people are interested though and hopefully some of those will convert to sales. Examstutor.com is now featuring the apps on the homepage and has a good set of pages describing the apps so it’s no surprise to learn that existing subscribers are already downloading the free apps and logging into them. If you want to see more information then take a look at the iPhone apps section on Examstutor.com
Getting these 18 apps live actually took me to 47 separate apps live on the app store, which I like to think - even if some of the apps are just different content - is a pretty good number. Because the Examstutor.com apps are intended to replace the old “Revise” apps I have now taken the A-level versions of those down but I’m still at 36 live apps. 17 of my own (3 not available in the UK), 18 examstutor.com apps and one Credit Cards app I did for Moneyextra.com. Many of these may be repeated but if we ignore the repetitions that’s 5 different apps I’ve got live on the store now and 3 others that are completed but waiting to go live for various reasons. Quite happy with my portfolio and hope I get to continue building interesting apps in the new year.
A quick update on my clock. I ended up deciding to bolt the motor onto the mechanism rather than using glue again. After various problems including broken drill bits I managed to get it secured and turning nicely. Unfortunately when I tried to get it working with the software it stopped working again. I’m now not sure if there’s a problem with my circuit board or somehow there’s a new problem in the software. I’ve had to leave it alone for a while though as I have other things to work on but hopefully I’ll get it going eventually.
I’ve also been working on a brand new start-up idea that someone came to me with a few months ago. I’ll be doing the bulk of the technical work including a Ruby on Rails back-end and supporting mobile apps. I can’t really go into too much detail about it right now but it’s hopefully going to be an interesting return to the location space for me. Watch this space for more info as things progress on that.
So these weeknotes are rapidly, or rather slowly, becoming “monthnotes”. The past few weeks have seen a few interesting things happen though so I thought I’d get another post up. Last week saw the completion of version 1.0 of the Examstutor iPhone apps. We managed to get the content for all of the 9 subjects we were hoping to support finished and so submitted 18 apps! We’re now just waiting to see what Apple will say, hopefully they’ll go straight through without any rejections but you can never be too sure. We’re submitting two apps per subject. One is quite simply a paid-for app, you pay your money and you get access to hundreds of questions on your subject. The other will be a free app in the store but is intended to be used by schools and colleges (and, in fact, individuals) who have signed up with examstutor.com for use of their online and mobile service. On downloading the free app they will be prompted for a username and password, when they enter this they will get full access to all the content “free”. We’ve included a few tests (generally around 20 questions) for people to use on the free app though so that if they download it by accident they still get to try it out and can then decide to upgrade if they want.
Last week I also managed to get the final few issues sorted out with my chess app, it’s looking better and hopefully I’ll be able to submit to the store pretty soon.
I also got most of the work done adding features to the Ruby on Rails site I mentioned in my last blog post. I needed to generate PDF invoices for a selection of addresses and services in the database and managed to get the work done pretty quickly. This was the whole point of converting from the awful PHP to a new codebase but I was still surprised by how quickly I got it done. I mainly need to make some changes to the PDF document to better match what the client wants and then let them try the system out and see what they think. I’ve used Prawn for the PDF generation and have been quite happy with it finding it pretty easy to use.
Final thing to mention is my location clock that has been quite popular on this blog previously. I originally built it at Howduino Liverpool in May 2009 - around 18 months ago. It has sat around with barely any updates since then and so was still made up of a big messy bundle of wires.
With the second Liverpool Howduino event coming up I decided I really had to get around to soldering this up, and I wanted to get it done before the event so that I could play with something else on the day (Xbee modules, lasers and a second four handed clock powered by servos were all on my list). Leaving it to the last minute I finally got started on building a circuit board on Thursday and did manage to get it finished off on Friday. Unfortunately I had a few issues with it, various connections connected that shouldn’t have been, and others that should have been were not. I then found that for some reason the software that had been working fine before had stopped working too. Fortunately I finally managed to get it all working on Saturday so now have a fully working circuit board and software.
So why am I not happy now? When I came into the office this morning I picked up the clock and started looking to see if I could get a chime to fit in the casing and where would be best to fit the arduino and cut holes for the ethernet, USB and power ports. I brushed past the motor and it wobbled, I thought “hmm, that seemed loose…. OMG, THAT’S LOOSE!!” After something like a year of being firmly stuck on by Araldite the stepper motor has now detached itself from the mechanism so my clock is still not working. I’m going to go out at lunchtime and see what glue I can find that might be able to stick a metal motor onto a brass mechanism, or possibly try to think of an alternative!
Wow, it’s been a while hasn’t it? I’ve been working hard the whole time so just not had time. Thought I should get a quick update out as some interesting things are happening.
The app I worked on back in July was for 7digital, a leading online media and music store. They’ve been talking about the app recently and announcing their mobile strategy. Unfortunately Apple have not allowed it into the store but we’re hopeful that it will get in eventually.
The revision apps I’ve been working on in the past few months are changing… I’m now working with ExamsTutor to build a bigger suite of apps selling to schools and individuals. This is great as it saves me a lot of work on the server side (even if it does mean my existing server-side stuff is now redundant) and also means I’ll have an established conduit to schools.
The chess app I was working on way-back-when is nearing completion and may be out by Christmas!
I’ve also been working on a project to rebuild a really simple PHP app with Ruby on Rails. The original system was very simple, essentially giving an admin interface to 4 tables, but was so badly written that adding any features was a complete pain. The migration to Rails hasn’t taken too long (though slightly longer than I expected really) and is now complete meaning I can begin adding some new features to the system.
I’ve started working with someone on a new location start-up, can’t say too much but hoping it’s going to be big and will be a good reintroduction into the geo-world for me.
We’re also still enjoying the office and have now achieved a full 6 days since the last explosion!
Just a quick one so that I’m not missing a week out. Last week I finally started developing the new iPhone revision apps. Development went well and progressed really quickly.
I also started planning for Howduino which is going to be returning to Liverpool (at long last!) on the 20th and 21st November. Howduino is a hardware hacking event inspired by the Arduino platform but encapsulating many other aspects of hardware hacking and “Internet of Things”.
I have a few ideas for what I want to work on, I’d like to do a new more capable version of my clock - ideally with more hands that are individually controllable. I’m also hoping to have a look at building a laser projector. Potentially for both ideas I’m going to need stepper motors so I put a request out on Freecycle/Freegle Liverpool requesting “broken printers”. I ended up getting 5 in the end. “Unfortunately” two of them seem to work fine so we’ve adopted them as office printers but the others will be taken apart for parts, I also managed to find an old SCSI scanner of my own to disassemble too as you can see in the photo above!