Blog What I Made

So “State of the Map” is over for another year But it was a really good weekend! The standard of the talks was really high and the organisation of the conference in general was very well handled.

Though Saturday might have seemed to have a slightly more commercial slant (partly due to my own talk) I think in general it didn’t detract from the experience at all. As far as I can remember ITO were the only company launching a product and that product has obvious and immediate benefits to the OSM community. The main other commercial speakers - myself and Ed Freyfogle of Nestoria were talking about our own experiences of trying to use OSM data, and other crowdsourced data, and hopefully the lessons we’ve learned will be valuable for the OSM community.

Apart from this the various “State of <insert country>” talks were interesting as it’s really good to see how various countries are progressing in OSM, and it’s good to see how coverage is increasing at a generally rapid pace. I’m just noticing that there was no “State of Great Britain” which is perhaps a little odd. I suppose we assume that everyone knows what it is, when in fact many of the attendees were visiting from foreign countries (they’d have to be or we couldn’t have so many “State of <insert country>” talks!) My favourite had to be the Italy one which featured this video:

Apart from this there were also lots of talks about how people were using the data. Unfortunately this year there was no pulsing maps but work is continuing on renderering the data, there’s a number of people aiming to get routing services going with the data, and there was also newer uses such as Wikitravel’s Free World Travel Guides and more active development on uses with mobile devices.

Ed Parsons also talked about Google’s Map Maker, in general covering the same ground as his earlier blog post but offering more reassurances that Google was not out to destroy OSM and would most likely be willing to work with the OSM community in the future. He raised similar concerns on the current OSM CC-BY-SA license to those I made so hopefully we will be able to make a switch to an alternative license before too long.

The new license was mentioned a few times during the weekend but the difficulty of obtaining pro bono legal help has been slowing it down. It’s hoped that a second lawyer may take a look at it in the next week or so. Also Ed and I both expressed interested in getting Google and Multimap to take a look at the license too with a view to the companies using OSM data if the license is suitable. No idea what will become of that though. In Ed’s talk about Map Maker he also listed the countries currently being mapped by Google. The long list of Caribbean countries inspired Gervase Markham to set up an OSM fieldwork pledgebank. The idea is that if 60 people pledge to donate 10 pounds and an hour’s mapping effort then Grenada, and other Caribbean countries, will be mapped in OSM in no time, and hopefully a lucky one of the 60 will get to go to Grenada to do some OSM fieldwork!

Chaitanya has already blogged his weekend review and I thought he summed it up pretty well in his post:

The main reason I see OSM getting viral adoption and growth in the future is the pragmatism within the community. From the founders down everyone is already (only 2-3 years into its existence) asking what do we do now that we have (or very quickly acquiring) data? What applications can we build with this? What do we need to change to make OSM even better, make it a viable data choice for commercial uses ahead of the likes of TA and Navteq etc etc. Bravo!

morning after sotm 2008

Technorati tags: sotm, sotm2008, sotm08, osm, map, openstreetmap

Not really a review, just thought I’d mention that day one went really well, even without SteveC managing to get there for his keynote. My talk seemed to go ok although I rushed through it and then was told I didn’t have any time for questions as the previous talks had overrun. A few people have grabbed me for questions though which is good. I’ve also published my slides on slideshare.

As I say the day went really well, the theme of the day was “OSM in the real world” which meant that there was a slight commercial slant, but also a number of demonstrations of how people are using the data. This is one of the most important things about OSM, the open availability of the data which I’ve been intending to blog about sometime but not had time.

If you want more information about what was talked about you can take a look at the schedule, some reports here or Chaitanya’s blog where he’s been putting notes for each talk.

Technorati tags: sotm, sotm2008, sotm08, osm, map, openstreetmap

So I’ve now been up basically all night (5:14am, no intention to go to bed). Unfortunately I haven’t spent the whole night hacking, in fact the hacking task I was working on - getting the search API into the ruby library - was completed hours ago. I’ve actually spent the last 5 hours or so playing Rock Band.

We started playing cooperatively with two guitars and a drum set until around 3am when we got complaints about the noise of the drums (even though they’re not real, whacking some bits of plastic with real drum sticks makes quite a lot of noise!) Funnily enough though, the complaints came not from the sleepers but from the people playing Werewolf in a big group!

But back to the search API. I’ve added three new classes to the mmruby library: MMSearch, MMSearchFilter and MMSearchRequester. To perform a search you need to create a MMSearch object and then pass it to the MMSearchRequester class’s static search method:

s = MMSearch.new( { :data_source => 'mm.poi.global.general.atm', :address => MMAddress.new( { :qs => "L19 4UD" } ) } )
r = MMSearchRequester.search(s)

I have realised that there’s actually the routing API that also needs doing. It’s unlikely that I’ll get around to doing that today but I’m sure I’ll do it before long. I’ll post again once that’s done.

Again, not thoroughly tested just hacked together, any comments appreciated. Download mmruby-0.2 here.

Technorati tags: ruby, mashed08, multimap, web, service, REST

I’m spending this weekend at Alexandra Palace at the Mashed event that’s being arranged by the BBC. I went to the similar “Hack Day” event that they ran last year (and blogged about it) and had a great time. This time I’ll actually be there in a vaguely official capacity as Multimap and Microsoft are supporting the event. Multimap have already blogged about it but I thought I’d write a few lines too and release some software that might come in handy.

One thing that we’re really proud of at Multimap is our RESTful web service APIs. These give you access to geocoding and (faux) reverse geocoding services as well as access to our great search APIs.

As part of a project I’ve been working on in my spare time I’ve put together some ruby wrappers for these APIs so I thought I’d get these released in case they come in handy for anyone during Mashed. I’m not an experienced Ruby programmer so I can’t give any assurances of the quality but hopefully they’ll come in handy. You’ll need to have the ‘json’ library installed as well as Net::HTTP. I’ve only used it in a rails setting by placing the .rb files in the lib directory so your mileage may vary. I’d really appreciate comments, bug fixes and anything more you’d like to add. The libraries give you access to geocoding, national grid conversion together with a few handy geo-related classes (lat/lon, bounding box class).

Unfortunately I haven’t got around to wrapping the search API but it shouldn’t be too hard to do, I might even do it as a hack during Mashed. I might see if anyone’s added any comments before deciding whether to do it though

So download mmruby 0.1 here and tell me what you think of it below!

Technorati tags: ruby, mashed08, multimap, web, service, REST

Just published my roundup of the first day. Found it pretty difficult as I hadn’t taken any notes yesterday so I’ll try taking some today I think. I’m going to try something a little different today. I’ll publish this article in the morning then keep typing in it during the day, unfortunately WordPress doesn’t seem to be autosaving but I’ll click “Save” from time to time. I’ll also put something at the end of the article to mark it finished.

Micro, Nano, Pico formats
Talking about marking up the various formats with geo information: KML, GeoRSS, Atom. Formats are too big, they need to be smaller. Need to be able to store multiple locations and time information potentially. Discussing the best ways of linking to multiple locations, whether to place them as separate entries within Atom (which in old viewers will show the locations as separate entries), also whether we should insert links within text, or have separate links. Do those links link within a file, or to separate files? Probably the best place to look for updates on this part of the session is Andrew Turner’s blog.

Geohash from geohash.org, simple way of giving bounding boxes or lat/lon coordinates. Easy to search within a bounding box though will have issues as it’s based on predefined tiles - actually apparently not, this does not use tiling.

Guy from xrosspath is talking about “Geotudes” but they do seem rather similar to basic tiling mechanisms, quad-tree indexing and morton numbers.

Is geo privacy shit?
Mapufacture has enabled fire eagle to generate an RSS/KML feed. Andrew Turner thinks lots of people will decide “screw privacy”, Fire Eagle think that’s fine, so long as it’s a choice.

Keys to allow specific people to get access.

Privacy issues stifle innovation, caused issues with Dopplr. Can we trust the government? Being near a criminal offence might cause you to become a suspect (or witness). Location black lists. Nathan Eagle relationship mining - ContextAware. Regulatory frameworks. Infer location through friends. Went through the six de bono hats method. Here’s the notes that Andrew Turner took during the discussion.

Lightning Talks
1. Platial - some maps that people have done.
2. State of the Map
3. Earthscape
4. www.gotalift.com
5. Geocoding news (at the source)
6. NNDB Mapper
7. Fixables
8. Home-brew 360-degree display
9. Quantum Nanotechnocracy?
10. Neigborhood map project
11. Image recognition game - Imagewiki
12. Abaqus

Winners of the talks were: 8, 6, 1 (in that order)

Google AppEngine GeoStore
This was mainly a roundup of the geo features available within the Google AppEngine.

Are Google and Microsoft killing the ecosystem?
Much discussion on the topic, are Google et al killing the “mom and pop” size businesses? They are getting so big that noone can even start to compete with them because you can’t get near them. It was mentioned though that there wouldn’t have been the innovation there was so far without Google releasing the free maps API in the first place. There was also a small discussion at the end about whether Google could release data - India data and map edits - that they have collected themselves. Unfortunately not much information was known about those but the Googlers did seem open to doing something.

Well that’s the whole thing over, it was a great conference, I’m really glad I made it over. Some interesting discussions. As I mentioned above I kinda launched something, I’m going to get the tidying up sorted then post something further on that later.

Technorati tags: wherecamp, where, where2.0, where2008

So we’re now halfway through WhereCamp. I decided not to live blog the sessions as there was just too much discussion going on to keep up, and I wanted to be involved with these discussions which was a little hard if I was taking notes all the time. I’ll give a quick roundup of the ones I went to here but for more notes (and some alternative choices) you can check out my colleague’s blog here.

4D Mapping
This was a pretty interesting look at the ways of representing time-based map data. We discussed the use of sliders for filtering visible data, spirals for representing the passage of time and various other methods. Take a look at the wiki page I’ve linked to for more detail.

Mapmaking and Visualization with Processing
This session covered various visualizations that have been done using the Processing graphics toolkit. A few examples were shown together with a brief look at the source code, websites such as Cabspotting, Hackety and Obsessing.

Cyclone Nargis
I actually ended up in this talk by accident, I got lost on the way to “Kiev”. It was, nonetheless, very interesting to hear about the ways in which Burmese volunteers are trying to get aid to the people in Burma. A number of resources were also mentioned including Sahana Disaster Management System and Myanmar Cyclone Relief Donations.

Is 3D Shit
Steve Coast requested this session to discuss whether all the money being put into 3D virtualization was really worth it. Discussion from a number of interested parties including people from Google and Planet 9. We basically decided that the people involved were generally investing for the future.

Location Tracking
The location tracking session was actually organized by me, I’ve been tracking myself at various points for the past two years and have been building a site on-and-off for the past year so I was hoping to get together with some other people doing similar things. We managed to get people from Fire Eagle, Loki and iPoki. We had a good discussion about privacy, about ways to track and about requirements for accuracy but didn’t really come out with too many conclusions. I also kinda launched the site that I’ve been working on but it still needs some more work so I’ll do another blog post on that when I get a little more time.

Xrosspath
Xrosspath is a new site that is intending to take location history for a number of users and compare them to find points in time that they have crossed paths. They’re also looking to link in world events and historical events to see what’s happened around you during your life. It was interesting how this session had links to earlier sessions including the 4D mapping and the location tracking sessions. It did seem like they should be linking into what other people are doing in location tracking. I definitely think their premise is valid though and it will be interesting to see what comes out of that.

I wussed out and didn’t stay overnight, I also didn’t get around to doing much hacking. After a couple of beers I was pretty tired. I did end up getting to bed earlier than I’ve managed for the whole trip and got a great night’s sleep. Hopefully that means I’ll be all ready to get hacking today!

Technorati tags: wherecamp, where, where2.0, where2008

Erik Hersman

Talking about real world uses of our work, in terms of issues.

Grew up in Sudan, Kenya, now write two blogs “White african” “afrigadget”.

Excited about geotate.com very light weight, good in the field with unskilled people.

Also AfricaMap, very exciting as geolocation data was hard to find in Kenya.

Buglabs, any time you can hack hardware and software is seriously good. As more people use it, it’ll get cheaper and might get to 3rd world.

DIY Drones - great for crisis scenarios, how normal citizens can use it to help out

Illustrating that tools that we make are being used by people, perhaps with different background. Interesting over the coming years as GIS tools become easier to use, similar to how we see in CMS areas. We’re going to see something really big happen in the next few years.

Kenya Elections 2007 - issues and irregularities, opposition leader didn’t agree with the outcome. What started as apolitical fracas devolved into ethnic one. Showing slides of maps of polarities. People were kicked out of differing descents.

At same time, there was a media blackout, difficult as no way to get information out. Only way to get messages was SMS and phone calls, outside could only see through blogs and social media. Harvard law grad’s blog in Kenya got information out on her blog - kenyanpundit.com

“Our Goals”
* way for everyday kenyans to report incidents of violence
* create an archive of news and reports
* show here majority of violence was happening

“Building It”
* Detailed geospatial data is hard to come by in Africa
* How much should it be web-based in a mobile phone culture?
* Mobile phones - getting a full report in 140 characters is not easy
* What data points do we need?

We’re not part of humanitarian industry so don’t know what’s needed.

[Calendar of events]
Dec 27 elections
Dec 27 - 30 - Period of uncertainty
Dec 30 - Jan 1 - Media blackout.
Launched Ushahidi.com by end of January.

[Demo]

Only took a few days but really worked. Allowed us to do something not everyone could do. Timeline of events, see events occurring in the field. Draggable timeline and events on updates.

Had the beginnings of a crowdsource crisis information, realised this was pretty new, we were knew so hadn’t realised. So now decided what we do next.

Lessons Learned
* The importance of mapping accuracy
* Data poisoning - what happens when your antagonist starts using is?
* Verification is difficult
* Clarify why it was created and make that inescapably obvious - this was for rough data, not for ICC
* Create a feedback loop to end user

“So, did it work?”
* Advocacy? - Yes, mainstream media was affected and brought attention to the situation
* Security? Probably not
* Monitoring? Probably not
* Information Gathering? Yes, pretty well.

Formed by Erik and four other Kenyans, also funded now.

Types of activist, beer activist, anarchics, passionate about illegal immigrants, passionate about immigrants.

Activism always has two sides, both sides are passionate.

“It turns out activists are just everyday people, most with limited technical acumen.”

Going to go through several activist sites.

Crisis in Darfur google earth application, US holocause museum and amnesty international.

Sudan ICC war criminals website, warrants out for their arrests

Access Denied map from global voices, online censorship maps in closed countries.

“Tunisian Prison Map” - applying transparency to prisons in tunisia.

Bahrain land rights, showing difference between quality of life between haves and have nots.

Operation Murambatsvina - zimbabwe - showing land distribution, very heavy handed.

Mapping election conditions in Zimbabwe - taking news data about heavy handed government acts against normal civilians well before the election time. Showing that there was a track record well before the election happened.

The Great Whale Trail Map

Planet action - different environmental causes around the world

“I love mountains” - US based, enter zip code or state or city and see how you’re connected to mountaintop removal.

Mapping for human rights violations vs Mapping for activism - two separate things.

First is GIS/neogeo - this is what happened, used for taking criminals to court, second is to create awareness of buy-in of an issue

“Think about how you can use your skills to help in a cause that is important to you”

We have the ability to affect issues miles away which we couldn’t do not long ago.

Enemies Around Every Corner: Mapping in an Activist World

Technorati tags: activist, where, where2.0, where2008

Jonathan Lowe

Several companies have noticed the value in collaboration, OSM, geocommons, google base, others..

Structured databases means they can collaborate world

Basics of freebase

Web based database of community entered data managed by metawhere.

Freebases data includes but is not exclusively spatial.

Have seen three glue domains, business, people, location.

Began with points, 262,000 locations, exposing an API this summer. Mashups will be able to query freebase for spatial data in multiple formats as well as the other data.

Examples on openlayers, google maps, others.

“What is semantically structured data?”
* Strongly typed data
* Hardwired data relationships

Enter “olympic torch” into google, from results extract descriptive text, filter out the search term. Send results to engine that generates tag clouds, then show resulting tag cloud to people who don’t know the search term. Ask them to guess.

foundation cookbook bertolli shattuck grouemet california restaurant berkeley alice waters - What is Chez Panisse?
Semantic meaning gives a lot of information

Brains are very good at semantics, which classify or type data, and form relationships.

What were brains doing? Noticing Alice and Waters were names, saw “co-founder” - that’s people. Maybe Alice Waters is a co-founder. Saw “restaurant”, that’s a business, started by a co-founder. Might have noticed references to place. Put them together and get restaurant in berkeley, california, co-founded by alice waters to get Chez Panisse.

How to put this into a database?

Entries have multiple types of properties, properties have relationships. Business entry has geospatial data, as does alice waters. Berkeley has its own properties, these properties relate to the chez panisse and alice waters.

If I went to Berkeley and want funding perhaps these relationships can help me, freebase can show these.

“Relevance by faceted browsers”

Search in results for references to people, then kinds of people.

A Data Source to Make Mashups Correct, Complete, Relevant and Revisited

Technorati tags: mashups, freebase, where, where2.0, where2008

Adel Youssef

First, what is GMM? Maps for mobile phones.

“My Location” shows blue dot with big circle giving idea of how accurate your location is.

Not GPS accurate, but very useful.

Why is “My Location” useful?

GPS free service, Free! Saves battery. no problem with line-of-sight. Many applications benefit from thie accuracy. No waiting for first fix. Works across many carriers and network types.

* Collect geocontextual information along with a cell-id
* Cell Tower Identifier (cell-id)
* Location: GPS vs center of the map

Difficult to make it work across platforms and carriers, there is no unique ID across techs/carriers. How do we get location? If you have GPS cell phone, we collect that and the cell ID. We can also benefit from geo info like where you look at. Anonymous, just stores cell tower and location information, GPS or non-GPS.

We store this in our platform and run algorithms to figure out the location.

* 100s of different platforms - causes many issues
* Area of interest vs actual location
* Noisy data
* * Oklahoma points
* * GPS errors
* Towers in the water!

This approach can cause problems, if we’re all looking at SF maps the Burlingame cell tower will be identified as SF. Wherever you are, we often think you’re in Oklahoma because that’s the “center” of the US on google maps. GPS points can also be wrong if you have bad signal. Sometimes we find we have towers in the center of the ocean due to averaging, can be due to cell towers on oil platforms, which alters the accuracy. Sometimes that’s right.

Clustering Algorithm
* GPS Clustering vs non-GPS
* Use data diversity to calculat accuracy

Have invested more time in analysing the data. Data collection has been growing exponentially, working around the world, including the himalayas. Shows that this can work, non-GPS data is providing large amount of the data, more than GPS.

“Why doesn’t it work on my cellphone?”

We’re trying to get it working on as many as possible, some platforms don’t provide API to get cell ID. Some give part, so we do smart techniques to partly work it out. Some give multiple cell IDs. Others give full information.

Privacy

* A balance between respecting user privacy and providing good useful functionality to the user
* How does My Location do this?
* * Anonymous: No PII, no session id
* * User has full control, can disable or enable it.

What next?
* Improve accuracy and coverage
* Continue improving security
* Enabling location for 3rd parties via Android, Gears (browser)

Can use gears to enable it for your website or application. Build innovative location-based applications

www.google.com/gmm
code.google.com/android
code.google.com/apis/gears

Google Maps for Mobile with My Location - Behind the Scenes

Technorati tags: google, mobile, map, my-location, where, where2.0, where2008

Aimee Stewart

I work at the biodiversity institute at the university of Canvas, have background in geography, GIS, remote sensing, computing. Most recently worked on “lifemapper” - creating archive of ecological niche models. Maps of where species might occur based on where we know they’ve been collected by scientists. Also creating web services that expose the archive.

Showing what this looks like. Showing google earth. Showing specimen locations of a plant. Red parts are where we expect species to occur, yellow where we’ve found it, using ecological niche models. Can look and see that these came from Burke museum at Washington. Goals are archive of niche models, predictions.

Spatial data in OGC web formats, WFS coming soon, WCS for raster data, WMS too. No query analysis yet but coming in next month or so. Landscape metrics, fragmentation of habitat, comparison of habitats of different species, predicted future climates…

Also have on-demand models. Niche-modelling on a web-service on a 64 node cluster. Anyone can use this, our archive has limitations, no significant quality control, assume it’s been done by museums, but could be more done really. On-demand service can be used by researchers with their own data perhaps at higher resolution.

Niche modelling becoming more popular because more specimens becoming available. Environmental and occurrence data, put into model and it calculates formula, also project onto a map to help visualise.

Data is more available as there’s a massive effort to database and geotag it. Might be paper catalogs as that’s how it’s been collected for 300 years, now putting into databases to digitise collections. Also exposing data via the internet using standard protocols. Slide shows examples of 3 collection that when put together give more powerful collection.

Several scenarios on economic development, regionalisation, environmental aspects, modelled by nobel prize winners with Al Gore. We use multiple scenarios and compare them “apples to apples”.

Use this distribution data, put together with ecological variables through a modelling algorithm to get ecological niche model of where the species would occur. Using 15-20 variables. Model is output through projection tool to project is onto a map.

Specimen point data is taken to create a model using an algorithm of the current environment, projected back to get distribution in the native region. Done with climate models get distribution after climate change. Significance is looking at non-native region can see what areas might be vulnerable to invasion by species after climate change.

Archive created with pipeline, constructs request, passed to cluster on a 64-node cluster, web services in front, nodes retrieve environmental data, using WCS, node dispatches to openModeller and pipeline polls for status and retrieves data and catalogs.

[Demo]

Exposing data on website but also exposed on web services, can see in google earth.

Showing samples that we have the data in the database but don’t have lat/lon, have around 80% of those. ~220 institutions are providing data, within those about 600 collections, fish, mammals, etc.

Other ways to access data is to request data, experiment consists of all the data and the models and maps produced on top. Just URLs, can be accessed programmatically.

Other thing is “submit an experiment”, constructing search parameters and get a URL back with information for this experiment. Get really basic data back for it, shown projected on 4 different climate scenarios, current and 3 future ones. Showing metadata for the collection and other properties.

Lifemapper 1.0
* Distributed computing
* Screen savers
* Competitive like SETI
* Captured wrong audience
* Limitations

People weren’t really interested in the topic, couldn’t handle the demands of the audience.

Lifemapper 2.0
* Funded, cluster computing, OSS, standards

Lifemapper 2.0: Using and Creating Geospatial Data and Open Source Tools for the Biological Community

Technorati tags: lifemapper, niche-modelling, where, where2.0, where2008