-
Where 2.0: The State of the Geoweb
John Hanke
Fourth Where 2.0 I’ve been to.
At their best mapping and geography make the world seem like a slightly smaller place. Opportunity for all of us… we should keep that in mind.
The idea that is shared by a lot of people that geography is a really useful lens through which to look at data about the world, that geography and maps are a really useful way to organize information we’re looking at. That mapping data provides context for the information once we find it.
We’ve been talking about things like the idea of a geoweb, that would allow us to search and interact with this level of geographic data.
Would like to talk about progress, collectively, that we’ve made. Also talk about some of the challenges.
May 2007 - Showing a picture of geographic information that was available on the web a year ago to now, showing over 300% growth in the places and annotations that they’re aware of. Much more densely covered. Not just where you’d expect, but also into the long tail of the developing world.
More and more rich media, millions and millions of geocoded photos. Panoramio, Flickr, youtube, 8 hours of video every minute uploaded, can be geocoded.
Almost drowning in a sea of geodata, though that’s a good thing. A while ago turned on geo search in Google. KML files. Incorporated into google maps. Can search across user generated and geocontent in addition to yellow pages. Made it much broader and globally relevant. Recently added to google maps mobile. Can render KML on phone(!) Much richer portal to the world around us.
Product announcement
Geo Search API… Launching Today!
Search box on google api will now search this geo data on the web too.
Mentioning “donating” KML to the OGC. Happy to say that as of a month(ish) ago it was officially voted in as a standard. Now owned by the community, managed by OGC.
Giving props to MS for supporting KML.
Kind of a dark web of geodata - the world of GIS. Thousands of servers full of geodata that we don’t have easy access to.
Stamen did a mashup of Oakland crime data but it had to be removed as they screen-scraped the data.
Reached out to ESRI - “leader in GIS” - over a million seats installed in their software, 50,000 servers, 250,000 clients use their data.
Welcoming on stage Jack Dangermond founder ESRI.
Slide on context showing geospatial applications growing, going from research communities through GIS professionals and enterprise users to consumers/citizens.
Hundreds of thousands of organisations, billions of dollars of investment and content management, can be, should be leveraged into this geoweb environment. Haven’t seen the potential realised to leverage this power. The geoweb is evolving, going to make a big jump, ESRI are engineering the 9.0 version of their software to plug in and become a support mechanism to the geoweb, providing:
Open meta-directory services, meta-data can be pulled off and integrated into consumer mapping environments, opens up to JS and flex APIs. Users can plug in GIS services directly.
Every piece of data on that server is going to be exposed on a HTML page that can be found by Google.
[Live demos]
City of Portland, searching for the Portland map server. Can pop up a metadata page which has been scraped off of the metadata service in the server. Can zoom in and look at the HTML page that has been scraped with lots of layer information available. Zooming in finds neighbourhoods, parcel data, sewer and water data. Whatever an organisation chooses to serve. Distributed services in the form of models and analysis. Can calculate on distributed server the drive times from an address and retrieve the polygon in KML. Can merge them with demographics, generate a demographic report. Can search and discover not just content but can reach in and get the live data. Important note, this is all coming out as KML.
Second demo showing “heatmap” of climate change. Next hundred years, being animated (slowly) off the server. Mid-west getting warmer. 5-10 degrees increase. Mashup with JavaScript mashing up analytic server with google maps.
Great that you can get information on a single site, but having it accessible on the geoweb allows it to be blended into a sea of mashups. These kinds of data can get out to a bigger audience. Big step forward. Integrating the geographic science with a collaborative environment. Should change our behaviour, if we’re fortunate. Requires collaboration from everyone in this room. Greenmaps.
Fires in southern california, my map created by UGC. Data being pulled into google earth, mashed up with aerial imagery of the smoke/fires. Showing burn/fire line, model of where the fire will go, all displayed in google earth. Knowing where the fire was going was missing, but now possible? Mashing up consumer geo stuff with commercial GIS is great. Can find the evacuation routes, road closures, all being generated in real time. Allows the data to be served out in these open formats.
This is all part of 9.3 which is all launching in a few weeks.
Google very excited about this with ESRI pushing to make this happen. Huge opportunities. Further expansion of the geoweb.
Questions…
… how quickly do we/ESRI expect people to take up the software?
Not sure, but adoption rate on new releases is months to 6 months. Software adoption isn’t the only issue, also “do I export my data?” Some agencies will be very interested in providing this public service of their knowledge, others won’t be. As this evolves, there’ll be the role for outsourcing public data by private companies. Two decisions, technology adoption and “do I release the data”? It’s now “one click” for people who do want to give their data away.
-
Where 2.0: From Data Chaos to Actionable Intelligence
Sean Gorman
[First slide was missing]
Three trends: Geoweb, handling large datasets, emerging semantic web
Story began as: Trying to GeoHack Without getting arrested
Sean started as a geographer but didn’t want to be a geographer really, looking at conceptualizing the internet, what the router graph looked like, etc. etc.
Big Data Sets -> Algorithmic Analysis -> Map Logical Results to physical realities
Made for some pretty interesting maps - “How to take down NYSE?”
Men in black suits turn up.
IN-Q-TEL came to the rescue, big history of taking ideas to market from the academic and start-up culure. Specifically in the geo-web. Have taken others, MetaCarta, keyhold, Last Software.
Tip of the iceberg that the role that the government has played to take these to market before there’s a big market, like GPS, satellite imagery.
Geography on the web became mainstream as we’d been playing around with geography of the web.
We’d always been focused on really large datasets. Stuff coming out was based on small
Geocommons - “Crowdsourcing large structured data sets with quantitative capabilities”
“Didn’t we try this last year”
“What happens when your database reaches 1,683,185,246 features”
Database goes a bit kaput.
Why so fast? Mainly it’s been on the long tail, we’ve been working on the short tail. 95% of the data
Data Normalization -> Tables get big -> Optimization -> Data Ingress -> Repeat
Everything helped but nothing really solved the problem.
Continue to fight
or
Build a lightweight object database
Can fit well over 1 billion features into ~16 Gigs of storage
Launching “Finder!”
Can search and can add your own data.
Link GIS and the real world.
[Demo]
Describe data, tag it, URLs, metadata URL for proper GIS marker, contact information…
Data was mined as it was uploaded. Can see statistics based on the data after uploading.
Can pull data out into KML, GML. Can view in Google Earth, MS Virtual Earth.
Links to metadata, FTGC[?], iso standard, micro formats.
Showing a KML file in virtual earth.
Census demographic information can be overlaid with sales information entered as part of the demo.
Can download the demographic information, pull it into google earth, can also pull it into GIS workflow.
[Demo showing polygon areas highlighted in colours on google earth with masses of information in an info window]
Expanding to: Maker! - ? Atlas! - share maps around stories and collaborate
Talking to other silos of data about trying to federate our data and interconnect it. Where the cloud and the data can be owned by everyone.
- Bringing a new class of content/data to the web
- Intelligently….
- Enable the content to answer meaningful questions for users
From Data Chaos to Actionable Intelligence
Technorati tags: geocommons, where, where2.0, where2008
-
Where 2.0: Merging Roadways: PC and Mobile Maps Coming Together
Michael Howard - Nokia
Nokia maps 2.0, mapping and routing in > 170 countries. “To be installed on 40 to 60 million gps handsets and close to 100 million location-aware devices.”
Commercial transactions in >100 countries per day.
Walking, pedestrian routing. Some phones have compass built in to tell which way you’re looking.
What’s next?
Innovation on “Drive, move, discover, create&collect;, share, meet”
Mobile, guidance centric. People get wherever they want to go regardless of transportation mode.
How they can do this by combining service across the web and mobile..
Video, music, woo!
Get rid of the idea of “mobile internet” and “internet”. There is one internet. This is a service that is complementing the mobile experience, not replicating it. On the phone, collect cool POIs, go back to the web and [upload?] them.
Sportstracker
-
Where 2.0: EveryBlock: A News Feed for your Block
Adrian Holovaty
Everyblock - Five EveryBlock lessons
“When I was your age we had to reverse engineer Google’s obfuscated JavaScript just to get maps on our pages!”
Mentioning Chicago crime maps, how we had to do things before the API was available.
Coolest and most useful part was every block in Chicago had it’s own page, listing all the crimes on the block. This made me think we should be able to do the same thing but bring in all sorts of other stuff that’s related, literally just for that block.
2 year grant from the Knight Foundation.
Chicago and NYC, expanding..
Can see every crime, business licenses, building permits, all this stuff is news to you if you live on the block.
Zoning changes, if a restaurant wants to build a bar, that’s very relevant to you. Filming locations. We also geocode news stories. Reviews, flickr photos, all sorts.
1st Lesson - “What is refreshing about Everyblock is that they’re feeding me existing data in an interesting way instead of asking me to give them my data”
Taking advantage of existing data, rather than the usual “give us your data” stuff.
“Be nice and appeal to civil servants”
Don’t bash people even if you’re improving their crappy site.
“Governments should focus on services [not on data]”
They shouldn’t have to concentrate on the cool site for viewing the data, they should just make sure they can provide the service for you to enter data.
“Plot cities/agencies against each other”
If one city is giving you good data, tell other cities.
2nd Lesson “The more local you get, the more effort it takes”
Slides shows “Convenient centralized USA building permits database” - on a clipart PC screen. Unfortunately there is no nice central database.
Lesson 3 - “Embrace hypertext”
Showing a blurred crime mashup, can restrict to show just certain crimes, certain dates. Not “webby” because you can’t link to results.
“Will my site work without maps?” - Chicago crime did because it had good text.
“Permalinkability”
Lesson 4 - “Move beyond points”
Not every story has to do with a point, some have to do with a neighbourhood. Don’t do a crappy job by choosing a centerpoint. Showing highlighting areas using polygons, or a part of a road using a line.
Can restrict zoom levels if necessary
Lesson 5 - “Roll your own maps”
Look at what Google give you for instance, they put building outlines, subway stops, one way arrows. Do you specifically need that for your mashup? You have no control with google maps over the colour of water, the colour of streets, the size of streets, language, etc. etc.
“One size fits all”
Would you seriously go to your web designer and say “yeah for our corporate website, use some standard wordpress template”
At every block we use mapnik to generate tiles allowing us to use our own styles, our own colour of blue. We use tile cache, openlayers and django.
No building outlines, no one way streets, basic style matches everyblock’s design.
‘Search for “take control of your maps”’ to see an article by Paul Smith from every block about how to do this.
Questions
Q: How are you going about managing relevance for users, allowing them to specify what they’re interested in?
A: They can say what schemas they’re interested in, to select don’t show me restaurants, only show me crimes. Can’t go further into schemas but we’re looking to add it.
Q: …. question concerned about privacy issues, a crime in a house
A: We only geocode to block level to partially help that. Also the cops have a limit on what they give out in some cases, for super sensitive stuff they don’t provide it. We mention this on the site. We don’t provide any names, e.g. for buyer/seller information on properties.
Q: … talk about the experience of parsing locations out of text
A: Kind of a big cludge, thinking that we solve things by throwing different methods at it rather than it being perfect natural language processing. Every time we find a new way of specifying an address that we haven’t handled before, we add it.
Q: Can you give us an idea of why mapnik for instance, rather than mapserver?
A: Would have to defer to Paul Smith, guessing it was about python bindings. Font rendering too.
Q: How many cities and how do you make money?
A: Funded by grants for 2 years, don’t know after that. Grant, VC, magically dream up business model? All the code will be open source. Not an incorporated company, just a bunch of guys in chicago. Not telling how many cities.
Q: Is there enough awareness of the project that cities are coming to you offering to help?
A: Yes in some cases, news organisations are asking how to format articles, bloggers too
Q: How do you get boundaries?
A: We get it from the government. Easier in the US, can be harder in other places, costs a lot in Germany for instance.
Where 2.0: EveryBlock: A News Feed for your Block
Technorati tags: everyblock, where, where2.0, where2008
-
Watch my Journey to Where 2.0
I’ve managed to knock something up that will let you track my journey from Liverpool in the UK over to San Francisco in the states. I do know I won’t be traveling the furthest but I still thought it would be fun ;-).
I’m currently (Friday lunchtime) sat in my house in Liverpool. I went for a run this morning so there is a little data in there already. I’ll be traveling down to London tonight and then going over to Heathrow and flying to San Francisco around lunchtime tomorrow. I’ll try to log my location as often as possible using the various means at my disposal. I have quite a few ways to do it so I’m hoping I’ll manage quite a good trace. I doubt I’ll get much if anything while I’m in the plane but I’ll give it a good go! Also I’ll be updating my position while I’m in California so it should end up with a full log of my trip over there. Finally, the page will automatically update itself every 5 minutes to see if my location has changed.
http://johnmckerrell.com/map/?show=john
Note the times at the bottom, “Last checked” is the last time that the page checked for an update and “last updated” is the time that I last sent it my position. So take a look and let me know any comments you have, especially if you find a bug! I may enhance it if I get time so you might want to refresh it as well from time to time.
One other thing to mention, even when I’m accurately updating my location with my GPS I’ll still only be updating once every minute, also that page isn’t capable of updating more than once every five minutes. I’m caching the location rather than using a database lookup to make it run a bit speedier. Oh yes, I do know the arrows don’t work in IE, unfortunately VML is not capable of drawing them.
Technorati tags: javascript, where, where2.0, where2008, map, tracking, wherecamp
-
Three month update
Wow, before starting this blog post I thought I’d check the date on my previous post. I haven’t written anything in nearly three months! That’s pretty terrible so I’ll try to do a few more posts sometime. I’ve been pretty busy with a number of other things recently but that’s no excuse, I know interesting things have happened that I should have found the time to mention.
I guess the biggest of the things that have been restricting my time recently was the recent redesign of the Multimap.com site and the availability of Microsoft Virtual Earth content within the Multimap API. Oddly enough, this was the third time that I’d worked on a project to pull VE content into our API. The first two times we had done it by wrapping VE’s API into our own, passing function calls to our API onto VE’s where necessary in a similar fashion to OpenLayers Base Layers. Though this gave us access to VE content and allowed us to resell the content to the UK Yell.com site, it was never going to be the best method. The finished product required loading the entirety of both APIS (nearly 300KB just for JS), and because we only had access to exposed functionality resulted in some odd behaviour, such as when our API and VE’s were both trying to smoothly pan a map to a new location.
In December of course we were bought out by Microsoft and as a result were able to get direct access to their imagery. Making use of VE’s maps and aerial data was fairly simple but including the Bird’s eye imagery was a little more difficult. Fortunately with the help of a few guys at MS who sent through some source code, and with the existing good “custom map type” functionality in the Multimap API, I had a good working model pretty soon. We then spent a long time trying to come up with a really good way of communicating “Bird’s Eye” mode to users. Though we had trials and tribulations along the way I think we did pretty well with the solution we came up with, it’s live on multimap.com if you want to take a look (or see the screenshots above and below).
Another thing that I’ve been trying to get sorted out is the server that hosts my blog. I’ve been using the same dedicated server for a good 4-5 years now. That server was hosted at Sago Networks and it’s been through a lot, including Florida hurricanes, but has recently started hanging on a regular basis. A replacement NIC a few weeks ago gave promise of a reprieve but it has again crashed since then. Though I’ve had really good service from sago who have been happy to manually fsck it every time it went down recently, I decided I had to go with price and have now switched to Hetzner. Their prices were too good to miss, and though the fact that I don’t speak a word of German has caused a few issues along the way everything seems to be going well with it now.
I’ve even decided to be all modern and am hosting my site on a VMWare Server virtual machine. I’m hoping that this will allow me more control over that machine, making it safer to upgrade and reboot as I’ll always be able to get access to the console through VMWare. Hopefully it’ll also make things much easier when I eventually decide to move to a new server again (I’ve had this new server for a month now and have only just found the time over the long weekend to move everything across!) I’m also hoping it will lend me a little more security by allowing me to segregate important sites that I need to keep secure away from older less reliable code.
One other thing to mention, I’m going to San Francisco next week! Where 2.0 is the second most important conference for location based services providers (the most important, of course, being OSM’s State of the Map) but in past years I haven’t been able to attend. Fortunately this year some budget has turned up and I together with four of my colleagues will be there. Though the main event is the O’Reilley conference, I’ll also be going along to WhereCamp 2008 the following weekend, and as many other events as I can cram in on the Thursday and Friday between. If you’re going and I don’t already know it then get in touch with me on twitter or friend me on the WhereCamp site.
I’m also intending to map my journey there as much as possible, I have to get from Liverpool to London on Friday night, over to Heathrow on Saturday morning and then fly to San Francisco International airport at lunchtime (leaving 10am, arriving 1pm, still freaks me out ;-). Obviously it’s highly unlikely that I’ll be able to log anything while I’m flying but I’ll do my best, and I shouldn’t have too much problem on the train down. Check back later this week when I’ve figured out how I’m going to share my location with you :-)
-
Apple invents iPod maps - podmaps?
The other day, a friend sent through a link to an interesting “new” technology that Apple have applied for a patent for - podmaps. Reading through the article it seemed oddly familiar, perhaps because I came up with the same idea two years ago. As most good ideas do, this one came about over a few beers when someone suggested that Multimap should do a podcast. Of course the idea of a Multimap podcast was perhaps a little odd, but it did get me thinking about what we could do with podcasts.
After a little investigation and playing around with Apple’s “ChapterTool” I knocked up the Multimap Travel Directions Podcast. You should be able to try it by downloading the file and launching it in either iTunes or Quicktime. It takes advantage of all the great features available to podcast files. The file is split into “chapters” with each chapter being a step of the route. 
Each chapter has audio with a computer voice talking you through the directions for that step, an image attached to it which shows a map of that step, the text is included as the “title” (and is readable on your iPod’s screen) and there’s even a link to the route on multimap.com for when you’re viewing the podcast on your computer. (Note that the podcast above was made 2 years ago so uses old maps and probably links to the old multimap.com, keep reading to hear about the improvements though…)So this is all very nice, but this thing took me a few hours to make by hand, surely there’s a better way? Well with all the great APIs that Multimap provides, yes there is. Though I was busy with a few other things I played around with this idea over the following 15 months or so. Every few months I’d write a few lines of code, come across a problem, get bored and put it down again. Finally though I got past all the problems (how to tell the length, in seconds, of an audio file, how to concatenate audio snippets, how to convert to AAC, and a few more) and managed to knock up a ruby script that could take a source and a destination and give you a podcast containing the directions. That was 9 months ago though and I’ve been sitting on it since. Seeing the news on Apple’s patent application has spurred me on to releasing it.
I spent a few hours last night updating the script so that it now works with our recently launched Static Maps API meaning that you’ll see a vast improvement in the map quality. The script requires OS X to work because it uses Apple’s ChapterTool and “say” command. It also needs SoX (for various sound conversions), the Perl CPAN module Audio::Wav and the FAAC library. I’ve packaged the script into a zipfile and put a README in there with some information on using it, I mention how to get and install the 3rd party packages in there too.
There’s obviously lots of improvements that could be made to the podcast, and I’m sure Apple will make many if they actually do release “podmaps”. All of the podcasts I’ve generated here have been made on my OS X Tiger installation, if you have Leopard you will be able to use the new speech synthesis voice that came with it to make better sounding podcasts. All the routes generated are currently driving directions but it would be a simple tweak to make walking directions. Extra functionality could also be added using more Multimap API functionality, such as highlighting places to eat as you’re passing them and things like that. For now I just thought I’d release it as-is and see what people think.
So, please download the script, play around with it, and let me know in the comments what you think of it. It might be fiddly to get it working but rather than get too technical in this post I thought I’d put that sort of thing into the README files. If you haven’t already, you’ll need to sign up for the Multimap Open API to get a key to use. Here’s a few more routes to give you an idea of what it can do too:
Technorati tags: apple, ipod, podmap, routes, map, multimap, maps
-
Ed Parsons gets heavy, Fake SteveC gets heavier
Some interesting and amusing posts have shown up over the weekend that I just couldn’t go without mentioning. Ed Parsons posted an article about Nestoria’s new property site based around OSM maps. In this article Ed brings up the obvious problem that OSM’s coverage isn’t quite there yet, and goes as far as comparing it to the infamous Rabbit phone. Cue much discussion in the comments. I had been intending to write a response on here over the weekend, but didn’t get around to it. I was reminded this morning though when I saw Fake Steve Coast’s response. Absolutely hilarious (especially the carefully researched graph). I’ve really been enjoying reading the blogs of Fake Ed Parson and Fake SteveC (FSC) over the past few months so I just thought I’d say that if you’re not already reading them, you really should be.
So, obviously there’s things I want to say in response to Ed’s post, but as FSC says, he’s declared a temporary ceasefire so I won’t bring up some of the things I was thinking, I’ll just point out that Google’s coverage hasn’t always been as good as it is now:
Also, even now, there’s places where Google’s coverage just isn’t up to scratch:
(I must give props to Ed for at least using recent screenshots though, some people have used very out of date screenshots in some of their presentations.)
-
Mate's great band releasing new single
Just a quick one to mention a friend’s band that have a new single out this week. Parka are a great band originating from Glasgow but currently residing in East London. They’ve been around for a couple of years but are preparing for the launch of their debut album with the re-release of their “manic anthem” Disco Dancer. Mark, the keyboard player for the band, used to work for Multimap and I first heard this single just over a year ago. Instantly loved it but at the time it wasn’t available to buy or download (legally).
In December a collection of Multimappers went down to see them play in a small club off Tottenham Court Road. Though they weren’t headlining (they were on second) they were definitely the best band who played on the night and had a good selection of really great sounding songs. I’m definitely looking forward to their debut album but for now I’ll have to make do with the single.
You’ll see a little flash widget thingey on the right which you can use to have a quick listen to the track but I recommend you head over to their website - parkamusic.com - where you can listen to the whole song (it’ll start playing a random track, to hear Disco Dancer keep skipping until it comes up, there’s also a remix which is pretty good too). If you like the track then you know what to do, either follow the links on their website to buy it or head over to disco.parkamusic.com to be linked straight to the shop. They have a special pack of the 7” Vinyl and CD that I’ve gone for, or you can just buy the CD using the link at the bottom of the page. I believe it’ll also be available on iTunes and probably other music download sites too.
Final note, they’re currently finishing off a tour - they’ve still got dates around the country for the next few months - I’m planning to go to their Liverpool gig on the 26th February so if you want to come along and you’re in the area, give me a shout.
-
New version of OSMify bookmarklet
I’ve put together a new version of my OSMify bookmarklet. The main reason for doing this is because the old version is linking to the wrong set of Osmarender tiles; it’s linking to the ones on dev.openstreetmap.org rather than the newer ones on tah.openstreetmap.org. Rather than force you to regularly visit this site and setup a new bookmark every time there’s an update, I’ve decided to host the JavaScript for this version of OSMify on my site.
In the previous version, all of the JavaScript to make the new map types appear was contained within the one line bookmark. This time I’ve put all of the important JavaScript into a separate file hosted at johnmckerrell.com/files/osmify.js. The bookmark now only contains some simple JavaScript required to load that file in.
As well as allowing me to make improvements to the code this also makes the code a lot more readable, allowing you to read through it and understand what it’s doing. If you have any improvements that you’d like to suggest then either drop them as comments on here or email them through to me.
One improvement I did want to attempt was to make OSMify work on other sites than Multimap’s. The Multimap API’s custom map type support works in a very similar way to the Google API so it would be easy to support. Microsoft Virtual Earth/Live Maps also have a method for putting custom tiles into their API. Unfortunately with Google, they don’t expose the full API on their public site (i.e. maps.google.com) so it isn’t possible to add custom map types there. The current version of OSMify will work with third-party implementations, though they will need to expose a global
mapvariable that is of typeGMaporGMap2. I have also had to tell the code to add a newGMapTypeControlafter adding the new map types as for some reason the Google API’s map type control doesn’t update when you add new map types (something the Multimap API does do). I haven’t bothered with Virtual Earth/Live Maps support either yet as I would need to code up my own widget/control for switching between these map types (or tile layers as they call them).So, the link is at the top of this post, have a go with it and let me know what you think. I’d especially be interested in some feedback on using this with Google maps implementations, and any thoughts you have on getting this to work with more APIs.
Technorati tags: javascript, bookmarklet, osm, openstreetmap, map, liverpool, multimap, maps, favelet
subscribe via RSS or via JSON Feed