Running paths in Amsterdam - Step 2

Visualizing the routes with R

Posted: March 8, 2014-Likes: 0-Comments: 4-Categories: Geospatial, R, Static-Tags: R, Sport, Static
You are here: ...
Home / Blog / Geospatial / Running paths in Amsterdam – Step 2 – Visualizing the routes with R and Cloudmade
Visualizing running paths - header

On to Step 2 of actually plotting the running routes in Amsterdam. If you’ve missed Step 1 in which I explained how I got the data of 1000 running routes in Amsterdam, you can check my previous post “Step 1 – Webscraping 1000 routes”

Getting the coordinates from the gpx files

On the blog from Flowingdata that gave me the idea to also try this for Amsterdam, Nathan shares a snippet of R code that reads all gpx files in a folder, extracts the latitude and longitude and puts all routes into one data frame which you can plot (for me, this was a major reason why I thought I might actually succeed in making the Amsterdam plot since I could use this as a backbone)

Sadly, the script gave an error while trying to read in the gpx files



route = readGPX(files[i])

I do not know the exact reason why, perhaps the MapMyRun gpx files weren’t formatted according to the normal standards.

Nevertheless, I went into the readGPX function to see why the error occurred and how I could solve it (to be honest I already did this bit before I found a way to get to 1000 route IDs, since I wanted to be really sure that I wouldn’t be doing the prework for nothing :) ).

In the end it wasn’t that difficult to use the lines of the readGPX function that I needed for the MapMyRun gpx structure and get the latitude and longitude out. You can find the function that works on the MapMyRun gpx files here:



ReadGPX_MMR <- function(gpx.file) {
  ret <- xmlTreeParse(gpx.file, useInternalNodes = TRUE)
  top <- xmlRoot(ret)  
  trkpt <- top[[1]][[2]]
  lon <- as.numeric(xmlSApply(trkpt, xmlGetAttr, "lon"))
  lat <- as.numeric(xmlSApply(trkpt, xmlGetAttr, "lat"))
  
  return(data.frame(lat,lon))
}

So I made a small change to Nathan’s R code and turned top portion where all routes are put together into one dataframe where the first column gives the map index (so you know which rows belong to the same route) and the other two columns the coordinates into this:



#Read in GPX files that are located in the working directory
files <- dir(pattern = "\\.gpx")

#Set up the three vectors to hold your coordinate and path data
index <- c()
latitude <- c()
longitude <- c()
for (i in 1:length(files)) {        
    route <- ReadGPX_MMR(files[i])
  
    index <- c(index, rep(i, nrow(route)))
    latitude <- c(latitude, route$lat)
    longitude <- c(longitude, route$lon)
}#for i
#Combine the three vectors into one data frame
routes <- data.frame(cbind(index, latitude, longitude))

Getting a map of Amsterdam into R

Now for the mapping itself. With Nathans code you get to see your routes, but without an actual background of the map. After a bit of browsing online I came across the ggmap package (you can find a presentation with a good tutorial here). With the ggmap package it become incredibly easy to plot a map of your choice. Just use the wrapper function qmap in which you specify either a location name or a bounding box (the lat/lon of the left, bottom, right and top corner) and a zoom level. With the following small piece of code I already have a Google map tile of Amsterdam



qmap(location = 'amsterdam', zoom = 12)

Or make it grey by adding a color argument



qmap(location = 'amsterdam', zoom = 12, color = 'bw')

I chose the grey background because Nathan’s images just looked so good :)

Overplotting the routes

Since ggmap has affinity with the ggplot2 package I went ahead and used the geom_path function of ggplot2 to overplot my map of Amsterdam with the routes. The online ggplot2 documentation site has many examples to help you get started. With the following piece of code I finally plotted the 1000 routes in Amsterdam into what I was looking for


#Get the map of Amsterdam from Google
AmsterdamMap <- qmap('amsterdam', zoom = 12, color = 'bw')

#Plot the map and the routes on top of that
AmsterdamMap +
  geom_path(aes(x = longitude, y = latitude, group = factor(index)), 
            colour="#1E2B6A", data = routes, alpha=0.3)

Creating a custom made map in Cloudmade

But wait, the plot is cutting of some routes, so I would need to either zoom out (which I didn’t want to) or use qmap to create a bigger section around Amsterdam. Luckily, instead of a location you can input a bounding box. Using Google Maps I found the locations of the top left and bottom right corners that I really wanted and tried the following line


qmap(location = c(4.734712,52.268002,5.048166,52.454599), zoom = 12, color = 'bw')

However, qmap already gives a warning that the bounding box given to Google Maps is only approximate and it resulted in exactly the same map as the images above

Besides Google Maps, qmap can load a lot more map providers. Personally I love the watercolor map from Stamen, which you can load with



qmap(location = 'amsterdam', zoom = 12, maptype = 'watercolor', source = 'stamen')

Note: the next section on Cloudmade is outdated because they suspended the mentioned service

Since I also wanted to try creating my own map style I created an account on Cloudmade since you can also use Cloudmade as a source. However, then you also need to provide an API key and map ID to the qmap call

This is very easy to create. After you have an account visit your account page on “http://account.cloudmade.com/user” and click the big blue not-to-miss “Get API key” button.

Even though you won’t be creating an actual app, you’ll have to fill in some details about you non existing app. Just fill in some bogus things (and don’t fill in a webpage) to create your key. After having created your key you can find the key if you visit your account page again.
Next click in the “Map Styles” tab and press “Create New Style” which will bring you to the Cloudmade editor

You will see several map styles that others have already created. In the bottom right corners of each map you can find the map ID of that particular map which, together with the API key is all we need to call a Cloudmade map into R

I picked a map that I already liked and pressed “Clone Style” in the bottom right. This will bring you to another window of that map where you can make all the adjustments you want, pink roads, purple water, whatever you like. Once you have the map that you like, press “Save this style” in the bottom right corner.

I’m not sure if this is the only way, but to get the Map ID of my own map I went back to the Cloudmade editor and pressed “My Styles”. The map ID can now be found in the bottom right corner of the example of your map

Visualizing the Final map

Alright, getting back to R I could now create a visualization using my own map with exactly the right size, since the Cloudmade map was able to use my exact bounding box coordinates:



#Get the map of Amsterdam from my own designed Cloudmade map
#left/bottom/right/top bounding box
AmsterdamMap <- qmap(location = c(4.734712,52.268002,5.048166,52.454599), zoom = 12, maptype = MAP_ID, 
                     api_key = "MY_API_KEY", source = "cloudmade")

#Plot the map with the routes
AmsterdamMap +
  geom_path(aes(x = longitude, y = latitude, group = factor(index)), 
            colour="#1E2B6A", data = routes, alpha=0.3)

I just created a simple map where the water is bright blue and the forest is bright green to see of those places attract running routes. In the end I was very happy that I managed to get it to work

Now I could really see from the data that the few parks in the center of Amsterdam have the biggest blue lines. Some routes are very long and go far beyond the borders of Amsterdam. Perhaps people training for marathons, perhaps people on bikes that marked their route as a running route.

And, also very important of course, I like the picture :)

MapMyRun webscraped running paths
Prev / Next Post
Comments (4)
  • sqiar - July 16, 2014 - Reply

    It’s actually a cool and useful piece of info. I am glad that you simply shared this useful info with us. Please keep us up to date like this. Thanks for sharing, You can also check out this http://www.sqiar.com

  • M - November 12, 2014 - Reply

    that’s pretty cool, thanks for sharing.

    if I had been aware of your blog and flowingdata’s, I could have saved some time when I was working on something very similar.

    basically, after attending a stats talk about bike safety in London, where the presenter showed a map with all bike accidents over a few years (as dots without a map) and where one could distinctly recognise London’s hallmark features (the contours of Hyde Park, Regent’s Park etc with all roads / bike paths going through them, Oxford Street, …), I wanted to find out what my ~4-5 years’ worth of run data in London looked like and whether I too would be able to recognise the city.

    I was just interested in my own runs, so have nothing like the data fetching you did, but for the plotting I basically just used bash & grep to extract the longitude and latitude coordinates (btw some GPX files use rtept rather than trkpt for storing that data) and then r for plotting the data (same idea here: draw transparent lines between connected GPS coordinates).
    Once I had my code working I turned it into a little web app that fellow runners from my club could use (http://marc-henrion.dyndns.org/routeHeat.html).

    I will now definitely add links to your blog and also the flowingdata webpage as the ideas are just so similar.
    I will also post direct links to my shell and R scripts.
    I might (if I get the time…) add a map feature using your hint how to get maps into R — I will of course credit you with that.

    Thanks again for your post.

Add comment

13 − thirteen =