…about maps, open data, Git, and other tech.
For my next web mapping project, we’ll use vector tiles. Great. And the data will come from OpenStreetMap. Excellent. Now you only have five more questions to answer.
For the front-end web application developer who wants to stick a map in their site, vector tiles open up lots of options and flexibility, but also lots of choices.
We’re very lucky that in a couple of these areas, a single standard dominates:
Which leaves three decisions to make.
There are several viable options for displaying your vector tiles, depending on whether you also want to display raster tiles or need creative styling, if WebGL (IE11+ only) is ok, and what else you need to integrate with.
Mapbox Terrain, a style rendered with Mapbox-GL-JS.
There are other combinations as well, such as Leaflet.VectorGrid.
The style mechanism tends to be closely tied to the display engine. (That was also true of CartoCSS, which was a pre-processor for Mapnik. RIP).
Finally, there are three distinct, well-defined schemas for packaging OpenStreetMap data into vector tiles. There doesn’t seem to be a formal specification for how you define a schema, so each is presented as documentation: a list of layers, each with a list of attributes (and their possible values), and at which zoom levels they appear.
Dark Matter, a Mapbox Style for OpenMapTiles.
class=major_rail”).
Now, the style needs to be designed for the schema: if the schema contains a layer called “roads”, your style can’t be expecting a layer called “transportation”. But it also needs to be expressed in the right format supported by the engine: don’t go feeding no YAML to Mapbox-GL-JS.
For instance:
Tron, a highly stylised style from Mapzen for Tangram.
Style Specification. And hence can be rendered by Mapbox-GL-JS, or OpenLayers. (Other standard Mapbox styles include Mapbox Streets, Mapbox Terrain and Mapbox Dark)
These styles kind of live within their company affiliations, however. How about styles rendered by one company’s engine, using data from a different schema:
Which brings us to the point of this post. How do you mix schemas and styles? That is, how do you take a style you designed for Mapbox Streets, and make it work on OpenMapTiles? Or port one of Mapzen’s kooky open-licensed styles so it works with Mapbox Streets? Well, you can’t – yet.
(Adapting a style from one engine to another, like what ol-mapbox-style does, is a tough ask, because engines’ capabilities differ.)
But adapting a Mapbox Style file from one OpenStreetMap schema to another? That seems totally doable – even if there isn’t yet a tool to make that happen.
My quick little proof of concept in NodeJS converted OpenMapTiles’ “OSM Bright” style (left) to versions for Mapbox Streets (centre) and Mapzen (right).
Want to give me a hand? Get in touch!
Cycletour.org is a tool for planning cycle tours in Australia, and particularly Victoria. I made it because Google Maps is virtually useless for this: poor coverage in the bush and inappropriate map styling make cycle tour planning a very frustrating experience.
Let’s say we want to plan a trip from Warburton to Stratford, through the hills. This is what Google Maps with “bicycling directions” offers:
Google Maps – useless for planning cycle tours.
Very few roads are shown at this scale. Unlike motorists, we cyclists want to travel long distances on small roads. A 500 kilometre journey on narrow backstreets would be heaven on a bike, and a nightmare in a car. So you need to see all those roads when zoomed out.
Worse, small towns such as Noojee, Walhalla and Woods point are completely missing!
Enter Cycletour.org:
You can plan a route by clicking a start and end, then dragging the route around:
It doesn’t offer safe or scenic route selection. The routing engine (OSRM) just picks the fastest route, and doesn’t take hills into account. You can download your route as a GPX file, or copy a link to a permanent URL.
The other major features of cycletour.org’s map style are:
Bike paths are shown prominently. Rail trails (old train lines converted into bike paths) are given a special yellow highlighting as they tend to be tourist attractions in their own right.
Train lines (in green) are given prominence, as they provide transport to and from trips.
Towns are only shown if there is at least one food-related amenity within a certain distance. This is by far the most important information about a town. Places that are simply “localities” with no amenities are relegated to a microscopic label.
Major roads are dark gray, progressing to lighter colours for minor roads. Unsealed roads are dashed. Off-road tracks are dashed red lines. Tracks that are tagged “four-wheel drive only” have a subtle cross-hashing.
And of course amenities 
useful to cyclists are shown: supermarkets, campgrounds, mountain huts, bike shops, breweries, wineries, bakeries, pubs etc etc. Yes, well-supplied towns look messy, but as a user, I still prefer having more information in front of me.
The terrain data is a 20 metre-resolution digital elevation model from DEPI, within Victoria, trickily combined with a 90m DEM elsewhere, sourced from SRTM (NASA). I use TileMill‘s elevation shading feature, scaled so that sea level is a browny-green, and the highest Australian mountains (around 2200m) are white, with green between. 20-metre contours are shown, labelled at 100m intervals.
I’m really happy with how it looks. Many other comparable maps have either excessively dark hill shading, or heavy contours – or both.
4UMaps
Komoot
OpenCycleMap
Sigma
Google Maps (terrain mode)
MapBox Outdoors
VicMap
I’ve included an assortment of common basemaps, including most of the above. But the most useful is perhaps VicMap, because it represents a completely different data source: the government’s official maps.
Vegetation
There are also optional overlays. Find a good spot to stealth camp with the vegetation layer.
Or avoid busy roads with the truck volume layer. This data comes from VicRoads.
The bike shops layer makes contingency planning a bit easier, by making bike shops visible even when zoomed way out. The data is OpenStreetMap, so if you know of a bike shop that’s missing (or one that has since closed down), please update it so everyone can benefit.
Unfortunately, the site is pretty broken on mobile. But you can download the tiles for offline use on your Android phone using the freemium app Maverick. It works really well.
is.cycletour.org for Iceland. Yes, it’s real – but I don’t know how long I will maintain it.
It’s a pretty major technical undertaking to run a map for the whole world. I’ve automated the process for setting up cycletour.org as much as possible, and created my own version for Iceland and England when I travelled there in mid 2014. If you’re interested in running your own, get in touch and I’ll try to help out.
I’d love to hear from anyone that uses cycletour.org to plan a trip. Ideas? Thoughts? Bugs? Suggestions? Send ’em to stevage@gmail.com, or on Twitter at @Stevage1.
Contributing to OpenStreetMap is diversely rewarding: you help other people, you make open data as a whole more viable, you learn a lot about the area you’re mapping, and it’s fun. But sometimes it’s just plain pragmatic. Last weekend, I organised a cycle tour from Bendigo to Avenel, via the O’Keefe Rail Trail, Lake Eppalock, Colbinabbin, Rushworth, Murchison and Nagambie. When I started planning the route, OpenStreetMap looked like this:
The major features are all there, but what’s missing is what matters most to cycle tourists: quiet country roads, and road surfaces. Is there a way to get from Eppalock to Colbinabbin on only sealed roads? Is Buffalo Swamp Rd (near Murchison) really sealed? A great way for me to research is to add to OpenStreetMap: use aerial imagery to add new roads, paying attention to whether they look sealed or not from the air.
So Buffalo Swamp Road is obviously not sealed after all. By the time I was done, the map of the area looked like this:
Notice how many “sealed” roads have turned out to be dirt, but also how many other unmapped little roads have been added to the map.
Once this is done, the steps are:
There’s still lots more to add, but it’s nice that just planning this one trip has significantly improved coverage in a whole region like this.
I created a basemap for http://cycletour.org with TileMill and OpenStreetMap. It looked…ok.
But it felt like there was something missing. Terrain. Elevation. Hills. Mountains. I’d put all that in the too hard basket, until a quick google turned up two blog posts from MapBox, the wonderful people who make TileMill. “Working with terrain data” and “Using TileMill’s raster-colorizer“. Putting these two together, plus a little OCD, led me to this:
Slight improvement! Now, I felt that the two blog posts skipped a couple of little steps, and were slightly confusing on the difference between the two approaches, so here’s my step by step. (MapBox, please feel free to reuse any of this content.)
My setup is an 8 core Ubuntu VM with 32GB RAM and lots of disk space. I have OpenStreetMap loaded into PostGIS.
You need to do this if you want to use the raster-colorizer. You want this while developing your terrain style, if you want the “snow up high, green valleys below” look. Without it, you have to pre-render the elevation color effect, which is time consuming. If you want to tweak anything (say, to move the snow line slightly), you need to re-render all the tiles.
Fortunately, it’s pretty easy.
The easiest place is the ubiquitous NASA SRTM DEM (digital elevation model) data. You get it from CGIAR. The user interface is awful. You can only download pieces that are 5 degrees by 5 degrees, so Victoria is 4 pieces.
If you’re downloading more than about that many, you’ll probably want to automate the process. I wrote this quick script to get all the bits for Australia:
for x in {59..67}; do
for y in {14..21}; do
echo $x,$y
if [ ! -f srtm_${x}_${y}.zip ]; then
wget http://droppr.org/srtm/v4.1/6_5x5_TIFs/srtm_${x}_${y}.zip
else
echo "Already got it."
fi
done
done
unzip '*.zip'
To have any fun with terrain mapping in TileMill, you need to produce separate layers from the terrain data:
Contours – they’re the best.
In addition, you’ll need some extra processing:
Hopefully you already have GDAL installed. It probably came with the development version of TileMill.
Here’s my script for doing all the processing:
#!/bin/bash
echo -n "Merging files: "
gdal_merge.py srtm_*.tif -o srtm.tif
f=srtm
echo -n "Re-projecting: "
gdalwarp -s_srs EPSG:4326 -t_srs EPSG:3785 -r bilinear $f.tif $f-3785.tif
echo -n "Generating hill shading: "
gdaldem hillshade -z 5 $f-3785.tif $f-3785-hs.tif
echo and overviews:
gdaladdo -r average $f-3785-hs.tif 2 4 8 16 32
echo -n "Generating slope files: "
gdaldem slope $f-3785.tif $f-3785-slope.tif
echo -n "Translating to 0-90..."
gdal_translate -ot Byte -scale 0 90 $f-3785-slope.tif $f-3785-slope-scale.tif
echo "and overviews."
gdaladdo -r average $f-3785-slope-scale.tif 2 4 8 16 32
echo -n Translating DEM...
gdal_translate -ot Byte -scale -10 2000 $f-3785.tif $f-3785-scale.tif
echo and overviews.
gdaladdo -r average $f-3785-scale.tif 2 4 8 16 32
#echo Creating contours
gdal_contour -a elev -i 20 $f-3785.tif $f-3785-contour.shp
Take my word for it that the above script does everything I promise. The options I’ve chosen are all pretty standard, except that:
Now you have four useful files, so create layers for them. I’d suggest creating them as layers in this order (as seen in TileMill):
For each of these, you must set the projection to 900913 (that’s how you spell GOOGLE in numbers). For the three ‘tifs’, set band=1 in the “advanced” box. I gather that GeoTiffs can have multiple bands of data, and this is the band where TileMill expects to find numeric data to apply colour to.
Mapbox’s blog posts go into detail about how to do this, so I’ll just copy/paste my styles. The key lessons here are:
My styles:
.hs[zoom <= 15] {
[zoom>=15] { raster-opacity: 0.1; }
[zoom>=13] { raster-opacity: 0.125; }
[zoom=12] { raster-opacity:0.15;}
[zoom<=11] { raster-opacity: 0.12; }
[zoom<=8] { raster-opacity: 0.3; }
raster-scaling:bilinear;
raster-comp-op:multiply;
}
// not really convinced about the value of slope shading
.slope[zoom <= 14][zoom >= 10] {
raster-opacity:0.1;
[zoom=14] { raster-opacity:0.05; }
[zoom=13] { raster-opacity:0.05; }
raster-scaling:lanczos;
raster-colorizer-default-mode: linear;
raster-colorizer-default-color: transparent;
// this combo is ok
raster-colorizer-stops:
stop(0, white)
stop(5, white)
stop(80, black);
raster-comp-op:color-burn;
}
// colour-graded elevation model
.dem {
[zoom >= 10] { raster-opacity: 0.2; }
[zoom = 9] { raster-opacity: 0.225; }
[zoom = 8] { raster-opacity: 0.25; }
[zoom <= 7] { raster-opacity: 0.3; }
raster-scaling:bilinear;
raster-colorizer-default-mode: linear;
raster-colorizer-default-color: hsl(60,50%,80%);
// hay, forest, rocks, snow
// if using the srtm-3785-scale.tif file, these stops should be in the range 0-255.
raster-colorizer-stops:
stop(0,hsl(60,50%,80%))
stop(392,hsl(110,80%,20%))
stop(785,hsl(120,70%,20%))
stop(1100,hsl(100,0%,50%))
stop(1370,white);
}
.contour[zoom >=13] {
line-smooth:1.0;
line-width:0.75;
line-color:hsla(100,30%,50%,20%);
[zoom = 13] {
line-width:0.5;
line-color:hsla(100,30%,50%,15%);
}
[zoom >= 16],
[elev =~ “.*00”] {
l/text-face-name:’Roboto Condensed Light’;
l/text-size:8;
l/text-name:'[elev]’;
[elev =~ “.*00”] { line-color:hsla(100,30%,50%,40%); }
[zoom >= 16] { l/text-size: 10; }
l/text-fill:gray;
l/text-placement:line;
}
}
And finally a gratuitous shot of Mt Feathertop, showing the major approaches and the two huts: MUMC Hut to the north and Federation Hut further south. Terrain is awesome!
Recently, I set up a server for a series of #datahack workshops. We used TileMill to make creative maps with OpenStreetMap and other available data.
The major pieces required are:
I’ve captured all those bits, and their configuration in this script. You’ll probably want to change the password – search for “htpasswd”.
The script below is out of date, contains errors, and is not maintained. Go to:
# This script installs TileMill, PostGIS, nginx, and does some basic configuration. # The set up it creates has basic security: port 20009 can only be accessed through port 80, which has password auth. # The Postgres database tuning assumes 32 Gb RAM. # Author: Steve Bennett wget https://github.com/downloads/mapbox/tilemill/install-tilemill.tar.gz tar -xzvf install-tilemill.tar.gz sudo apt-get install -y policykit-1 #As per https://github.com/gravitystorm/openstreetmap-carto sudo bash install-tilemill.sh #And hence here: http://www.postgis.org/documentation/manual-2.0/postgis_installation.html #? sudo apt-get install -y postgresql libpq-dev postgis # Install OSM2pgsql sudo apt-get install -y software-properties-common git unzip sudo add-apt-repository ppa:kakrueger/openstreetmap sudo apt-get update sudo apt-get install -y osm2pgsql #(leave all defaults) #Install TileMill sudo add-apt-repository ppa:developmentseed/mapbox sudo apt-get update sudo apt-get install -y tilemill # less /etc/tilemill/tilemill.config # Verify that server: true sudo start tilemill # To tunnel to the machine, if needed: # ssh -CA nectar-maps -L 21009:localhost:20009 -L 21008:localhost:20008 # Then access it at localhost:21009 # Configure Postgres echo "CREATE ROLE ubuntu WITH LOGIN CREATEDB UNENCRYPTED PASSWORD 'ubuntu'" | sudo -su postgres psql # sudo -su postgres bash -c 'createuser -d -a -P ubuntu' #(password 'ubuntu') (blank doesn't work well...) # === Unsecuring TileMill export IP=`curl http://ifconfig.me` cat > tilemill.config <<FOF { "files": "/usr/share/mapbox", "coreUrl": "$IP:20009", "tileUrl": "$IP:20008", "listenHost": "0.0.0.0", "server": true } FOF sudo cp tilemill.config /etc/tilemill/tilemill.config # ======== Postgres performance tuning sudo bash cat >> /etc/postgresql/9.1/main/postgresql.conf <<FOF # Steve's settings shared_buffers = 8GB autovaccuum = on effective_cache_size = 8GB work_mem = 128MB maintenance_work_mem = 64MB wal_buffers = 1MB FOF exit # ==== Automatic start cat > rc.local <<FOF #!/bin/sh -e sysctl -w kernel.shmmax=8000000000 service postgresql start start tilemill service nginx start exit 0 FOF sudo cp rc.local /etc/rc.local # === Securing with nginx sudo apt-get -y install nginx cd /etc/nginx sudo bash printf "maps:$(openssl passwd -crypt 'incorrect cow cell pin')\n" >> htpasswd chown root:www-data htpasswd chmod 640 htpasswd exit cat > sites-enabled-default <<FOF server { listen 80; server_name localhost; location / { proxy_set_header Host \$http_host; proxy_pass http://127.0.0.1:20009; auth_basic "Restricted"; auth_basic_user_file htpasswd; } } server { listen $IP:20008; server_name localhost; location / { proxy_set_header Host $http_host; proxy_pass http://127.0.0.1:20008; auth_basic "Restricted"; auth_basic_user_file htpasswd; } } FOF sudo cp sites-enabled-default /etc/nginx/sites-enabled/default sudo service nginx restart echo "Australia/Melbourne" | sudo tee /etc/timezone sudo dpkg-reconfigure --frontend noninteractive tzdata
Steve Bennett blogs 
Recent Comments