…about maps, open data, Git, and other tech.
National Map is a pretty awesome place to find geospatial open data from all levels of Australian government. (Disclaimer: I work on it at NICTA). But thanks to some not-so-obvious features in TerriaJS, the software that drives it, you can actually create and share your own private version with your own map layers – without programming, and without deploying any code.
What you get:
Want to create your own spatial layer – polygons, lines and points? Use geojson.io and choose Save > Gist to save the result to Github Gist. (Gist is just a convenient service that stores text on the web for free).
How about a layer of data about suburbs by postcode? Create a Google Sheet that follows the csv-au-geo specification (it’s easy!), download as CSV, paste it into a fresh Gist.
Using the new TerriaJS Data Source Editor (I made this!), create your new catalog. You’re basically writing a JSON file but using a web form (thanks json-editor!) to do it.
To add one of your datasources on Gist, make sure you link to the Raw view of the page:
Don’t forget to select the type for each file: GeoJSON, CSV, etc.
You might want to bring in some other data sources that you found on National Map. This can be a little tricky – there’s a lot of complexity in accessing data sources that National Map hides for you.
But here’s roughly how to go about it for a WMS (web map service) data source.
In the layer’s info window, grab the WMS URL
You’ll need to put “http://nationalmap.gov.au/proxy/” in front of a some layers, because their WMS servers don’t support CORS.
You’ll also need the value of the “Layer name” field. (For Esri layers you need to dig a bit further.)
(Yes, this layer is called “2”)
Add a WMS layer, and add “Layers Names” as an additional property. So it looks like this:
You can add extra properties to layers to fine tune their appearance. For example, for our CSV dataset:
You might want to set “Is Enabled” and “Is Shown” on every layer so they display automatically.
And finally, you might want to set an initial camera and base map, so the view doesn’t start off the west coast of Africa with a satellite view.
As you make changes, click “Save to Gist” to save your configuration file to a secret location on Gist. You can then click “Preview your changes in National Map”.
Make a note of the Gist link so you can keep working on it in the future. You can’t modify an existing configuration, but you can load from there and save a new copy.
Now you have a long URL like this: http://nationalmap.research.nicta.com/#clean&https%3A%2F%2Fgist.githubusercontent.com%2Fanonymous%2Fc3f181ca742b9ed94fe4%2Fraw%2F10853f7d8bb33610e4f2ce26947eaf6882192957%2Fdatasource.json
So, use tinyurl.com or another URL shortening service to get something more useful:
Cycletour.org is a tool for planning cycle tours in Australia, and particularly Victoria. I made it because Google Maps is virtually useless for this: poor coverage in the bush and inappropriate map styling make cycle tour planning a very frustrating experience.
Let’s say we want to plan a trip from Warburton to Stratford, through the hills. This is what Google Maps with “bicycling directions” offers:
Google Maps – useless for planning cycle tours.
Very few roads are shown at this scale. Unlike motorists, we cyclists want to travel long distances on small roads. A 500 kilometre journey on narrow backstreets would be heaven on a bike, and a nightmare in a car. So you need to see all those roads when zoomed out.
Worse, small towns such as Noojee, Walhalla and Woods point are completely missing!
Enter Cycletour.org:
You can plan a route by clicking a start and end, then dragging the route around:
It doesn’t offer safe or scenic route selection. The routing engine (OSRM) just picks the fastest route, and doesn’t take hills into account. You can download your route as a GPX file, or copy a link to a permanent URL.
The other major features of cycletour.org’s map style are:
Bike paths are shown prominently. Rail trails (old train lines converted into bike paths) are given a special yellow highlighting as they tend to be tourist attractions in their own right.
Train lines (in green) are given prominence, as they provide transport to and from trips.
Towns are only shown if there is at least one food-related amenity within a certain distance. This is by far the most important information about a town. Places that are simply “localities” with no amenities are relegated to a microscopic label.
Major roads are dark gray, progressing to lighter colours for minor roads. Unsealed roads are dashed. Off-road tracks are dashed red lines. Tracks that are tagged “four-wheel drive only” have a subtle cross-hashing.
And of course amenities 
useful to cyclists are shown: supermarkets, campgrounds, mountain huts, bike shops, breweries, wineries, bakeries, pubs etc etc. Yes, well-supplied towns look messy, but as a user, I still prefer having more information in front of me.
The terrain data is a 20 metre-resolution digital elevation model from DEPI, within Victoria, trickily combined with a 90m DEM elsewhere, sourced from SRTM (NASA). I use TileMill‘s elevation shading feature, scaled so that sea level is a browny-green, and the highest Australian mountains (around 2200m) are white, with green between. 20-metre contours are shown, labelled at 100m intervals.
I’m really happy with how it looks. Many other comparable maps have either excessively dark hill shading, or heavy contours – or both.
4UMaps
Komoot
OpenCycleMap
Sigma
Google Maps (terrain mode)
MapBox Outdoors
VicMap
I’ve included an assortment of common basemaps, including most of the above. But the most useful is perhaps VicMap, because it represents a completely different data source: the government’s official maps.
Vegetation
There are also optional overlays. Find a good spot to stealth camp with the vegetation layer.
Or avoid busy roads with the truck volume layer. This data comes from VicRoads.
The bike shops layer makes contingency planning a bit easier, by making bike shops visible even when zoomed way out. The data is OpenStreetMap, so if you know of a bike shop that’s missing (or one that has since closed down), please update it so everyone can benefit.
Unfortunately, the site is pretty broken on mobile. But you can download the tiles for offline use on your Android phone using the freemium app Maverick. It works really well.
is.cycletour.org for Iceland. Yes, it’s real – but I don’t know how long I will maintain it.
It’s a pretty major technical undertaking to run a map for the whole world. I’ve automated the process for setting up cycletour.org as much as possible, and created my own version for Iceland and England when I travelled there in mid 2014. If you’re interested in running your own, get in touch and I’ll try to help out.
I’d love to hear from anyone that uses cycletour.org to plan a trip. Ideas? Thoughts? Bugs? Suggestions? Send ’em to stevage@gmail.com, or on Twitter at @Stevage1.
TileMill wants to know: what projection is this data?
If you’re making maps, you will probably need to know something about cartographic projections. Here’s the minimum.
+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs
I help researchers make maps of their research. An archaeologist recently wanted to visualise the distribution of some iron-age artefacts around the Levant, based on a spreadsheet of thousands of rows. Each row represents one kind of artefact at a given site, such as “3 incised bangles, subtype I.b.iv, at Gath.” What are these maps called? I’ll go with “multivariate binary symbol map”.
It sounded like a job for CartoDB, but as the requirements unfolded, she wanted pretty specific cartography, plus a custom base map of rivers, historical boundaries etc. So we used TileMill instead, although we didn’t end up getting all that done.
This is where we got to. Each symbol next to a place name represents the presence of a specific type of artefact. ‘Eitun has pins of Type 1 with “incised decorations”, Far’ah has pins of Type 1 with “incised decorations”, “plain decorations” and “ribbed/grooved decorations”.
The most complex of these maps has 6 different attributes:
With a clearer understanding of exactly what we were trying to achieve, I probably would have done something simpler to calculate each of these attributes, such as using Excel. Instead, I loaded the data into PostGIS and wrote some queries. TileMill supports CSV files directly, but unlike CartoDB, doesn’t load the data into a database, so you can’t run SQL queries.
This post from “The World is a Village” explains how to load CSV into PostGIS, but in summary:
The most interesting line is:
update artefacts set geom = ST_SetSRID(ST_MakePoint(lon,lat),4326);
That’s what converts the raw lon and lat columns into a geometry column so that TileMill can plot it.
To determine “are there any artefacts of type X in location Y”, an easy way is to write a view. Each column is a different subquery, for a different X.
That gives data like this:
So, in TileMill we can now use a filter like [subtype_1a>0] to decide whether to place a symbol.
Because there were so many maps to produce (5 of this type, plus another 11), I created them all in one project, each as a single layer.
The #map1 to #map12 layers refer to a different set of data. Each layer pulls in the same spreadsheet, and styles it identically, with the only difference being a single filter.
That turned out to work really well.
But back to the main problem of showing symbols for attributes. It’s easy to show a single symbol if an attribute is present (like a coffee icon if a site is a cafe). But how do you show 4 symbols simultaneously, without them overlapping?
I thought of two approaches.
It’s theoretically possible to construct a text string, with an appropriate font. The string could look like “A Q Z”, where A gets rendered as a square, Q as a circle and Z as a star. Unfortunately I couldn’t make it work. I just couldn’t find an open truetype font that would behave like this. I tried loading various WingDings fonts, but always got little boxes instead of symbols.
There are projects like Map Icons or Font Awesome which sort of do this, but using web technologies that aren’t compatible with TileMill. The only proof of concept I achieved was using punctuation.
Using fonts makes it very easy to space icons appropriately:
Using punctuation in this way just doesn’t look good.
So the second approach is using traditional markers, and finding a way to position them appropriately. In CartoCSS, there’s no “marker-dx” to offset a marker, but there is “marker-transform“. So you can use SVG transforms, such as translate().
marker-transform:translate(10,-5);
That positions your marker 10 pixels right, and 5 pixels up.
Each different symbol has to be given its own layer (::square, ::circle…), and a different translation offset: (10, -5), (10, 5), (20, -5) etc.
This guarantees that they don’t collide, and mostly looks good:
although it inevitably leads to odd positioning:
With enough time, you could some write some fancy SQL that would stack symbols from the left, avoiding any gaps.
The only other styling of note is that the text labels should appear right-justified, to the left of the exact position. The CartoCSS designation for this is text-horizontal-alignment: left.
You can see the full TileMill project on Github.
Contributing to OpenStreetMap is diversely rewarding: you help other people, you make open data as a whole more viable, you learn a lot about the area you’re mapping, and it’s fun. But sometimes it’s just plain pragmatic. Last weekend, I organised a cycle tour from Bendigo to Avenel, via the O’Keefe Rail Trail, Lake Eppalock, Colbinabbin, Rushworth, Murchison and Nagambie. When I started planning the route, OpenStreetMap looked like this:
The major features are all there, but what’s missing is what matters most to cycle tourists: quiet country roads, and road surfaces. Is there a way to get from Eppalock to Colbinabbin on only sealed roads? Is Buffalo Swamp Rd (near Murchison) really sealed? A great way for me to research is to add to OpenStreetMap: use aerial imagery to add new roads, paying attention to whether they look sealed or not from the air.
So Buffalo Swamp Road is obviously not sealed after all. By the time I was done, the map of the area looked like this:
Notice how many “sealed” roads have turned out to be dirt, but also how many other unmapped little roads have been added to the map.
Once this is done, the steps are:
There’s still lots more to add, but it’s nice that just planning this one trip has significantly improved coverage in a whole region like this.
Github, the online version control repository host for Git, recently added support for GeoJSON files. Sounds boring, right? It actually lets you do something very cool: build your own “dots on a map” website with virtually zero code.
An example of GeoJSON on Github I whipped up. Click it.
Here’s what you need to do.
{
"type": "FeatureCollection",
"features": [{ "type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [144.9,-37.8]
},
"properties": {
"title": "Scooter",
"description": "Here's a dot",
"marker-size": "medium",
"marker-symbol": "scooter",
"marker-color": "#a59",
"stroke": "#555555",
"stroke-opacity": 1.0,
"stroke-width": 2,
"fill": "#555555",
"fill-opacity": 0.5
}
},
{ "type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [144.4,-37.5]
},
"properties": {
"title": "Cafe",
"description": "Coffee and stuff",
"marker-size": "medium",
"marker-symbol": "cafe",
"marker-color": "#f99",
"stroke": "#555555",
"stroke-opacity": 1.0,
"stroke-width": 2,
"fill": "#555555",
"fill-opacity": 0.5
}
}]
}
It’s worth validating with GeoJSONLint.
The test GeoJSON file, as seen on GitHub.
<!DOCTYPE html> <html> <body> <script src="https://embed.github.com/view/geojson/stevage/georly/master/test1.geojson?height=500&width=1000"></script> </body> </html>
If you don’t have a website, site44 is an extremely easy way to get started. You place HTML files into your DropBox, and they get automagicked onto the web, with a subdomain: something.site44.com.
That’s it! What’s especially interesting about the hosting on GitHub is it’s a very easy way to have a lightweight shared geospatial database of points, lines or polygons. Here’s how you could add dots to my map:
Using this method, we have a “review before publish” workflow, and a full version history of every change.
This is a nifty tool for prototyping social mapping applications, but it obviously won’t cut it for production purposes:
Soon you’ll want to build a proper application, using tools like MapBox, CloudMade, CartoDB, Leaflet etc.
Steve Bennett blogs 
Recent Comments