maptastik
4/25/2016 - 3:01 PM

Workflow for downloading data from ArcGIS service and preparing for work with Mapbox services

Workflow for downloading data from ArcGIS service and preparing for work with Mapbox services

#Preparing data from an ArcGIS REST Service for Use with Mapbox Services

##Situation##

We want to make a simple web map or static map of Graves Rd in Scott County using Mapbox services. We don't want to open GIS software because that takes forever. This workflow generally describes how we can access data from an ArcGIS REST service and process it for use with Mapbox.

##What we'll be using:##

  • Example Service: http://gis.gscplanning.com/arcgis/rest/services/Transportation/MapServer/0
  • Python 2.7.x
    • This is installed with ArcGIS and QGIS so you likely already have it
  • Mapbox account
  • shapefile-ags
  • 7zip for the command line
  • arcpy (optional)
    • Used for dissolving line features. Would be better to have a FOSS solution, but I'm not sure what that is at the moment. If you don't have ArcGIS, you can still use QGIS for the same functionality we'll be using. Additionally, it may be possible to access QGIS from the command line but I'm not quite up on how that works.
  • mapshaper
  • geotype (optional)
    • This is a nifty little utility for viewing geographic data in your terminal. This workflow doesn't hinge on it, but it is a nice tool for checking the output.
  • mapbox-cli-py (optional)
    • If you want to upload our data as a vector tileset that can be styled in Mapbox Studio, this is a nice tool to facilitate that. you'll need to generate an access token where the scope data:upload is turned on. You can do that on your account's token page, https://www.mapbox.com/studio/account/tokens/

##Process##

###Query and download the service as a zipped shapefile###

We want only the road segments where the field FULLNAME='GRAVES RD'. We'll name the output zip graves_rd.zip.

shapefile-ags http://gis.gscplanning.com/arcgis/rest/services/Transportation/MapServer/0 --where="FULLNAME='GRAVES RD'" --file-name=graves_rd.zip

###Unzip using 7zip###

shapefile-ags returns a zipped directory so we need to extract it. It's possible doing this from the command line in OSX/Linux is straightforward, but for Windows users it's less so. We'll use 7zip to help us unpack graves_rd.zip.

7z e graves_rd.zip

If you look at the output, the shapefile will be named after its geometry. In this case that's POLYLINE.shp.

###Dissolve line features into a single feature (Optional)###

We'll use this to the Dissolve tool using arcpy to dissolve the individual line features into a single line feature. We might want to do this in Mapbox Editor individual feature. There is no automatic way to style based on a common attribute that I'm aware of.

We'll export our dissolved POLYLINE.shp to polyline_dissolve.shp. We'll dissolve on the FULLNAME field.

import arcpy
arcpy.env.workspace = "<whatever the directory is>"
arcpy.Dissolve_management("POLYLINE.shp", "polyline_dissolve.shp",["FULLNAME"])

###Convert to GeoJSON using mapshaper###

Mapbox Editor doesn't accept shapefiles as a data format so we need to convert our file to something else. In this case, we'll use mapshaper to convert our dissolved polyline of Graves Rd into GeoJSON.

mapshaper -i polyline_dissolve.shp -o graves_rd.geojson format=geojson

###Check that it all makes sense with mapshaper and geotype (Optional)###

At this point it might be good to see if your data make sense. We'll use mapshaper to check that there's only one feature:

mapshaper graves_rd.geojson -info

...geotype to print a rendering of that feature to the terminal so we can make sure the feature looks generally how we'd expect it to look!

geotype graves_rd.geojson

At this point if you just want to make a simple map using Mapbox Editor, you're done. Just drag your final GeoJSON file into Editor and you're set. However, if you want to create a tileset from the data you processed, you can use mapbox-cli-py to upload your data to Mapbox.

###Upload to Mapbox with mapbox-cli-py (Optional)###

We'll do three things. We'll create a Map ID, <account_name>.graves_rd, the data we're uploading, graves_rd.geojson, and a name/alias for the data, graves_rd.

mapbox upload <account_name>.graves_rd graves_rd.geojson --name graves_rd

Once our data are uploaded, we can use them in Mapbox Studio and Mapbox GL JS and more!