Powershell & GIS

The short version

I’ve never really liked Python, and I’m sure that this will upset many people! Why do I not like Python? I dislike the rules around indents, and powershell is far more forgiving and it’s just easier to read.

These are my ramblings on using a single Powershell script and GDALs ogr2ogr command to process data.

Disclaimers: I am a Mac fan boy, and use Macs at work and in play 99.8% of the time. So this post is written with Mac users in mind.

Fact: Powershell is Open-Source and runs on Mac, and not just Windows.

Install Powershell & GDAL:

I use homebrew to install Powershell and GDAL on a Mac, it's nice and easy. Mircosoft have a nice page on setting up Homebrew and Powershell here, and there is a Homebrew page on the install of GDAL here

I use VS code to run & test my Powershell but the below could equally be run in terminal.

Quick tests:

To test that GDAL is installed in terminal use:

gdal-config --version 

To test that Powershell is working using the below in terminal:

pwsh -v

Lets dive into some proper code:

In the below example we are just using Powershell to call and download a geoJSON file to a destination directory and output the file name as ca_DATESTAMP.geojson

$url = "https://files.planning.data.gov.uk/dataset/conservation-area.geojson"
$dateDOWNLOAD = Get-Date -Format "dd-MM-yyyy"
$dataDOWNLOADname = "ca_$dateDOWNLOAD.geojson"
$destination = "/SOME_PATH/data"
$GEOJSONdestination = "$destination/$dataDOWNLOADname"
Invoke-WebRequest -Uri $url -OutFile $GEOJSONdestination

Now let's add in some GDAL into the mix by adapting the Powershell by adding in an ogr2ogr to convert the GeoJSON to SHP, but also adding additional variables for the name and destination of the SHP.

Run this in Terminal (invoking pwsh first) or save to a .ps1

$url = "https://files.planning.data.gov.uk/dataset/conservation-area.geojson"
$dateDOWNLOAD = Get-Date -Format "dd-MM-yyyy"
$dataDOWNLOADname = "ca_$dateDOWNLOAD.geojson"
$dataSHPname = "ca_$dateDOWNLOAD.shp"
$destination = "/SOME_PATH/data"
$GEOJSONdestination = "$destination/$dataDOWNLOADname"
$SHPdestination = "$destination/$dataSHPname"
Invoke-WebRequest -Uri $url -OutFile $GEOJSONdestination
ogr2ogr -f "ESRI Shapefile" $SHPdestination $GEOJSONdestination

Ok this script could be shorter, an example of that is below but the long version gives you flexibility to

a) Download the source data e.g GeoJSON.

b) Set where you want the data to be downloaded to

c) Gives you room to expand the script to do more funky things like test if the GeoJSON has downloaded.

d) Gives you an easy to use method to replicate and rename variables.

While the below is short and sweet, the source data isn't downloaded, and the destination of the SHP is set to the directory where the Powershell is run from.

$url = "https://files.planning.data.gov.uk/dataset/conservation-area.geojson"
$dateDOWNLOAD = Get-Date -Format "dd-MM-yyyy"
$dataSHPname = "ca_$dateDOWNLOAD.shp"
ogr2ogr -f "ESRI Shapefile" $dataSHPname $url

How might I flesh my script out to be more funky?

##SET A DATE STAMP
$dateDOWNLOAD = Get-Date -Format "dd-MM-yyyy"

##SET THE URL AND DATA TO DOWNLOAD
$downloadURL = "https://files.planning.data.gov.uk/dataset"
$downloadDATA = "conservation-area.geojson"
$url = "$downloadURL/$downloadDATA"

##SET NAMES FOR THE FILES TO PROCESS
$dataDOWNLOADname = "ca_$dateDOWNLOAD.geojson"
$dataSHPname = "ca_$dateDOWNLOAD.shp"

##SET DIRECTORIES
$GEOJSONdestination = "/SOME_PATH/data/historicDOWNLOADS/$dataDOWNLOADname"
$SHPdestination = "/SOME_PATH/data/currentDATA/$dataSHPname"

##DOWNLOAD THE DATA 
Invoke-WebRequest -Uri $url -OutFile $GEOJSONdestination

##TEST IF THE GEOJSON HAS DOWNLOADED AND IF IT HAS DO THE GDAL
if (Test-Path $GEOJSONdestination) {
    ogr2ogr -f "ESRI Shapefile" $SHPdestination $GEOJSONdestination
    Write-Host "All done"
    exit
  }
  else{
    Write-Host "There was an error"
    exit
  }

In fact this could even be shortened. Equally it could be expanded to test that that URL path is correct and passes back a 200 status, we could also have logging to a text file, expand it out to send Teams messages, send the data straight into PostGIS, but that might be for another day.