My own GIS weapon of choice was born of a development by the U.S. Army Corps of Engineers’ Construction Engineering Research Laboratory over 35 years ago - and I attest that GRASS GIS is still one of the best GIS desktop tools in my opinion (but that’s for another Geotech Friday).
However, two other weapons that have recently come to the fore and have changed how we process and store data are; gaming and Google. Before these came along, you could feel the pressure building up in the remote sensing world.
Contributing to the pressure build up was the increased accessibility of data generated from drones, high resolution satellite imagery options, camera technology and battery technology. I still remember the day I heard of the extraordinary decision by the US Geological Survey to make Landsat data freely available.
However, thanks to tech giants like Google, we now have access to greater graphics processing units (GPU) with increased computing power. This is born from the fierce competition in the race of data availability vs. processing capability and storage.
With this, we have welcomed cloud computing services and fibre optic network links to the party, allowing us to store massive amounts of data with the flexible processing power provided by third parties (mainly Google and Amazon).
One of the major players that we’re going to explore today is Google Earth Engine.
How Google Earth Engine operates in default
Google Earth Engine (GEE) a tool that has an immediate impact on the way you approach certain tasks. This is because it works equally well as a global quick look window, a fast prototyping tool and an all-out finished application builder.
Not only does GEE provide a platform to easily access open source data on a global scale, it also provides the computing power to be able to do something with it.
Python API as an alternative IDE in GEE
Earth Engine data can be passed to Python in array form, and almost seamlessly be picked up as a Numpy Array object. There is little need to explain the potential analysis that can be done once this is achieved - the full gamut of Python (and R, and Matlab...) functionality is available from this point.
I have two observations about this method of operation:
- There is an up-front overhead. The decision to “bring the data home” requires a forced transfer of the contents of an Earth Engine object to the client-side environment. This is essentially a divorce from the Earth Engine, forcing it to resolve all latent computations (which are always carried out “just in time”) to convert the object to GeoJSON or TFRecord form.
- The transfer to a Python object is relatively easy because the Numpy constructor will understand the GeoJSON form directly. In the same way, this transfer may be done to any programming language.
Apart from the availability of all Python packages, there is another advantage to the use of the Python API, and this is to do with workflow and collaboration.
IPython provides a Python shell designed for interaction and exploratory coding. Using the web-based Jupyter notebook, this involves the writing and running of individual code cells, interspersed with Markdown explanatory notes.
Collaboration in the IPython notebook environment is done by sharing notebook files on Github, or using Google’s Colaboratory (Colab). Colab provides an IPython notebook in a virtual computer environment with free access to GPUs and access to the user’s Google Drive via a local link. This approach makes sharing notebooks very easy.
The Jupyter Notebook format (jupyter.org)
Are there downsides to the Python approach?
IPYLeaflet map control taken from the ipyleaflet documentation
The IDE provides the means to go only so far and throws in some analytical functionality. The ability to export TFRecords means that you can take the data out of the Earth Engine and into the realm of analytical capabilities - only limited by your ideas.
In terms of the immediate potential of what it provides, the functionality of Google Earth Engine can be considered as complete. New datasets are added as they become available. When questions arise that can’t be met, Python packages invariably appear quickly to meet the demand. I believe that a major turning point in remote sensing occurred when open source data came together with (near) open source access to computing power with Google Earth Engine. This will prove to be a game-changer, particularly in the scale of uptake by the global community. The consequences can only be imagined at this stage, but it’s great to be a small part of it.
Back To News Stories