WUR Geoscripting WUR logo

Week 3, Lesson 12: Google Earth Engine

Good morning! Today we will start working with Google Earth Engine.

If you do not have an activated account yet, register an account now and hopefully your account will be activated soon. If it doesn’t work out, ask someone to share a repository with you, that should grant you an account.

Today’s schedule

We will have a look at the purpose of Earth Engine, what Earth Engine can do for you and how to use Google Earth Engine with JavaScript and Python.

Purpose Earth Engine

The purpose of Earth Engine is:

  • Perform highly-interactive algorithm development at global scale
  • Push the edge of the envelope for big data in remote sensing
  • Enable high-impact, data-driven science
  • Make substantive progress on global challenges that involve large geospatial datasets

The platform for global analysis is there, but it needs specialists, students and researchers like you, to use the Earth Engine platform to do high-impact data-driven science.

What does Google Earth Engine do?

Google Earth Engine is a cloud-based platform for planetary-scale environmental data analysis. The main components of Earth Engine are:

  • Datasets: A petabyte-scale archive of publicly available remotely sensed imagery (such as Landsat, Sentinel and Modis) and other data (e.g. elevation data). Explore categories of data in the archive or browse and search datasets.
  • Compute power: Google’s computational infrastructure optimized for parallel processing of geospatial data. Learn more from Gorelick et al. (2017).
  • APIs: APIs for JavaScript and Python (see GitHub) for making requests to the Earth Engine servers. These docs focus on JavaScript; (Guide for JavaScript API). For Python, see the Python install guide and the Python examples in the Earth Engine GitHub repository.
  • Code Editor: An online Integrated Development Environment (IDE) for rapid prototyping and visualization of complex spatial analyses using the JavaScript API Code Editor docs.

Google Earth Engine: Code Editor in JavaScript

Today you will learn about Google Earth Engine by going through 9 tutorials made by Google. In the morning you will work on these tutorials. Have fun working in the cloud!

  1. Get familiar with basic JavaScript syntax
  2. Get familiar with advanced JavaScript syntax
  3. Get familiar with the Code Editor
  4. Visualizing Images and Image Bands
  5. Computations using Images
  6. Image Collections
  7. Compositing, Masking, and Mosaicking
  8. NDVI, Mapping a Function over a Collection, Quality Mosaicking
  9. Exporting Charts and Images
ProTip: at almost any point you can use .aside(print) for debugging purposes. Come back to this Debugging guide when in need

Google Earth Engine is a nice platform for processing and visualization. Even though raw data can be downloaded, such an attempt would defeat its intended goal of uploading the code next to the data instead of downloading data next to the code.

Question 1: Is the code on the online editor reproducible for other users?

If you finished these tutorials and want to do more advanced stuff with the Code Editor you can have a look at other tutorials made by universities or at more advanced guides of Earth Engine (you can find the guides in the left pane) about the Image (e.g. gradients, edge detection, object detection), ImageCollection, Geometry & Features, Reducer, Join, Chart (e.g. time series analysis), Array, User Interfaces, Specialized Algorithms, Asset Management, Custom Applications and Earth Engine setup.

Google Earth Engine: Python API

Although Earth Engine focuses on using JavaScript in the online Earth Engine Code Editor, an Earth Engine Python module was developed. The module allows you to connect via an API to to the Earth Engine servers.

This presentation on the Earth Engine user summit explains more about the Python API.

Earth Engine normally uses Python 2.7. However Earth Engine is trying to make code compatible for both Python 2 and Python 3. Since Python 3 is the future, we will use Python 3. Your Google account needs to be accepted to authenticate your local Earth Engine Python module. So we need to setup the Python environment and get authenticated:

# Create Conda environment
conda create -n ee_py3 -c conda-forge python=3 google-api-python-client pyCrypto spyder jupyter

source activate ee_py3

# Ensure that a crypto library is available
python -c "from oauth2client import crypt"

# Install earthengine-api
pip install earthengine-api

# Authenticate earth engine
earthengine authenticate
# Follow procedure to authenticate and paste the access token in your terminal

# Check if earth engine has been installed
python -c "import ee; ee.Initialize()"
# If you don't get an error, you are good to go

After setting up the Python environment, you can start a jupyter notebook or spyder and run some Python code.

# Import the Earth Engine Python Package
import ee

# Initialize the Earth Engine object, using the authentication credentials.

# Print the information for an image asset.
image = ee.Image('srtm90_v4')

Now we want a Python script that shows a time series of Landsat 8 imagery.

import ee
from ee import batch

## Initialize connection to server

## Define your image collection 
collection = ee.ImageCollection('LANDSAT/LC8_L1T_TOA')

## Define time range
collection_time = collection.filterDate('2013-04-11', '2018-01-01') #YYYY-MM-DD

## Select location based on location of tile
path = collection_time.filter(ee.Filter.eq('WRS_PATH', 198))
pathrow = path.filter(ee.Filter.eq('WRS_ROW', 24))
# or via geographical location:
#point_geom = ee.Geometry.Point(5, 52) #longitude, latitude
#pathrow = collection_time.filterBounds(point_geom)

## Select imagery with less then 5% of image covered by clouds
clouds = pathrow.filter(ee.Filter.lt('CLOUD_COVER', 5))

## Select bands
bands = clouds.select(['B4', 'B3', 'B2'])

## Make 8 bit data
def convertBit(image):
    return image.multiply(512).uint8()  

## Convert bands to output video  
outputVideo = bands.map(convertBit)

print("Starting to create a video")

## Export video to Google Drive
out = batch.Export.video.toDrive(outputVideo, description='Netherlands_video_region_L8_time', dimensions = 720, framesPerSecond = 2, region=([5.588144,51.993435], [5.727906, 51.993435],[5.727906, 51.944356],[5.588144, 51.944356]), maxFrames=10000)

## Process the image
process = batch.Task.start(out)

print("Process sent to cloud")

To explain what is going on in the script: an ImageCollection is created and queried by filtering on date, location, clouds, bands. The ImageCollection object holds the information of the query and is sent to the Google Earth Engine server. The Google Earth Engine server then performs the task of gathering the data, making the imagery into a video and then exporting it to your Google Drive.

There are some more advanced Python scripts made by the Google Earth Engine developers that work with bqplot and ipyleaflet. If you want you can have a look at those too.

# Install bqplot from conda-forge channel and ipyleaflet with pip
source activate ee_py3
conda install -c conda-forge bqplot
pip install ipyleaflet
jupyter nbextension enable --py --sys-prefix ipyleaflet

References and more info

Most of the information on Google Earth Engine in this tutorial comes directly from Google and from universities, whom developed tutorials for students to work with Google Earth Engine.