Introduction

Good afternoon and welcome to the python part of this course! Today we will introduce how we will work with Python during this course and show some alternative methods. If you are unfamiliar with Python and/or feel that you need more training, follow one of the Datacamp courses as introduction into Python before today:

• Introduction to Python | recommended to follow if you haven’t any scripting experience so far
• Python for R users | recommended if you have experience already in R

Today’s Learning objectives

• Know how to work with virtual environments: Conda + Mamba
• Know how to run a Python script from the terminal
• Get introduced to Python editors and IDEs
• Refresh Python programming knowledge
• Familiarize yourself with some visualization techniques

Python package management with Conda

The high availability of packages is also a threat sometimes. Packages also make use of code and functionality from other packages. We call this dependancy. If a piece of software is developed depending on a package, but this packages changes later on, the initial piece of software not work anymore. Different packages require different dependencies that they are built upon. It is important to make sure all these dependencies are working together and that the right versions of the dependencies are used. Luckily, a set of tools exist for installing and managing Python packages. It is possible to install packages on your main Python installation (This main python is called the base python interpreter), but sooner or later you will get conflicting Python packages since packages have varying dependencies and you might have installed several versions of the same package. It can even break your system Python interpreter.

Therefor, we strongly recommend to use a Python package manager that uses of virtual environments. This way, you can create a separate environment on your machine for each project. In these environments, any dependency of the project, such as python packages, other software or C libraries can be installed. We will use them here for installing Python packages. Packages installed in one environment do not interfere with your base Python or with other environments. Additionally, it is possible to export and share a complete list of the requirements for your (open source) project with collaborators or users of your code. In this way, collaborators can install the dependencies and start working with your code right away, instead of struggling with dependecies first (this is called Dependency Hell.

Mamba installation

Conda is a widely used tool to manage Python environments and install packages. For this course, we will make use of Mamba instead, a fast drop-in reimplementation of the Conda package manager. It has its core parts implemented in C++ for maximum efficiency, makes use of parallel downloading of repository data and package files using multi-threading, and uses libsolv for (much) faster dependency solving. To install Mamba in your Linux environment, we have prepared a short Bash script for you. Just run the following lines of code, line by line, in a new terminal window.

git clone https://github.com/GeoScripting-WUR/InstallLinuxScript.git
cd InstallLinuxScript/user
chmod u+x ./install.sh
./install.sh

This will install Mamba into ~/mamba. Finally, restart your terminal to be able to use Mamba and Conda in the terminal. Next, let’s see how to use Mamba in case you want make new virtual environments by yourself, or install packages after creating the environment.

Mamba usage

Mamba creates isolated conda environments with sets of packages, that do not interfere with your base Python or with other conda environments. To create an environment:

mamba create --name geotest python numpy folium

This creates a new environment called geotest with Python and NumPy installed into the conda environment. Another option is to create an environment from a file. The advantage of using a file is that the file, with the dependencies can be shared within a project. For this applciation a YAML file is used. In this file all required packages are listed and if required which version should be used. An example of a YAML file is the following:

name: geotest
dependencies:
  - python
  - numpy
  - folium

The first line defines what the evnironment will be called (geotest in this example) and what packages should be installed (python and numpy). As you can see, this definition of the geotest environment is the exact same as the geotest environment as defined before. To create the geotest environment from a file, you first need to create a YAML file. You can do this by opening a text editor such as gedit or RStudio, pasting the lines above into a new file, and saving it to a new file named env.yaml, or however you want to call it and use the argument --file (or --f in short):

mamba env create -f env.yaml

Let’s list the currently available environments:

mamba info --envs

Mamba puts an asterisk (*) in front of the active environment. Now we activate the environment. While Mamba replaces Conda for most commands, it does not replace Conda entirely. For example, this is not the case for (de)activating environments:

source activate geotest

After this, the current environment is shown in parentheses in front of your prompt ((geotest)$). Note that the activated environment is only valid for the shell in which you activated it. For instance, if you close the shell window and open a new one you will have to activate it again.

After creating a conda environment, (additional) Python packages can be installed. There are three possible ways to install packages, which we list below.

• Using Mamba to install and manage conda packages. This downloads conda packages using conda channels, which are URLs to directories containing the conda packages. Generally, installing conda packages using Mamba is the preferred method.
• Using pip to install packages and Mamba to manage these packages. pip is available for Windows, macOS and Linux. pip can also install binary wheels on Windows. You should generally not install packages from pip in a conda environment unless it’s the last resort. This is because after you use pip to modify an environment, you can no longer use conda/mamba to do so (trying that will break your environment, because pip does not communicate its changes to Conda). Hence install packages with mamba that you can first, and only then use pip, and then never touch the environment with mamba again (delete and start fresh if you need to).
• Using the distribution’s package manager (only on Ubuntu, that is sudo apt-get install python-*).

The mamba search command searches a set of channels. By default, packages are automatically downloaded and updated from the default channel. To search for a package, type:

mamba search pandas

This gives a list of all packages that have “pandas” in the name and lists all available versions. To install:

mamba install pandas

This installs the latest compatible version of Pandas. Note that this would install it into your currently activated environment.

Note that you can also install multiple packages at the same time:

mamba install geopandas matplotlib

Mamba is also able to install some non-Python packages that have Python bindings. For example with mamba we can install jupyter, a web based editor for writing and evaluating code per cell (more on jupyter later). This is useful for making sure your Python and binary versions match and do not interfere with the system-wide ones.

Some additional helpful utilities for package management in this context are:

• mamba list to check which packages are installed in root or in the active environment;
• python --version or gdal-config --version to check which Python or GDAL version is used in the environment;

Removing packages is just as simple:

mamba remove geopandas pandas folium

Now, we deactivate the environment and return to base environment.

source deactivate

Running a Python script in the terminal

Within a conda environment, Python can be started directly, or can be called to run a script file. To start Python directly:

python

Now, you can type Python expressions that will be executed one by one:

import sys
print(f'Good morning, you are running Python version: {sys.version}')

To go back, type:

exit()
# or 
quit()

Usually, we do not want to run expressions one by one, but build scripts instead, to ensure transferability and reproducibilty. Create a new text file and (re)name it (to) test.py. Open it, for example with a text editor, paste in the code you used above (import sys etc.), and save the script. Navigate in the terminal to the location where this script is stored, using cd. Finally, run the script with:

python test.py

The output is printed to the terminal. Running a script from the terminal is less error-prone than running it from an IDE (see the next section), such as Spyder or Visual Studio Code, as IDEs often keep variables in memory after the script has finished running. Therefore, running a script from the terminal is a good final test before submitting an exercise or assignment.

Visual Studio Code

Visual studio code is a complete IDE that is used by many developers, Python developers but also developers for other languages. To get started in VSCode we will set up a basic project structure and run some very basic code. There is a difference opening a file and opening a project in VSCode. Opening a file will let you edit that file and possibly run it. Opening a directory or a project consisting of multiple (sub)directories will let you set a default interpreter, search and replace throughout the project and navigate to function imports and many more advantages. We recommend you to generally use a seperate project for each tutorial, exercise and assignment. In that way you can use dedicated environments and keep a coherent folder structure throughout the course.

Create the following files:

  MyPackage/
  ├── environment.yaml
  ├── main.py
  └── MyPackage
      └── __init__.py

Create an empty directory MyPackage and open it using the button that shows in the main screen when you open VSCode. Create the files as the file tree shows above. Open main.py, and write the following code:

print('hello world')

VSCode directly recognizes the python file extension (.py) and does some suggestions. In the bottom right you can what type of file VSCode thinks you are working in, it also shows a number, most likely 3.12.x if you are following this course in 2025. This number is the python version of the interpreter that is now running, more on this later. In the left there is a column with some buttons. These buttons are, from top to bottom, for:

• Files, showing the project files. Clicking on a file opens it.
• Search, giving the option to searching and replacing text throughout the project
• version control, showing the git status of the project.
• Run and debug, showing results and the state during debugging
• Extensions, for installing and maintaining extensions.
• Testing, for testing code (we will not use this throughout the course)

Open the environment.yaml and paste the following in there:

name: geotest
dependencies:
  - python
  - numpy
  - folium

def some_function(a_number):
    print(f'I am printing the number {a_number}') # This is is a formatted string

some_function(5)

# One of the following
mamba env create -f environment.yaml

# or 

source activate geotest 

import folium

m = folium.Map(location=[51.9700000, 5.6666700], zoom_start=13)
m

Different ways to run notebooks

We have seen how we can use notebooks in the REPL, but there are different ways and this is the power of notebooks. Originally, notebooks were introduced by Jupyter.

jupyter notebook

spyder

Setting up the environment

Now that we know how to set up an environment and run code, lets use this new knowledge and run some Python code. Again, during this course advise you to code in VSCode, as this IDE is the recommended IDE for the Python part of this course.

Create a directory structure for this tutorial using the terminal:

cd ~/Documents/
mkdir PythonRefresher #or give the directory a name to your liking
cd ./PythonRefresher
mkdir output

And open it with VSCode by clicking File and Open Folder.

We only made a directory for output, because no input data or separate scripts are created in this tutorial. Next, we will create a conda environment from a file. First create a text file, (re)name it (to) refresher.yaml, and copy the following content into the file:

name: refresher
dependencies:
  - python
  - numpy
  - matplotlib
  - geopandas
  - spyder

Now, create a new conda environment based on this file. You can do this in a ubuntu terminal, or a terminal in VSCode. To open the terminal in VSCode press Ctrl + shift + P and type terminal and select Create New Terminal (With Profile)

mamba env create --file refresher.yaml

Once everything is installed, associate the refresher environment to this project. The easiest way is press Ctrl + shift + P and type Python: select interpreter.

Create a new Python script and save it.

Important to note: for compatibility, it is best to install packages from the same channel as much as possible. Given that packages in the file refresher.yaml are installed from the conda-forge channel, it is wise to use this same channel when you want to install additional packages in your environment.

Quick refresher

In the tutorial about R and Python we have gone over the differences and similarities of python and R. This tutorial also contains some basic python syntax, in this tutorial we assume you know this content, but we will go over a few basics here as well. The examples below are mostly meant for reference purposes, we assume you understand most of this refresher already.

# Integer
age = 25

# Float
height = 1.75

# String
name = "John Doe"

# Boolean
is_student = True

# Print a name
print(name)

# String formatting and printing 
print(f'{name} is {age} years old and is {height} meters tall.')

a = 10
b = 5

addition = a + b
subtraction = a - b
multiplication = a * b
division = a / b
modulo = a % b
exponentiation = a ** b

print(addition, subtraction, multiplication, division, modulo, exponentiation)

x = 15

if x > 10:
    print("x is greater than 10")
elif x == 10:
    print("x is equal to 10")
else:
    print("x is less than 10")

# For loop
for i in range(5):
    print(i)

# While loop
count = 0
while count < 5:
    print(count)
    count += 1

# Creating a list
fruits = ["apple", "banana", "orange"]

# Accessing elements
print(fruits[0])  # Output: "apple"

# Adding elements
fruits.append("grape")

# Removing elements
fruits.remove("banana")

# Length of the list
print(len(fruits))  # Output: 3

# Function to add two numbers and return the result
def add_numbers(a, b):
    return a + b

result = add_numbers(5, 3)
print(result)  # Output: 8

import os
def make_directory(directory_name):
    # Check if directory already exists
    if not os.path.exists(directory_name):
        os.mkdir(directory_name)

import requests
import zipfile
import os

def download_and_unzip(url: str, extract_to: str = "."):
    """
    Downloads a ZIP file from a URL and extracts it.
    This function was written by ChatGPT and edited and checked for correctness by Arno Timmer

    Args:
        url (str): The URL to the ZIP file.
        extract_to (str): Directory where the files will be extracted.
    """
    # Get filename from URL
    local_zip_path = os.path.join(extract_to, url.split("/")[-1])

    # Download the file
    response = requests.get(url)
    response.raise_for_status()

    with open(local_zip_path, "wb") as f:
        f.write(response.content)

    # Extract the zip file
    with zipfile.ZipFile(local_zip_path, "r") as zip_ref:
        zip_ref.extractall(extract_to)

    # remove the zip file
    os.remove(local_zip_path)

def multiply_value(a):
    ''' 
    This function multiplies value a with 100 and with 1000. It returns both the results.
    '''
    hundredfold = 100 * a 
    thousandfold = 1000 * a
    return hundredfold, thousandfold

# When called like this both values are stored as a tuple
results = multiply_value(5)
print(type(results))
print(f'We calculated the hundredfold of a number: {results[0]} and the thousandfold: {results[1]}' )

# But we can also call it like this:
hundreds, thousands = multiply_value(10)
print(f'We calculated the hundredfold of a number: {hundreds} and the thousandfold: {thousands}' )

# Creating a dictionary
person = {
    "name": "Alice",
    "age": 30,
    "is_student": False
}

# Accessing values
print(person["name"])  # Output: "Alice"

# Adding a new key-value pair
person["occupation"] = "Engineer"

# Removing a key-value pair
del person["is_student"]

import pandas as pd

data = {
    'Name': ['Alice', 'Bob', 'Charlie'],
    'Age': [25, 30, 22],
    'City': ['New York', 'San Francisco', 'Chicago']
}

df = pd.DataFrame(data)
print(df)

# Display the first few rows of the DataFrame
print(df.head())

# Get statistical information about the DataFrame
print(df.describe())

# Access a specific column
print(df['Age'])

import geopandas as gpd

# Dummy data for the GeoDataFrame
data = {
    'Name': ['Location A', 'Location B', 'Location C'],
    'Latitude': [40.7128, 34.0522, 41.8781],
    'Longitude': [-74.0060, -118.2437, -87.6298]
}

# Create the GeoDataFrame with a single line of code
gdf = gpd.GeoDataFrame(data, geometry=gpd.points_from_xy(data['Longitude'], data['Latitude']))

# Display the GeoDataFrame
print(gdf)