What git does

Git keeps track of changes in a local repository you set up on your computer. Typically that is a directory that contains all your code and optionally the data your code needs in order to run. The local repository contains all your files, but also (in a hidden directory) all the changes to the files you have made. It does not keep track of all files automatically: you need to tell git which files to track and which not. Therefore a repository contains your current tracked files (workspace), an index of files that are being tracked, and the version history.

Every time you make significant changes to the files in your workspace, you have to add the changed files to the index, which selects the files whose changes you want to save, and commit them, which means saving the changes to the history tracking of your local repository.

Often you also setup a remote repository, stored on an online platform like GitHub, GitLab or others. It is simply a remotely-hosted mirror of your local repository and allows you to have your work stored in a safe place and accessible from your other computers and potential collaborators. Once in a while (at the end of the day, or every new commit if you want) you can push your commits, which means sending them to the remote repository so it keeps in sync with your local one. When you want to update your local repository based on the content of a remote repository, you have to pull the commits from the remote repository.

Summary of git semantics

• add: Tell git that you want a file or changes to be tracked. These files/changes are not yet saved in the repository! They are listed as “staged” in the index or staging area for the next commit.
• commit: Save the staged changes to your local repository. This is like putting a milestone or taking a snapshot of your project at that moment. A commit describes what has been changed, why and when. In the future you can always revert all tracked files to the state they were at when you created the commit.
• push: Send previous changes you committed to the local repository to the remote repository.
• pull: Update your local repository (and your workspace) with all new stuff from the remote repository. This command is simple, but potentially destructive, since it overwrites your files with the ones in the remote server. Hence it is not available in the Git GUI.
  • fetch: Get information about the latest commits from the remote repository, but do not apply them to your local repository automatically. This is always safe as it does not change your workspace.
  • merge: Merges two versions (branches) into one, applying the result to the workspace. This includes merging commits from the remote repository with the commits of the local repository. In effect, a fetch followed by a merge is the same as a pull, but it allows you more fine-grained control and is available through the Git GUI.
• clone : Copy the content of a remote repository locally for the first time.
• more advanced:
  • branch : Create a branch (a parallel version of the code in the repository)
  • checkout: load the status of a branch into your workspace

Client setup

1. Launch Git GUI

Launch a program called Git GUI (e.g. from the Applications → Programming menu). Git GUI is a graphical interface to Git that comes with Git itself, and is thus cross-platform and always available. When launched, it looks something like this:

Main screen of Git GUI

2. Create an SSH key pair

In order for GitLab (and other services) to identify that the machine connecting to it is indeed owned by you, there are two options: using a password, or using an SSH key. SSH keys are much more secure than passwords, and it doesn’t require you to enter a password every time you try to communicate with the server. Therefore throughout the course we will use SSH keys.

You can generate a new SSH key pair in Git GUI by going to Help → Show SSH Key and pressing the Generate Key button. It will ask you for a passphrase. This is not a password and is completely optional: it is useful in the case your SSH key is stolen, for instance by a thief stealing your laptop or a virus; however, SSH keys are specific to each machine and are never sent over the network, so most of the time it is completely fine to leave the passphrase empty. If you keep it empty, you will not need to enter it every time you try to push your changes, yet the connection will be even more secure than when using a password.

Once done, you will see your new public key:

SSH public key generated

Account setup

Next, we will link our client with a Git host so that we can download and upload repositories.

3. Log into GitLab

Go to GitLab and log in (using your email address as the user name and your password). You also need to set up two-factor authentication by going to your profile page.

4. Enrol the public key to your user account

The SSH key pair is used to identify that you own the machine. On WUR Linux virtual machines (VMs), the keys are stored on your personal VM, so you can access them from any computer that can access the VM.

Now you need to tell GitLab about your new key. To do that, copy the public key from the dialog, then in GitLab click on your avatar in the top right and go to Edit profile → SSH keys. Paste the public key in the box, give it a title describing your machine, and press Add key. You might need to confirm the key by email for added security.

This only has to be done once (per machine/OS you use GitLab on).

Creating a new repository

5. Create remote repository

Now we are ready to start making new repositories! In GitLab, press the “+” button at the top right, select New repository/project (GitLab uses both terms somewhat interchangeably) and Create blank project. Give it a descriptive name and a short description, choose the visibility of the repository and check Initialize repository with a README.

New repository creation on GitLab

6. Configure repository settings

Explore your new blank repository a bit. In the middle, you have buttons to add new files. Choose to add a LICENSE file, as we have discussed in the previous chapter. See Choose a License for a quick overview of what licenses are available. Make sure to choose a license, otherwise basic copyright applies to your code.

On the tabs to the left, you can find that the repository can have issues and merge requests assigned to them. Issues is what is used to give feedback on code, so try and make a few issues and close them. It is useful to know how to use these, as for personal projects it can be used as a to-do list, and for others’ projects you can use it to report bugs or propose suggestions. You may be surprised how responsive developers can be to newly raised issues!

Example issue on GitLab

Next, check out the repository settings. Under the Members subtab of the Project information tab, you can invite other people to collaborate on your repository. Go ahead and invite your team member to be a collaborator with a maintainer role.

7. Get the URL of your new repository

Now that you have a remote repository, it’s time to create a local repository that links to it! Open the main page of your new repository, click the blue Clone button at the top right of the page, and copy the Clone with SSH address of your new repository.

Blank GitLab repository

8. Clone your repository

Go back to Git GUI, and press Clone Existing Repository. Paste the URL you just copied to the Source Location field, and choose a directory you want to store your code in in the Target Directory field. Note: the Target Directory must not already exist! Git GUI will create it for you.

Once you click Clone, you will get a question about whether you trust the remote machine (if you ran git gui from a command line, it will appear stuck, but actually the question will appear in the terminal and you need to answer it there). You need to answer this with yes (the full word). This puts the GitLab server into a list of trusted servers, to guard against potential impostor servers.

You will end up in an empty Git GUI window:

Git GUI in an empty directory

9. Tell Git who you are

Before you start using Git, you should tell it what your name and email address is. You need to do that only once per Git installation. You should go to Edit → Options… and fill out the Global (All Repositories) options User Name and Email Address. These will be displayed on GitLab.

git config --global user.name "Your Name"
git config --global user.email you@example.com

Working with Git GUI

10. Make changes

To see Git in action, you need to make some changes in your repository. Try it by creating a new R script file in the directory where you cloned your new project.

Once you are done, go back to Git GUI. If you closed the window, you can get back to your repository by launching Git GUI and clicking on its path in the Open Recent Repository list. If you did not close it, click the Rescan button. You will see some changes:

Changes pending in Git GUI

At the top left corner, the Unstaged Changes panel, you can see all the files that changed in your workspace. If you click on the name of the file, the main panel will show you what changed since the last commit. Unless it is a non-text (data) file, in which case it will just note that something has changed. Note: Git is very efficient with storing changes in text files: these diff files are all it stores internally, it does not copy the whole file on each commit. However, it does not deal efficiently with non-text files, and thus you should limit the amount and size of such files as much as possible.

If you click on the icon of the file in the Unstaged Changes panel, the file changes will be staged and appear at the Staged Changes (Will Commit) panel. These are the file changes you want to save and sent to GitLab. You don’t have to stage all files for each commit, only those you actually want to be tracked by git. You can safely ignore some files such as manual backups, temporary files, and the like and they will remain untracked by git, as long as you never stage them. If you do want to stage everything, you can press the Stage Changed button. If you staged more than you wanted to, you can click on the file icon in the Staged Changes panel to unstage it.

Remember: clicking the name of the file shows the changes you made, clicking the icon of the file stages or unstages the change!

11. Commit changes

Once you staged the files that you want to commit, you need to fill out the commit message. This is a brief description of what changes you made between the last commit and the one you are about to create. The first line you enter is the title of the commit, keep that one short. Subsequent lines are the description. You may notice that the Commit message box does not have a horizontal scrollbar: that is intentional, because your commit message should fit within that box without the need for scrolling. Use new lines to break the text.

If it is the first time you use Git GUI to make a commit, and you haven’t filled out your user name and email, it might complain about it not knowing who you are. In that case go back to step 9.

Next press the Commit button and your commit will be saved locally. A commit is like a saved state: you are always able to roll back the contents of your tracked files to the state they were in when you committed the changes.

In the case you made a mistake (a mistake in the message, forgot to stage something, etc.), you can press the Amend Last Commit button and get right back to where you were when you made the last commit; but use this functionality very sparingly, as it does not work with changes that have already been pushed to GitLab.

10. Push changes to the server

Press the Push button, and confirm the push, to send all your changes to your GitLab repository. You can now refresh the GitLab page to see your changes. Well done!

GitLab repository with content

11. Pull changes from the server

One of the major uses of Git is collaboration and the ability to synchronise changes across different devices. Multiple users can do changes in the same Git repository (as long as you change the repository settings in GitLab to allow another user to do that), and you can work on the same code on different devices yourself. In both cases, it is important to keep all local repositories in sync with the remote repository. That is done via Git GUI by using Fetch and Merge. If you like, you can test it by cloning the same repository in another directory, making changes and pushing them to the server, then using fetch in the other copy.

If there are any changes on GitLab that are not on your local copy yet, in Git GUI go to Remote → Fetch from → origin to download all changes. This will not apply them yet, however.

To attempt to apply the changes, go to Merge → Local Merge…. If all goes well, the changes will be applied.

There may be cases where files go out of sync in incompatible ways, however, like two people editing one file at the same time. In that case you may hit a merge conflict. It is best to try to avoid them. In case it happens, you need to go through the conflicting files in a text editor and edit them by hand, keeping the parts of the files you need. The conflicting parts will be in between lines of of >>>> and <<<< symbols. Once you remove the parts you don’t need (including the separators), you can solve the conflict by committing the changes.

Forks and merge requests

Now we know how to work with Git and GitLab for our personal work, and how to collaborate on a project with your team member. But what if you want to contribute code to someone else who has not given you access rights, or what if you want to review the code before it’s accepted to your repository? That’s where forking and merge requests come in handy (respectively)!

A fork is your own personal copy of someone else’s repository. GitLab allows you to fork any public repository. You want to make forks whenever you want to edit code but do not have direct commit rights.

12. Fork a repository

Go to your team member’s repository that they created by following the steps above, and then click the Fork button at the top right. (If you can’t find it, alternatively you can go to some other repository, and fork it.) You will find a new repository under your profile, by default with the same name as the original.

13. Make changes, commit and push

After you have your own fork, it is the same as having your own personal repository with the code from the original (upstream) repository in it. Clone it locally, make some changes, commit them and push them back to GitLab, as per steps 7-10. You should see that your changes take effect in your own downsteam fork, but not in the upstream repository.

14. Make a merge request

If you are ready to ask the upstream developers to incorporate your code into their repository, go to the Merge Requests tab and press the blue New merge request button. Select your main branch of your fork as the source.

This will show you all the changes you have made, and if that is what you want to propose for the upstream developers to incorporate, give the name for your merge request (changeset) and a description as to why the upstream developers would want to incorporate your code. After you confirm clicking Create merge request, the merge request will be visible in the merge requests tab of the upstream repository:

A submitted pull request

Then it’s up to the upstream developers to perform a code review and either accept or reject the pull request in the end.

Other Git GUI functionality

You might run into a situation when you have made changes in tracked files, but do not want to keep some of the changes. You can revert one file by selecting it in Git GUI, then clicking Commit → Revert changes.

Git GUI not only provides a way to make, push and pull commits, but also to visualise the commit history of your repository in a tree graph. Go to Repository → Visualise Master’s History to see it. For larger and more complex projects with lots of contributors and merges, it might look like some sort of a subway map:

Git GUI history (gitk)

The history view also allows you to reset the state of the repository to any previous commit by using the context menu. Note, however, that you can only push if you are on the latest commit. So the easiest way to revert changes is to copy over the files to a temporary directory outside of git, reset back, and move the files back into your repository.

You can also browse the history of a repository from your Git hosting service, and GitHub/GitLab even allow editing files from a web interface.

Below you can see a visual summary of what we have described above.

Git workflow overview

That’s it: now you know how to keep track of all your files, so you will never lose them again, and no longer have to worry about making backups or saving multiple versions. In addition, this is the way that free and open-source code development happens in actuality. Also, the exercises and assignments in the course will be delivered and submitted this way, so make sure you are familiar with the whole process!

in R main.R

# Team Teamname (John Doe and Jane Smith)
# January 2020
# Import packages
library(raster)
library(sf)
# Source functions
source('R/function1.R')
source('R/function2.R')
# Load datasets 
postboxes <- st_read('data/postbox_locations.gpkg')
# Then the actual commands

and now in Python

# Team Teamname (John Doe and Jane Smith)
# January 2020
# Import packages
import geopandas as gpd
import matplotlib.pyplot at plt

# Import functions
import Python.functions as funcs

# Load datasets 
postboxes = gpd.read_file('data/postbox_locations.gpkg')
# Then the actual commands