Git for Wesnoth Crash Course

Foreword

(An essay by, mainly, iceiceice.)

This page is intended for new Battle for Wesnoth developers and contributors who have never used Git before. I'm hoping that it will be written mainly by newbies like myself who have just figured out enough about Git to use it successfully, and therefore not include a bunch of information superfluous for that purpose. At the time that I am writing this, I have managed to write about eight GitHub pull requests and get them merged into Wesnoth, and I hope you will be able to do the same after reading this.

What is Git?

The most effective "absolute beginner" introduction to Git that I found when I started is here: gitref.org. - (Note: this gitref.org link seems to have died. Another resource, that is only slightly less introductory-level, is Pro-Git Book. Don't worry about reading the whole thing though.)

I suggest that you read this through, beginning to end, navigating through the pages by using the red arrows at the bottom of each page. You can skip the part about "stashing", but you should carefully read and understand the parts about:

• cloning a repository;
• checking out a branch, creating a new branch, and merging branches;
• adding and committing changes;
• viewing diffs to make sure you understand exactly what you are committing;
• resetting your repository state in case you make a mistake and need to undo some steps, and using "git log" and "git reflog" to assist with this; and
• moving content between local and remote repositories, with "push" and "pull".

Setting up our workflow for Wesnoth in Git

It is common when reading about Git to see the terms "upstream" and "downstream". This is often confusing at the beginning -- after all, information is usually flowing in both directions; sometimes, you are getting code from the Wesnoth repository, and sometimes you are sending code to it. Which way is up and which is down?

Analogously to a river, downstream is the direction toward which most information flows. Any user that simply wants to build Wesnoth from source, and not make any changes to it, will generally do so by cloning or pulling from <https://github.com/wesnoth/wesnoth>. So wesnoth/wesnoth (the main Wesnoth repository) is the upstream repository, and the users are downstream.

If you are a developer, you will fork the upstream repository and edit the source code, and -- hopefully, with work, eventually -- get your edits pulled into the upstream repository. That is the development cycle, and contributing changes back to upstream is what makes you a developer.

In this guide, we are going to assume that you will use GitHub in addition to the official Git command-line program, because GitHub has many useful tools to offer and makes some things nicely easy, especially when comparing a topic branch to the master branch and seeing diffs in a somewhat more advanced interface, and when making pull requests. Technically, this means that you will have two repositories -- your local repository on your computer, and a fork of the wesnoth/wesnoth repository, on GitHub.

The first step of becoming a Wesnoth developer is to set up these two repositories. Follow the instructions in GitHub's "Fork A Repo" help article, which may be summarized as follows:

1. Fork the wesnoth/wesnoth repository, using the GitHub website.

2. Clone the fork onto your computer, with the command "git clone".

3. Configure remotes (remote repositories that are essentially bookmarked).

   GitHub's Fork A Repo article describes how to set the wesnoth/wesnoth repository as a remote with the name "upstream". Because you cloned from your fork, the fork is automatically set as a remote with the name "origin". You can see your remotes with the command "git remote -v".

   You might find this naming scheme confusing, given that the name "origin" might also aptly describe the wesnoth/wesnoth repository. So it might be good to rename that remote to "fork" (with the command "git remote rename origin fork"). But if you do that, note that much of the GitHub help pages will assume the name "origin".

Additionally, we ask that you configure Git to use your full real name or a consistent alias, per the instructions in GitHub's "Setting your username in Git" help article, so that we can associate a concrete person with every commit merged into Wesnoth. Your GitHub profile should also display the same name.

The basic workflow

Here's the picture we now have, with 3 repositories:

   upstream


                    origin (fork)


    local

There are four basic steps in the development cycle, which are as follows:

1. ensuring that your local repository is up-to-date with the upstream repository;
2. creating a topic branch for your changes;
3. committing your changes to the topic branch and pushing them to your fork; and
4. filing a GitHub pull request, having that request granted, then finally cleaning up.

Keeping your local repository up-to-date

In this step, you will make sure your local master branch is up-to-date with the most recent development changes before beginning work.

First make sure you are on your master branch:

 git checkout master
 git status

Now, pull the upstream master branch to your local repository:

   upstream
      |
      |
      |             origin (fork)
      |          
      v      
    local

 git pull upstream master

At this point your local master is up to date. While you are at it, you might as well update your fork as well:

   upstream
        
       
               .--> origin (fork)
              /  
             /    
    local --'

 git push origin master

At this point, the master branch should be the same in all three repositories.

Creating a topic branch

In this guide, we will never make any changes directly on the master branch, and will only make code changes on a topic branch, so regardless of which repository you look at, the master branch should be the same, and Git will be comparing your work against the upstream master branch.

Now you are ready to create a new topic branch, which will hold your work for this patch. We are on the master branch, as running "git status" will confirm. Create your new topic branch -- named, in this example, "new-feature" -- by running the following command:

 git checkout -b new-feature

As shown in the gitref.org guide, you could also have done this with the following commands:

 git branch new-feature
 git checkout new-feature

Given that you were on the master branch when you created the new-feature topic branch, the new-feature branch will branch off from the tip of your current master branch, which is best, to avoid conflicts when the new-feature branch is eventually merged upstream.

Committing your changes and pushing to your fork

Now, you will make your changes to Wesnoth, test them, and commit them to your local repository, with the commands "git add" and "git commit", as shown in the gitref.org guide.

Committing guidelines

1. Write good commit messages

   A commit message should fully explain what changes were made in the commit, and why those changes were made. A good commit message is structured like an email message.

   The first line, which summarizes the commit, has special importance -- to quote the DeveloperGuide page, This is the first (and often the only) line someone using browsing tools will see. It should ideally be at most 80 characters (TODO: inconsistent with current policy), and is like the subject line of an email message. Commit message summary lines should be written in the imperative present tense, e.g., "Add field to some_class, with accessor methods", "Fix other_class's constructor", "Delete unnecessary #include", etc.

   After the summary line, there must be a blank line, after which comes the commit message body, which is analogous to the body of an email message.

   Optimally, a commit's message would be so clearly and comprehensively explanatory that it would answer any questions that anyone might have about the commit, now, the next day, or years into the future -- which is especially important because the author of the commit likely won't always be around to answer those questions.

   See also the DeveloperGuide page's "Commit messages" section and the official Git book's "Commit Guidelines" section.

2. Make appropriately sized commits

   Each commit should correspond to a single logical step in your programming task. The commits constitute the project's development history; people will look at your commits to try to figure out how we got to where we are today, and may at some point need to undo the changes made in some of your commits to fix a future issue.

   If you find it difficult to write a succinct commit message that fully explain what changes were made in the commit, and why those changes were made, then the commit is insufficiently fine-grained -- too large and too complicated. Another guideline is, you should at least think about committing almost every time you compile (depending on your habits).

   On the other hand, you shouldn't make a commit for every line of code typed. If it would never make sense to revert the repository's code to its current state from a future state, you probably shouldn't commit. For example, if you define a field of an object, but it has no accessors and no way to be used in code, it would never make sense to revert to that time in the repository's history, so those changes should be saved together in a single commit. If you have a commit that merely adjusts whitespace in a file, that should be squashed in with another commit, according to current (2015-07-25) policy.

   It is better to commit more often than less often, because, before you submit your changes to upstream, you will have the opportunity to edit your commit history, and it is much easier to squash commits together than to split up an overly-large commit.

   See also the DeveloperGuide page's "Commits" section.

3. Keep the change-logs, release notes, and credits up-to-date

   After committing your changes, add an entry in the changelog document, and commit it with the message "Update changelog".

   We also have a players_changelog document, which should contain only changes that most players are likely to notice, and be less technical than the main, technical changelog.

   There is also a RELEASE_NOTES document, which basically is copied into the next Wesnoth release's release announcement post in the forums, so if there is a new feature or bug fix that should be advertised, add that to RELEASE_NOTES.

   If this is your first commit, the development team will ask you to add an appropriate note about yourself in the credits document, data/about.cfg.

   See also the DeveloperGuide page's "Changelogs and release notes" section.

Finally, when your branch is ready, push it to your fork.

   upstream
       
       
               .--> origin (fork)
              /
             /
    local --'

 git push origin new-feature

Now, if you navigate to your fork's page on GitHub, you should see a message such as "recently pushed branch new-feature (3 minutes ago)". Click "Compare" to compare it against the master branch. You should now see a list of the commits you made on the new-feature branch, and color-highlighted diffs of the files you changed. You can click through the commits in order to see each of the changes you made and understand how your project progressed -- this is exactly what the development team will do when they review your pull request.

In the GitHub interface, you can always see a list of branches by clicking on "branches" in the line near the middle of the page that looks like the following:

10,000+ commits     27 branches     248 releases     85 contributors

You can also look at a specific branch using the "Branch" menu below that line, and you can compare branches using the green "Compare" button to the left of the "Branch" menu.

Cleaning up your branch

Before filing your pull request, if, looking at the diffs and the history (the list of commits) for your topic branch, you saw anything confusing or not quite right, now is your opportunity to change it.

From the gitref.org guide, you should know how to add more commits, and you should know that you can use "reset" to go back in time and redo things.

"reset" is especially good if your last commit was too big and you want to break it up into a series of smaller commits -- the command "git reset "@~"" will leave your working directory the same, but jump back in time one commit and unstage all of those changes, so you can add a smaller subset of the files, commit those, then add the others, etc.

More generally, you can use "git reflog" to see how to reset back over any number of commits or other Git operations.

You can also use "git commit --amend" to edit the commit message of the most recent commit.

There are several more advanced cleanup features available with Git, but we'll defer any further discussion of them.

Every time you make changes, you should to push them to your fork to keep it up-to-date. If you rewrote your commit history, then the push would be rejected, because the commit histories won't match. Therefore, to push a rewritten history, you must force push, which will -- this is a potentially dangerous operation! -- discard the old commit history and replace it with the new history:

   upstream
         
        
               .--> origin (fork)
              /
             /
    local --'

 git push --force origin new-feature

You can and should do this throughout your development if you want to look at the diffs of your changes as you work -- after all, the Wesnoth code-base is vast and it is easy to forget exactly where you are in your project. You can and should keep cleaning up in this way until you are satisfied with the result. Doing this, your GitHub fork ends up being rather like a personal Web-based IDE plug-in or extension to help you with development.

Filing a pull request

When you finish working on your feature or bug fix, select your topic branch on your fork (with the "Branch" menu), and click the green button to file a pull request. By default, the target of the pull request -- the repository and branch into which it is requesting that your changes be pulled -- will be "wesnoth:master", the master branch of the main Wesnoth repository.

The message that you enter for the pull request should become the commit message of the merge operation performed in the upstream repository if the pull request is granted. (Note: if you are granting someone's PR through the GitHub Web interface, please ensure that this happens.)

If you fixed a bug, you should say, e.g., "Fixed bug #5678" in the pull-request message, which should automatically link to our Gna! bug tracker.

If you want more information, you could read GitHub's "Creating a pull request" help article.

   upstream <-.
               \
                \
                 '- origin (fork)
 
 
    local     

You should then see your pull request in the main Wesnoth repository's "pull requests" page.

At the time of writing this, we also have configured Travis CI, a continuous integration system, to automatically try to compile Wesnoth with your changes, to make sure that it still works before your pull request is granted. However, it is not mandatory for the Travis build to complete successfully for the pull request to be granted. Additionally, you may still make changes to your pull request by pushing more changes to your topic branch in your fork, after which Travis should try to compile the updated topic branch. You can even still force push to erase the content of the topic branch (on which the pull request is based) and replace it with new content -- however, you shouldn't file a pull request for a topic branch until you are ready for it to be merged.

Now, wait for a member of the development team to review your pull request. You can often find someone on the development IRC channel, #wesnoth-dev. Additionally, developers may comment directly on your pull request with questions, so you should check it periodically.

Cleaning up at the end

Congratulations, a developer granted your pull request! Now it's time to clean up and update your local repository.

On the GitHub page for your pull request, you should now see a button with the option "delete this branch". Since the content of new-feature is now merged into master, this branch no longer needs to exist as you won't work on it anymore, and future work will go onto a different branch. So you should delete it, which will delete the branch on *origin*, your github-associated repo.

Since master has now changed and we want master to be synced up, you should now do step 1 again:

   upstream
      |
      |
      |             origin (fork)
      |          
      v      
    local

 git checkout master
 git pull upstream master

At this point git will understand that new-feature has been merged into master, so when you ask to delete this branch from local as well, it will do it:

 git branch -d new-feature

If you do these steps out of order, i.e. try to delete the branch before before syncing master, it will make a warning "Are you sure? You have unmerged changes on this branch..." etc.

Finally, sync master on the origin as well:

   upstream
         
         
                    origin (fork)
              /-->  
           --/    
    local     

 git push origin master

That's the end of the development cycle, and now you can begin on your next patch. Note that throughout the cycle, information only flowed counter-clockwise:

   upstream
      |     <--\
      |         \--
      |             origin (fork)
      |       /-->  
      v    --/    
    local

If for some reason you find yourself doing something where information is flowing the other way, that is a good sign that something is going wrong, at least as far as this guide is concerned! (The exception is the steps in which we set up our repos -- don't overthink this.)

Advanced / Alternative workflows

You can use pull requests as described above whether you have commit access or not -- if you have commit access, then you can click to accept your own pull requests.

However, if you have commit access, you may prefer not to use github / pull requests at all, and instead to push directly to wesnoth:master. You can still use topic branches to hold your work, and merge them to your local master using git merge, then use 'git push upstream master' to push them to the main wesnoth repo.

If you are lazy, you might even skip making a topic branch and just commit to local master, although IMO this can make things more confusing if you get a merge conflict later.

Either way, afaik the vast majority of wesnoth developers prefer to push directly to wesnoth:master.

An extreme option appropriate for small changes is to skip the local repo as well, and simply commit changes directly from the github web interface. This might be appropriate for example if you forgot to commit a changelog entry in an earlier patch, or are simply changing some numbers used as parameters somewhere. Obviously if you change source code this way, you should make certain that everything still compiles immediately after! (Note: Don't actually do that.)

The ultimate cleanup power tool: git rebase

Using things in the gitref guide, you can see how to use reset to undo mistakes you made by backing up and trying again, and for small commits that is probably fine. However, if you have a large and complicated series of changes, the better way to do this is using git rebase. Using git rebase in interactive mode, you can rewrite history by

• reordering your commits
• discarding unwanted commits
• squashing commits together
• editing the content changes represented by an individual commit
• editing the commit message

For our purposes, we assume you are working on a topic branch, and then you will only need to use the form

 git rebase -i master

You can see some documentation for using this command here: https://help.github.com/articles/interactive-rebase

Using 'git rebase', you can make your commit history very *clean* -- instead of the history saying "I tried a bunch of things, undid some of them, tried some other things, and found something I liked", your commit history can basically be "I took the simplest and most logical route to the solution that was finally the best". When you review your commits, instead of feeling like a nasty blood-and-elbow-grease engineering project, it should feel like folding a perfect origami crane, to the extent possible ;)

But to reiterate, this has a purpose.

1. Other devs need to be able to understand the history.
2. It should be possible to jump back in time by checking out one of your commits and find ourselves in a sensible state when we do so.
3. If something breaks or some feature become incompatible, it should be possible to roll back only a piece of the implementation of your feature without removing the whole thing.

If you want to use rebase to cleanup your history, it is important to do it *before* it is pulled into wesnoth, as once that happens it is basically no longer possible. If we rewrite history in the main wesnoth repo, then afterwards whenever any dev tries to 'git pull upstream master', their git will give them strange merge errors, stemming from the incompatible history, and then they will become nervous, jump on irc and exclaim some variation of "omg wtf bbq". So except in extreme circumstances, we will never rebase the main wesnoth master, and for the same reason, we will never use "git push --force upstream master". That's why if you want to do these things, it is important to do them on your *fork* before it makes it onto master.

If you feel like it, you might read this, which is a historical email between early users / developers of git, in which Linus Torvalds explains about shared history of repos and when it is appropriate to use git rebase.

http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html

When you become comfortable with git rebase -i, it will greatly improve your workflow, as you will know that you can easily review and revise commit messages later, and easily reorder and squash commits. Typically I will now commit every single time I compile, and even if it is a commit which e.g. fixes a compile time error, or for some other reason I know it will be squashed in somewhere else, I give it the commit message "fixup", as a note to myself to mark it thusly in the first git rebase pass. (Actually, while writing this I have just learned about the --autosquash feature of git rebase which takes this idea further, good stuff there.)

Note that unlike Linus and github, in this guide we aren't thinking about your github fork repo as public -- we prefer to think of it as your personal private IDE/tool as I described earlier. If other wesnoth devs are cloning it or pulling your topic branches, we assume you will work it out with them. With that caveat the philosophies expressed on github and by Linus should generally apply to the wesnoth project as well.

Solving merge conflicts with git rebase

Github version: https://github.com/edx/edx-platform/wiki/How-to-Rebase-a-Pull-Request

Suppose that you modify a file in the same place as someone else, and their change gets merged before yours. If git can't figure out how to merge, then someone will have to fix it. This could be done by whoever is merging the new content into master, but if you don't have commit access then that person is not you. One way that *you* could fix it is to resync your master to get the new changes locally, and then rebase your topic branch so that your changes are applied *after* the most recent changes.

 git rebase master

To understand what is happening, we should understand a bit more about how git works. Intuitively, git is all about "snapshots" of the project content. Every commit represents a snapshot, and you can look at this snapshot by checking out the commit. However git does not store each snapshot separately -- instead git defines the snapshots in terms of one another, and stores only the diffs. When you rebase your topic branch onto master, this will also update the "point of departure" where your branch was created, so that in the history, it will depart from the current head of master with the most recent changes. (Obviously, if you don't sync master until you are done working, then this subtlety to 'git rebase' is irrelevant.)

If merging your branch with the current master would create a conflict, then when git rebase gets to the commit that creates the conflict, it will get confused when it tries to apply that diff, stop, tell you about the problem, and ask you to resolve it -- you can open up the offending file and go to the point where the conflict is happening, and git will leave a note for you of the content it is having trouble with. The note will look similar to what happens in the gitref guide here, under "merge conflicts".

http://gitref.org/branching/#merge

You just have to remove the note, make the file look like it should to make everything compatible, and type

 git add .
 git rebase --continue

Then the rebasing process will continue, and at the end your branch should be compatible with master and ready to be merged in.

You can read more about this aspect of git rebase here if you like, although most likely you won't actually need more than we just talked about for wesnoth development.

http://git-scm.com/book/en/Git-Branching-Rebasing

That's all for this guide, have fun hacking on wesnoth!

Addendum

The following section contains commands with the -f / --force options. Those are usually used to rewrite history, which is fine for your own fork but disallowed for upstream. Do not use them on the main repository.

If a command goes awry, you can go to "git reflog", find the point just before you typed it, and reset --hard to that commit, and it should be like the bad thing never happened

revert every file in the repo to look like it did at the HEAD commit (last successful sync)

       git reset --hard HEAD

get a clean slate by syncing with upstream

       git fetch upstream
       git checkout -B master upstream/master
       git push --force origin master

if this errors, try again after

       git reset --hard HEAD

undo n commits, split/squash and apply again

       git reset HEAD~n
       git add -p
       git commit
       git push -f

You can also push from one ‘ref’ to a different one:

       $ git push <remote> <from>:<to>

completely overwrite branch b with a (from a to b)

       git push -f origin a:b

before any commit:

• if the diff shows an invisible change in the first line - stop! Fix the file encoding back to UTF8 without BOM.

• don't forget the changelog (and, for visible changes, the players_changelog)

• try to keep the summary at 50 characters or less, maximum 72

See Also

• Project#Developers