View source for Git for Wesnoth Crash Course

== Foreword ==

'''''(An essay by, mainly, iceiceice.)'''''

This page is intended for new <cite>Battle for Wesnoth</cite> developers and contributors who have never used '''[[WesnothRepository|<cite>Git</cite>]]''' before. I'm hoping that it will be written <em>mainly</em> by newbies like myself who have just figured out enough about <cite>Git</cite> to use it successfully, and therefore not include a bunch of information superfluous for that purpose. At the time that I am writing this, I have managed to write about eight GitHub pull requests and get them merged into <cite>Wesnoth</cite>, and I hope you will be able to do the same after reading this.

== What is <cite>Git</cite>? ==

The most effective '''"absolute beginner" introduction to <cite>Git</cite>''' that I found when I started is here: '''[http://gitref.org <cite>gitref.org</cite>]'''. - ''(Note: this gitref.org link seems to have died.  Another resource, that is only slightly less introductory-level, is '''[https://git-scm.com/book/en/v2 <cite>Pro-Git Book</cite>]'''.  Don't worry about reading the whole thing though.)''

I suggest that you <strong>read this through</strong>, beginning to end, navigating through the pages by using the red arrows at the bottom of each page. You can skip the part about ''"stashing"'', but you should carefully read and understand the parts about:

* '''''cloning''''' a ''repository'';
* '''''checking out''''' a ''branch'', creating a '''new branch''', and '''merging branches''';
* '''''adding''''' and '''''committing changes''''';
* viewing '''''diffs''''' to make sure you understand exactly what you are committing;
* '''''resetting''''' your repository state in case you make a mistake and need to undo some steps, and using "'''<kbd>git log</kbd>'''" and "'''<kbd>git reflog</kbd>'''" to assist with this; and
* moving content between local and remote repositories, with "'''<kbd>push</kbd>'''" and "'''<kbd>pull</kbd>'''".

== Setting up our workflow for <cite>Wesnoth</cite> in <cite>Git</cite> ==

It is common when reading about <cite>Git</cite> to see the terms '''''"upstream"''''' and '''''"downstream"'''''. This is often confusing at the beginning -- after all, information is usually flowing in both directions; sometimes, you are getting code from the <cite>Wesnoth</cite> repository, and sometimes you are sending code to it. Which way is up and which is down?

Analogously to a river, ''downstream'' is the direction toward which most information flows. Any user that simply wants to build <cite>Wesnoth</cite> from source, and not make any changes to it, will generally do so by cloning or pulling from <https://github.com/wesnoth/wesnoth>. So [https://github.com/wesnoth/wesnoth <code>wesnoth/wesnoth</code>] (the main <cite>Wesnoth</cite> repository) is the <dfn>upstream</dfn> repository, and the users are <dfn>downstream</dfn>.

If you are a developer, you will '''''fork''''' the upstream repository and edit the source code, and -- hopefully, with work, eventually -- get your edits pulled into the upstream repository. That is the '''<dfn>development cycle</dfn>''', and contributing changes back to upstream is what makes you a developer. 

In this guide, we are going to assume that you will use GitHub in addition to the official <cite>Git</cite> command-line program, because GitHub has many useful tools to offer and makes some things nicely easy, especially when comparing a topic branch to the master branch and seeing diffs in a somewhat more advanced interface, and when making ''pull requests''. Technically, this means that you will have '''<em>two</em> repositories''' -- your <dfn>local repository</dfn> on your computer, and a ''fork'' of the <code>wesnoth/wesnoth</code> repository, on GitHub.

The first step of becoming a <cite>Wesnoth</cite> developer is to set up these two repositories. Follow the instructions in [https://help.github.com/articles/fork-a-repo/ GitHub's "<cite>Fork A Repo</cite>" help article], which may be summarized as follows:

<ol>
<li><p>'''''Fork''''' the <code>wesnoth/wesnoth</code> repository, using the GitHub website.</p></li>
<li><p>'''''Clone''''' the fork onto your computer, with the command "<kbd>git clone</kbd>".</p></li>
<li><p>Configure '''<dfn>remotes</dfn>''' (''remote'' repositories that are essentially bookmarked).</p>
<p>GitHub's <cite>Fork A Repo</cite> article describes how to set the <code>wesnoth/wesnoth</code> repository as a remote with the name "<code>upstream</code>". Because you cloned from your fork, the fork is automatically set as a remote with the name "<code>origin</code>". You can see your remotes with the command "<kbd>git remote -v</kbd>".</p>
<p>You might find this naming scheme confusing, given that the name "<code>origin</code>" might also aptly describe the <code>wesnoth/wesnoth</code> repository. So it might be good to rename that remote to "<code>fork</code>" (with the command "<kbd>git remote rename origin fork</kbd>"). But if you do that, note that much of the GitHub help pages will assume the name "<code>origin</code>".</p></li>
</ol>

Additionally, we ask that you configure <cite>Git</cite> to use your full real name or a consistent alias, per the instructions in [https://help.github.com/articles/setting-your-username-in-git/ GitHub's "<cite>Setting your username in Git</cite>" help article], so that we can associate a concrete person with every commit merged into <cite>Wesnoth</cite>. Your GitHub profile should also display the same name.

== The basic workflow ==

Here's the picture we now have, with 3 repositories:



    upstream
 
 
                     origin (fork)
 
 
     local



There are four basic steps in the development cycle, which are as follows:

# ensuring that your local repository is '''up-to-date''' with the upstream repository;
# creating a '''topic branch''' for your changes;
# '''committing''' your changes to the topic branch and '''pushing''' them to your fork; and
# filing a GitHub '''pull request''', having that request granted, then finally cleaning up.

=== Keeping your local repository up-to-date ===

In this step, you will make sure your local <code>master</code> branch is up-to-date with the most recent development changes before beginning work. 

First make sure you are on your <code>master</code> branch:

  git checkout master
  git status

Now, pull the upstream <code>master</code> branch to your local repository:


    upstream
       |
       |
       |             origin (fork)
       |          
       v      
     local



  git pull upstream master

At this point your local master is up to date. While you are at it, you might as well update your fork as well:



    upstream
         
        
                .--> origin (fork)
               /  
              /    
     local --'


  git push origin master

At this point, the <code>master</code> branch should be the same in all three repositories.

=== Creating a topic branch ===

In this guide, we will <strong><em>never</em> make any changes directly on the <code>master</code> branch</strong>, and will <strong><em>only</em> make code changes on a topic branch</strong>, so regardless of which repository you look at, the <code>master</code> branch should be the same, and <cite>Git</cite> will be comparing your work against the upstream <code>master</code> branch.

Now you are ready to create a new '''[https://git-scm.com/book/en/Git-Branching-Branching-Workflows#Topic-Branches <dfn>topic branch</dfn>]''', which will hold your work for this patch. We are on the <code>master</code> branch, as running "<kbd>git status</kbd>" will confirm. Create your new topic branch -- named, in this example, "<code>new-feature</code>" -- by running the following command:

  git checkout -b new-feature

As shown in the <cite>gitref.org</cite> guide, you could also have done this with the following commands:

  git branch new-feature
  git checkout new-feature

Given that you were on the <code>master</code> branch when you created the <code>new-feature</code> topic branch, the <code>new-feature</code> branch will branch off from the ''tip'' of your current <code>master</code> branch, which is best, to avoid conflicts when the <code>new-feature</code> branch is eventually merged upstream.

=== Committing your changes and pushing to your fork ===

Now, you will make your changes to <cite>Wesnoth</cite>, test them, and commit them to your local repository, with the commands "<kbd>git add</kbd>" and "<kbd>git commit</kbd>", as shown in the <cite>gitref.org</cite> guide.

==== Committing guidelines ====

<ol>
<li>
<h5 style="font-size: larger"><strong>Write good commit messages</strong></h5>

<p><strong>A commit message should fully explain what changes were made in the commit, and <em>why</em> those changes were made</strong>. A good commit message is structured like an email message.</p>

<p>The first line, which <strong>summarizes</strong> the commit, has special importance -- to quote the <cite>[[DeveloperGuide]]</cite> page, <q cite="[[DeveloperGuide#Commit_messages]]">This is the first (and often the only) line someone using browsing tools will see</q>. It should ideally be at most 80 characters (TODO: inconsistent with current policy), and is like the subject line of an email message. Commit message summary lines should be written in the imperative present tense, e.g., "Add field to <var>some_class</var>, with accessor methods", "Fix <var>other_class</var>'s constructor", "Delete unnecessary #include", etc.</p>

<p>After the summary line, there must be a blank line, after which comes the '''commit message body''', which is analogous to the body of an email message.</p>

<p>'''Optimally''', a commit's message would be so clearly and comprehensively explanatory that it would answer any questions that anyone might have about the commit, now, the next day, or years into the future -- which is especially important because the author of the commit likely won't always be around to answer those questions.</p>

<p>See also [[DeveloperGuide#Commit_messages|the <cite>DeveloperGuide</cite> page's "Commit messages" section]] and [https://git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project#Commit-Guidelines the official Git book's "Commit Guidelines" section].</p>
</li>
<li>
<h5 style="font-size: larger"><strong>Make appropriately sized commits</strong></h5>

<p>Each commit should correspond to a <strong>single logical step</strong> in your programming task. The commits constitute the project's development history; people will look at your commits to try to figure out how we got to where we are today, and may at some point need to undo the changes made in some of your commits to fix a future issue.</p>

<p><strong>If you find it difficult to write a succinct commit message that fully explain what changes were made in the commit, and why those changes were made, then the commit is insufficiently fine-grained -- too large and too complicated.</strong> Another guideline is, you should at least think about committing almost every time you compile (depending on your habits).</p>

<p>On the other hand, you shouldn't make a commit for every line of code typed. <strong>If it would never make sense to revert the repository's code to its current state from a future state, you probably shouldn't commit.</strong> For example, if you define a field of an object, but it has no accessors and no way to be used in code, it would never make sense to revert to that time in the repository's history, so those changes should be saved together in a single commit. If you have a commit that merely adjusts whitespace in a file, that should be squashed in with another commit, according to current (2015-07-25) policy.</p>

<p><strong>It is better to commit more often than less often</strong>, because, before you submit your changes to upstream, you will have the opportunity to edit your commit history, and <strong>it is much easier to ''squash'' commits together than to split up an overly-large commit</strong>.</p>

<p>See also [[DeveloperGuide#Commits|the <cite>DeveloperGuide</cite> page's "Commits" section]].</p>
</li>
<li>
<h5 style="font-size: larger"><strong>Keep the change-logs, release notes, and credits up-to-date</strong></h5>

<p>After committing your changes, add an entry in the '''<code>changelog</code>''' document, and commit it with the message "Update changelog".</p>

<p>We also have a '''<code>players_changelog</code>''' document, which should contain only changes that most players are likely to notice, and be less technical than the main, technical <code>changelog</code>.</p>

<p>There is also a '''<code>RELEASE_NOTES</code>''' document, which basically is copied into the next <cite>Wesnoth</cite> release's release announcement post in the forums, so if there is a new feature or bug fix that should be advertised, add that to <code>RELEASE_NOTES</code>.</p>

<p>If this is your first commit, the development team will ask you to add an appropriate note about yourself in the '''credits document''', '''<code>data/about.cfg</code>'''.</p>

<p>See also [[DeveloperGuide#Changelogs_and_release_notes|the <cite>DeveloperGuide</cite> page's "Changelogs and release notes" section]].</p>
</li>
</ol>

Finally, when your branch is ready, push it to your fork.



    upstream
        
        
                .--> origin (fork)
               /
              /
     local --'


  git push origin new-feature


Now, if you navigate to your fork's page on GitHub, you should see a message such as "recently pushed branch new-feature (3 minutes ago)". Click "Compare" to compare it against the <code>master</code> branch. You should now see a list of the commits you made on the <code>new-feature</code> branch, and color-highlighted diffs of the files you changed. You can click through the commits in order to see each of the changes you made and understand how your project progressed -- this is exactly what the development team will do when they review your pull request.

In the GitHub interface, you can always see a list of branches by clicking on "branches" in the line near the middle of the page that looks like the following:

 10,000+ commits     27 branches     248 releases     85 contributors

You can also look at a specific branch using the "Branch" menu below that line, and you can compare branches using the green "Compare" button to the left of the "Branch" menu.

=== Cleaning up your branch ===

Before filing your pull request, if, looking at the diffs and the history (the list of commits) for your topic branch, you saw anything confusing or not quite right, now is your opportunity to change it.

From the <cite>gitref.org</cite> guide, you should know how to add more commits, and you should know that you can use "<kbd>reset</kbd>" to go back in time and redo things.

"'''<kbd>reset</kbd>'''" is especially good if your last commit was too big and you want to break it up into a series of smaller commits -- the command "<kbd>git reset "@~"</kbd>" will leave your working directory the same, but jump back in time one commit and unstage all of those changes, so you can add a smaller subset of the files, commit those, then add the others, etc.

More generally, you can use "'''<kbd>git reflog</kbd>'''" to see how to reset back over any number of commits or other <cite>Git</cite> operations. 

You can also use "'''<kbd>git commit --amend</kbd>'''" to edit the commit message of the most recent commit.

There are several more advanced cleanup features available with <cite>Git</cite>, but we'll defer any further discussion of them.

Every time you make changes, you should to push them to your fork to keep it up-to-date. If you rewrote your commit history, then the push would be rejected, because the commit histories won't match. Therefore, to push a rewritten history, you must '''''<strong>force push</strong>''''', which will -- <strong>this is a potentially dangerous operation!</strong> -- discard the old commit history and replace it with the new history:



    upstream
          
         
                .--> origin (fork)
               /
              /
     local --'


  git push --force origin new-feature

You can and should do this throughout your development if you want to look at the diffs of your changes as you work -- after all, the <cite>Wesnoth</cite> code-base is vast and it is easy to forget exactly where you are in your project. You can and should keep cleaning up in this way until you are satisfied with the result. Doing this, your GitHub fork ends up being rather like a personal Web-based IDE plug-in or extension to help you with development.

=== Filing a pull request ===

When you finish working on your feature or bug fix, select your topic branch on your fork (with the "Branch" menu), and click the green button to file a pull request. By default, the target of the pull request -- the repository and branch into which it is requesting that your changes be pulled -- will be "<samp>wesnoth:master</samp>", the <code>master</code> branch of the main <cite>Wesnoth</cite> repository.

The message that you enter for the pull request should become the commit message of the merge operation performed in the upstream repository if the pull request is granted. (Note: if you are granting someone's PR through the GitHub Web interface, please ensure that this happens.)

If you fixed a bug, you should say, e.g., "Fixed bug #5678" in the pull-request message, which should automatically link to [https://gna.org/bugs/index.php?group=wesnoth&set=custom&report_id=101&status_id=1&chunksz=150&boxoptionwanted=1 our Gna! bug tracker].

If you want more information, you could read [https://help.github.com/articles/creating-a-pull-request/ GitHub's "<cite>Creating a pull request</cite>" help article].



    upstream <-.
                \
                 \
                  '- origin (fork)
  
  
     local     



You should then see your pull request in [https://github.com/wesnoth/wesnoth/pulls the main <cite>Wesnoth</cite> repository's "pull requests" page].

At the time of writing this, we also have configured [https://travis-ci.org Travis CI], a [https://en.wikipedia.org/wiki/Continuous_integration continuous integration] system, to automatically try to compile <cite>Wesnoth</cite> with your changes, to make sure that it still works before your pull request is granted. However, it is not mandatory for the Travis build to complete successfully for the pull request to be granted. Additionally, you may still make changes to your pull request by pushing more changes to your topic branch in your fork, after which Travis <em>should</em> try to compile the updated topic branch. You can even still force push to erase the content of the topic branch (on which the pull request is based) and replace it with new content -- however, you shouldn't file a pull request for a topic branch until you are ready for it to be merged.

Now, wait for a member of the development team to review your pull request. You can often find someone on the development [[Support#IRC|IRC]] channel, <code>#wesnoth-dev</code>. Additionally, developers may comment directly on your pull request with questions, so you should check it periodically.

=== Cleaning up at the end ===

Congratulations, a developer granted your pull request! Now it's time to clean up and update your local repository.

On the GitHub page for your pull request, you should now see a button with the option "delete this branch". Since the content of new-feature is now merged into master, this branch no longer needs to exist as you won't work on it anymore, and future work will go onto a different branch. So you should delete it, which will delete the branch on *origin*, your github-associated repo.

Since master has now changed and we want master to be synced up, you should now do step 1 again:


    upstream
       |
       |
       |             origin (fork)
       |          
       v      
     local


  git checkout master
  git pull upstream master

At this point git will understand that new-feature has been merged into master, so when you ask to delete this branch from local as well, it will do it:

  git branch -d new-feature

If you do these steps out of order, i.e. try to delete the branch before before syncing master, it will make a warning "Are you sure? You have unmerged changes on this branch..." etc.

Finally, sync master on the origin as well:



    upstream
          
          
                     origin (fork)
               /-->  
            --/    
     local     


  git push origin master

That's the end of the development cycle, and now you can begin on your next patch. Note that throughout the cycle, information only flowed counter-clockwise:


    upstream
       |     <--\
       |         \--
       |             origin (fork)
       |       /-->  
       v    --/    
     local


If for some reason you find yourself doing something where information is flowing the other way, that is a good sign that something is going wrong, at least as far as this guide is concerned! (The exception is the steps in which we set up our repos -- don't overthink this.)


== Advanced / Alternative workflows ==

You can use pull requests as described above whether you have commit access or not -- if you have commit access, then you can click to accept your own pull requests.

However, if you have commit access, you may prefer not to use github / pull requests at all, and instead to push directly to wesnoth:master. You can still use topic branches to hold your work, and merge them to your local master using git merge, then use 'git push upstream master' to push them to the main wesnoth repo. 

If you are lazy, you might even skip making a topic branch and just commit to local master, although IMO this can make things more confusing if you get a merge conflict later.

Either way, afaik the vast majority of wesnoth developers prefer to push directly to wesnoth:master.

An extreme option appropriate for small changes is to skip the local repo as well, and simply commit changes directly from the github web interface. This might be appropriate for example if you forgot to commit a changelog entry in an earlier patch, or are simply changing some numbers used as parameters somewhere. Obviously if you change source code this way, you should make certain that everything still compiles immediately after! (Note: Don't actually do that.)

== The ultimate cleanup power tool: git rebase ==

Using things in the gitref guide, you can see how to use reset to undo mistakes you made by backing up and trying again, and for small commits that is probably fine. However, if you have a large and complicated series of changes, the better way to do this is using git rebase. Using git rebase in interactive mode, you can rewrite history by

* reordering your commits
* discarding unwanted commits
* squashing commits together
* editing the content changes represented by an individual commit
* editing the commit message

For our purposes, we assume you are working on a topic branch, and then you will only need to use the form

  git rebase -i master

You can see some documentation for using this command here: https://help.github.com/articles/interactive-rebase

Using 'git rebase', you can make your commit history very *clean* -- instead of the history saying "I tried a bunch of things, undid some of them, tried some other things, and found something I liked", your commit history can basically be "I took the simplest and most logical route to the solution that was finally the best". When you review your commits, instead of feeling like a nasty blood-and-elbow-grease engineering project, it should feel like folding a perfect origami crane, to the extent possible ;)

But to reiterate, this has a purpose. 
# Other devs need to be able to understand the history.
# It should be possible to jump back in time by checking out one of your commits and find ourselves in a sensible state when we do so.
# If something breaks or some feature become incompatible, it should be possible to roll back only a piece of the implementation of your feature without removing the whole thing.

If you want to use rebase to cleanup your history, it is important to do it *before* it is pulled into wesnoth, as once that happens it is basically no longer possible. If we rewrite history in the main wesnoth repo, then afterwards whenever any dev tries to 'git pull upstream master', their git will give them strange merge errors, stemming from the incompatible history, and then they will become nervous, jump on irc and exclaim some variation of "omg wtf bbq". So except in extreme circumstances, we will never rebase the main wesnoth master, and for the same reason, we will never use "git push --force upstream master". That's why if you want to do these things, it is important to do them on your *fork* before it makes it onto master.

If you feel like it, you might read this, which is a historical email between early users / developers of git, in which Linus Torvalds explains about shared history of repos and when it is appropriate to use git rebase. 

http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html

When you become comfortable with git rebase -i, it will greatly improve your workflow, as you will know that you can easily review and revise commit messages later, and easily reorder and squash commits. Typically I will now commit every single time I compile, and even if it is a commit which e.g. fixes a compile time error, or for some other reason I know it will be squashed in somewhere else, I give it the commit message "fixup", as a note to myself to mark it thusly in the first git rebase pass. (Actually, while writing this I have just learned about the --autosquash feature of git rebase which takes this idea further, good stuff there.)

Note that unlike Linus and github, in this guide we aren't thinking about your github fork repo as public -- we prefer to think of it as your personal private IDE/tool as I described earlier. If other wesnoth devs are cloning it or pulling your topic branches, we assume you will work it out with them. With that caveat the philosophies expressed on github and by Linus should generally apply to the wesnoth project as well.

=== Solving merge conflicts with git rebase ===

Github version: https://github.com/edx/edx-platform/wiki/How-to-Rebase-a-Pull-Request

Suppose that you modify a file in the same place as someone else, and their change gets merged before yours. If git can't figure out how to merge, then someone will have to fix it. This could be done by whoever is merging the new content into master, but if you don't have commit access then that person is not you. One way that *you* could fix it is to resync your master to get the new changes locally, and then rebase your topic branch so that your changes are applied *after* the most recent changes.

  git rebase master

To understand what is happening, we should understand a bit more about how git works. Intuitively, git is all about "snapshots" of the project content. Every commit represents a snapshot, and you can look at this snapshot by checking out the commit. However git does not store each snapshot separately -- instead git defines the snapshots in terms of one another, and stores only the diffs. When you rebase your topic branch onto master, this will also update the "point of departure" where your branch was created, so that in the history, it will depart from the current head of master with the most recent changes. (Obviously, if you don't sync master until you are done working, then this subtlety to 'git rebase' is irrelevant.) 

If merging your branch with the current master would create a conflict, then when git rebase gets to the commit that creates the conflict, it will get confused when it tries to apply that diff, stop, tell you about the problem, and ask you to resolve it -- you can open up the offending file and go to the point where the conflict is happening, and git will leave a note for you of the content it is having trouble with. The note will look similar to what happens in the gitref guide here, under "merge conflicts".

http://gitref.org/branching/#merge

You just have to remove the note, make the file look like it should to make everything compatible, and type

  git add .
  git rebase --continue

Then the rebasing process will continue, and at the end your branch should be compatible with master and ready to be merged in.

You can read more about this aspect of git rebase here if you like, although most likely you won't actually need more than we just talked about for wesnoth development. 

http://git-scm.com/book/en/Git-Branching-Rebasing

That's all for this guide, have fun hacking on wesnoth!

== Addendum ==
The following section contains commands with the -f / --force options. Those are usually used to rewrite history, which is fine for your own fork but disallowed for upstream. Do not use them on the main repository.


If a command goes awry, you can go to "git reflog", find the point just before you typed it, and reset --hard to that commit, and it should be like the bad thing never happened
 
revert every file in the repo to look like it did at the HEAD commit (last successful sync)
        git reset --hard HEAD

get a clean slate by syncing with upstream
        git fetch upstream
        git checkout -B master upstream/master
        git push --force origin master
if this errors, try again after
        git reset --hard HEAD

 
undo n commits, split/squash and apply again
        git reset HEAD~n
        git add -p
        git commit
        git push -f

You can also push from one ‘ref’ to a different one: 
        $ git push <remote> <from>:<to>

completely overwrite branch b with a (from a to b)
        git push -f origin a:b




before any commit:

* if the diff shows an invisible change in the first line - stop! Fix the file encoding back to UTF8 without BOM.
 
* don't forget the changelog (and, for visible changes, the players_changelog)
 
* try to keep the summary at 50 characters or less, maximum 72

== See Also ==

* [[Project#Developers]]

[[Category:Development]]