Difference between revisions of "SoC 2014 Spoffy Finishing and Enhancing the UMCD"

From The Battle for Wesnoth Wiki
Line 17: Line 17:
 
= Design Specification =
 
= Design Specification =
  
This is the current design proposal for continuing work with the UMCD. This is likely to be a continous work in progress up until the Summer of Code deadline. Firstly, I intend to outline the series of steps involved. After this, I will outline a timetable before going into more information on each step. Hopefully, this means the document will get more detailed as you read on, allowing for a logical progression.
+
This is the current design proposal for continuing work with the UMCD. This is likely to be a continous work in progress up until the Summer of Code deadline and possible after it.
  
== Steps ==
+
DISCLAIMER: Parts of this, primarily with respect to the WML packet details, are built on the formats, or indeed may be identical in some cases, to the formats detailed at [http://hyc.io/wesnoth/umcd.pdf his proposal's UMCD pdf]
  
# Bring the existing UMCD up to date with the latest Wesnoth build.
+
== Timetable ==
# Refactor the existing server for Neev.
 
# Basic Testing Client implementation.
 
# Update the WML Protocol schemas.
 
# Implement requests - Core functionality
 
  
== Timetable ==
+
This is a rough timetable full of approximations. As progress is made, I intend on adjusting the timetable to reflect likely completion dates.
  
 
<table>
 
<table>
Line 38: Line 34:
 
<tr class="odd">
 
<tr class="odd">
 
<td align="center">19/05/14</td>
 
<td align="center">19/05/14</td>
<td align="center">1. Updating the UMCD</td>
+
<td align="center">1. [[#step-1|Preliminary Tasks]]</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">Uni Exams 19 May - 7th Jun My Exams: 3 Days in this period</td>
 
<td align="center">Uni Exams 19 May - 7th Jun My Exams: 3 Days in this period</td>
 
</tr>
 
</tr>
 
<tr class="even">
 
<tr class="even">
<td align="center">24/05/14</td>
+
<td align="center">29/05/14</td>
<td align="center">2. Refactoring to Neev</td>
+
<td align="center">2. [[#step-2|Refactoring to Neev]]</td>
 
<td align="center">Optional Step, see description</td>
 
<td align="center">Optional Step, see description</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="odd">
 
<tr class="odd">
<td align="center">01/06/14</td>
+
<td align="center">04/06/14</td>
<td align="center">3. Standalone Test Client</td>
+
<td align="center">3. [[#step-3|Client Library]]</td>
 +
<td align="center">None</td>
 +
<td align="center">None</td>
 +
</tr>
 +
<tr class="even">
 +
<td align="center">07/06/14</td>
 +
<td align="center">4.2 [[#step-4.2|Uploading Addons]]</td>
 +
<td align="center">None</td>
 +
<td align="center">None</td>
 +
</tr>
 +
<tr class="odd">
 +
<td align="center">20/06/14</td>
 +
<td align="center">4.3 [[#step-4.3|Listing Addons]]</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="even">
 
<tr class="even">
<td align="center">08/06/14</td>
+
<td align="center">27/06/14</td>
<td align="center">4. WML Schema Completion</td>
+
<td align="center">4.4 [[#step-4.4|Downloading Addons]]</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="odd">
 
<tr class="odd">
<td align="center">15/06/14</td>
+
<td align="center">04/07/14</td>
<td align="center">5. Core Functionality</td>
+
<td align="center">4.5 [[#step-4.5|Deleting Addons]]</td>
<td align="center">Should produce working server</td>
+
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="even">
 
<tr class="even">
<td align="center">01/07/14</td>
+
<td align="center">07/07/14</td>
<td align="center">6. Security and Encryption</td>
+
<td align="center">4.6 [[#step-4.6|Update Password]]</td>
<td align="center">Basic production ready server</td>
+
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="odd">
 
<tr class="odd">
 
<td align="center">10/07/14</td>
 
<td align="center">10/07/14</td>
<td align="center">7. Refactoring</td>
+
<td align="center">4.7 [[#step-4.7|Request New Password]]</td>
<td align="center">Cleaning up produced code.</td>
+
<td align="center">None</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="even">
 
<tr class="even">
<td align="center">17/07/14</td>
+
<td align="center">13/07/14</td>
<td align="center">8. Server Administration</td>
+
<td align="center">5. [[#step-5|Encryption]]</td>
<td align="center">(For those with server access)</td>
+
<td align="center">Encryption in Net library</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="odd">
 
<tr class="odd">
<td align="center">24/07/14</td>
+
<td align="center">20/07/14</td>
<td align="center">9. Client integration</td>
+
<td align="center">6. [[#step-6|Client Integeration]]</td>
<td align="center">Adding client to Wesnoth</td>
+
<td align="center">Attaching client lib to Wesnoth</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="even">
 
<tr class="even">
<td align="center">31/07/14</td>
+
<td align="center">27/07/14</td>
<td align="center">10. Documentation</td>
+
<td align="center">7. [[#step-7|Non-Code Documentation]]</td>
<td align="center">Making it easy to get started.</td>
+
<td align="center">Tutorials, how to use.</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="odd">
 
<tr class="odd">
<td align="center">07/08/14</td>
+
<td align="center">29/07/14</td>
<td align="center">11. Build and Test Scripts</td>
+
<td align="center">8. [[#step-8|Improved Build Scripts]]</td>
<td align="center">None</td>
+
<td align="center">Improving build scripts further</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
 
<tr class="even">
 
<tr class="even">
<td align="center">18/08/14</td>
+
<td align="center">03/08/14</td>
<td align="center">12. &quot;Spare Time&quot;</td>
+
<td align="center">9. [[#step-9|Console Administration]]</td>
<td align="center">Not really. See description.</td>
+
<td align="center">Server admins controlling addons</td>
 
<td align="center">None</td>
 
<td align="center">None</td>
 
</tr>
 
</tr>
Line 116: Line 124:
 
== Details ==
 
== Details ==
  
=== Step 1 - Bringing the UMCD up to date. ===
+
=== 1 - Preliminary Tasks. ===
 +
 
 +
* Merge master into UMCD, in order to bring it up to date. Check that it compiles, everything is working. If not, fix it. Messy, but it'll save a lot of conflicts later on when we merge UMCD back into master hopefully. Should be a one-off.
 +
* Redo the database interface, as OTL is out of date and requires SQL2CPP to maintain a single schema copy and maintain type safety in the code. SQL2CPP is apparently hard to maintain and not a suitable long term solution.
 +
*# (Preferred) Switch to sqlpp11 - A type safe SQL library with MySQL and SQLite connectivity that could be nested as a git submodule.
 +
*# (Little Change) SOCI, or equivalent - Switch the library from OTL to another non-typesafe database access library such as SOCI and use a connector or ODBC.
 +
*# (Most Work) Switch to a different library as in 2, but wrap it with factory functions to produce type-safe access objects.
 +
* Change custom logging code in the UMCD to a logging library such as Boost.Logging, to avoid having to maintain a custom logging library. This just be removing the existing code, and replacing the calls to existing logging functions, hopefully.
 +
* Replace the boost shared_ptr and unique_ptr with their standard library counterparts, and similar wherever possible. Possibly also change bits to use variadic templates rather than strange boost includes.
 +
* Extract the UMCD's build scripts from the main Wesnoth scripts and put them in their own location.
 +
* Give the UMCD its own repository and initialise it as a submodule. See controverisial summary [[#seperate-repo|here]]
 +
 
 +
=== 2 - Refactoring the existing server for Neev ===
 +
 
 +
(Optional Step - Depending upon discussion with Trademark/Mentor) The existing server uses the boost.asio library for networking. Trademark has abstracted much of this into a simple networking library called Neev (available at http://github.com/ptal/Neev, or my fork at Spoffy/Neev).
 +
 
 +
This factors out much of the boilerplate we have in the UMCD for basic networking into a seperate library, promoting reusability in the long run and allowing it to be developed seperately.
 +
 
 +
This would involve initialising neev as a submodule of the UMCD
 +
 
 +
This would ultimately require more dicussion, and can easily be skipped if it's decided to not be required. Refactoring early should make it easier in the long run.
 +
 
 +
=== 3 - Client Library and Test Client ===
 +
 
 +
The client library for Wesnoth needs to be created for communicating with the UMCD. This code would be gradually built up over time into a library that can effectively be slotted into Wesnoth, in place of the current addon code.
 +
 
 +
In order to facilitate testing of both the library and server, a simple console frontend would be built over the library implementing the functionality expected from wesnoth. This would act the main testing tool for the server.
 +
 
 +
The majority of this code would go on to be reused in the client. At this point, the client should not take long to build at all, due to the lack of server functionality.
 +
 
 +
=== 4.1 - Special Messages ===
 +
 
 +
These are messages which the server may reply with as a response, instead of the expected reply packet.
 +
 
 +
===== Error Packet =====
 +
 
 +
Sent when the server could not understand the request, or the request is badly formatted, e.g missing values.<br />Request format by Trademark [http://hyc.io/wesnoth/umcd.pdf here] covers this packet
 +
 
 +
===== Request Unsuccessful Packet =====
 +
 
 +
Sent when the packet was correctly formed, but some part of the data of the packet, say a password or ID, was incorrect.<br />[[#packet-unsuccessful|View here]]
 +
 
 +
===== Acknowledgement Packet =====
 +
 
 +
Sent when a non-WML data payload was sent to confirm it has been received, so the communication can move on to the next stage.<br />[[#packet-ack|View here]]
 +
 
 +
=== 4.2 - Uploading to the Server ===
 +
 
 +
The initial functionality to be added to the server is uploading, as all other functionality is (arguably) a way to manipulate uploaded content.<br />Request format by Trademark [http://hyc.io/wesnoth/umcd.pdf here]
 +
 
 +
Note: Translation support is lacking in this description, as at the time of writing, I don't have time to learn the inner workings of Wescamp. Note2: I've read enough Translation stuff to realise it likely needs a module of its own in the source. Not sure if we just pull from Git and handle things ourselves, or use Wescamp.py though.
 +
 
 +
# Client sends server upload request - Request contains contents of umc.cfg detailed in the link, rather than it being a file. Also has hash of archive. The request is constructed using a clientside file (similar to .pbl) which also contains a path (relative or absolute) to the addon, meaning the creator can store the files wherever they want.
 +
# (Depends if ID is autogenerated or manual) Check if ID matches existing content. If not, upload new content. If it does, check password and [[#updating|update]].
 +
# Basic collision check - Content can't have the same name as existing content unless the two contents have the same ID.
 +
# Server sends status response, either 'Accepted', 'Error', 'Invalid' response.
 +
# If 'Accepted', client uploads archive.
 +
# The archive arrives.
 +
# Archive's hash verified, and a 'Acknowledgement' or 'Unsuccessful' packet is then sent, marking a successful or unsuccessful the upload.
 +
# Database stores:
 +
#* Id
 +
#* Name
 +
#* Description
 +
#* Type
 +
#* Version
 +
#* Authors
 +
#* Email
 +
#* Hash of password
 +
#* Archive hash
 +
#* Version
 +
#* Translatable
 +
#* Dependencies
 +
#* Icon path
 +
# Uploader IP, date + time of upload, path to addon archive are calculated and stored.
 +
# Translation stuff happens (I'll expand this when I have time!)
 +
 
 +
===== Updating Content =====
 +
 
 +
If we're updating content with a new version, follow the following procedure instead. It starts like this as the initial request is the same.
 +
 
 +
# Check password of the request. If it isn't correct, reply with Request Unsuccessful
 +
# Check version of new content &gt; old version. If not, return Request Unsuccessful. They can delete old versions if they need to reupload.
 +
# Send status response, &quot;Accepted&quot;, &quot;Error&quot; or &quot;Invalid&quot;
 +
# Client uploads addon archive.
 +
# Store the file in the addon's directory, probably addons/id/version
 +
# Update databases. Add a new version, and update the addon's current version to 'point' to the latest version.
 +
# Reply with an Acknowledgement packet when the update is complete, or an error if the archive got corrupted in transit.
 +
# More translation stuff?
 +
 
 +
On Upload fail:
 +
 
 +
Archive is deleted, databases are cleared (if anything has been added, which in this model shouldn't be the case)
 +
 
 +
Issues and Notes:
 +
 
 +
* ID calculated by the server, or assigned by the content's creator?
 +
* Translation integration - Need more time to learn about Wescamp
 +
* Protocol modification: No 'state' information, as it's the server's concern and the content doesn't need to know.
 +
* Protocol modification: Client sends a SHA-2 (or equivalent) hash to the server in the initial request to verify integrity. Better safe than corrupted.
 +
* umcd.cfg information now sent in the request. This is to avoid having to open the archive, remove umcd.cfg and repackage it due to it containing a pass. Alternatives are having umcd.cfg and the passphrase seperate, though this would mean it still needs umcd.cfg extracting.
 +
* Icons: Either send it up seperately, or extract from the archive once arrived. Discuss.
 +
* Alternative way of sending the payload covered [[#payload|here]]
 +
 
 +
=== 4.3 - Request List of Addons ===
 +
 
 +
This involves requesting a list of addons from the server.<br />Request and reply format by Trademark [http://hyc.io/wesnoth/umcd.pdf here]
 +
 
 +
# Client sends a request_umc_list to the server, detailed in the format pdf.
 +
# Server checks databases for all the latest versions of addons. Returns a WML file containing, as per format:
 +
#* ID
 +
#* Type
 +
#* Version
 +
#* Size
 +
#* Authors
 +
#* Last Updated Timestamp
 +
#* Downloads
 +
#* Dependencies (And whether they're automatically resolvable)
 +
#* Available Languages
 +
#* Icon File Hash
 +
# Server sends an archive of icon files. Either a tar, or a custom format.
 +
# Client replies with an Acknowledgement or Error packet.
 +
# Client displays this data in the addons menu.
  
The UMCD was, I believe, primarily coded during Summer of Code last year. At the time of writing, the asio_umcd branch is around 2100 commits behind Wesnoth master. The first step would be to rebase the UMCD onto master, avoiding merges where possible. Having briefly attempted this, there are a number of conflicts, so this would be the first task.
+
Issues and Notes:
  
=== Step 2 - Refactoring the existing server for Neev ===
+
* Not sure where the quality of the translation is stored. It'd be good to send it, but I need to locate it. Talk to AI?
 +
* Client might need to send an acknowledgement between list and icons, as I don't think we can guarantee order with async transmissions.
 +
* Alternative way of sending the payload covered [[#payload|here]]
  
(Optional Step - Depending upon discussion with Trademark/Mentor) The existing server uses the boost.asio library for networking. Trademark has abstracted much of this into a simple networking library called Neev (available at http://github.com/ptal/Neev, or my fork at Spoffy/Neev. After discussion, it seems like it would be a better idea to use Neev than a custom abstraction over asio, as it will hopefully make the server more maintable in the long run. This would ultimately require more dicussion, and can easily be skipped if it's decided to not be required. Refactor is early to make it easier in the long run.
+
=== 4.4 - Download Addon ===
  
=== Step 3 - Basic standalone client implementation ===
+
Request a download of a specific addon from the server.<br />Request format by Trademark [http://hyc.io/wesnoth/umcd.pdf here], minus the language.
  
DISCLAIMER: At the current point of writing, I am unsure of how the client will integrate with Wesnoth. I presume it will interface with Wesnoth's GUI OR be a standalone tool. I will attempt to clarify this on IRC. Until then, take this section lightly. Consider it a work in progress.
+
# Send a request to the server asking download of addon with specific ID.
 +
# Server replies with:
 +
#* ID
 +
#* Size
 +
# Server sends addon, which is already stored as an archive.
 +
# Client acknowledges receipt of addon, sending an Acknowledgement packet or an 'Error' packet if for some reason the data isn't received.
 +
# If addons have dependencies which aren't downloaded, offer to the user the option to try and download them (if it has them?) from the addon server this addon was taken from. If yes, launch more download requests for each addon.
  
UPDATE: On studying the existing server, the client appears to be integrated into Wesnoth. This client will be a standalone tool for testing with reusable code.
+
Issues and Notes:
  
A basic client is necessary to test the functionality of the server. I propose creating a test client (using Neev or existing asio) early so that it can be develop in parallel with the server, allowing for constant testing. The code would consist of a basic method for interfacing with the console, as well as a class of methods designed to be reused in Wesnoth itself.
+
* Not specifying a specific language, as I /believe/ Wescamp translates all of them anyway, and I'd rather just store it permanently as a compressed archive and serve up that, than re-pack the archive every transmission with different languages.
 +
* Client might need to acknowledge receipt of WML in order to signal the server to send the addon, not necessary with alternative method mentioned below.
 +
* Alternative way of sending the payload covered [[#payload|here]]
  
At this point, the test client would simply be created and brought into parellel with the current state of the server. It would be a basic command-line client on Linux, however it would naturally be buildable on Windows as well.
+
=== 4.5 - Delete Addon ===
  
=== Step 4 - Complete UML Protocol Schema ===
+
Request the deletion of a specific addon from the server.<br />Request format by Trademark [http://hyc.io/wesnoth/umcd.pdf here] with minor changes detailed below.
  
The current UML Protocol Schemas are incomplete, offering a small number of the schemas required to implement a functioning UMCD. Naturally, these need to implemented. Implementation details based on Trademark's existing work to follow.
+
# Send a request to the server asking deletion for the addon with a specific ID. Optionally contains a Version, specifying deletion of specific version.
 +
# Server verifies password, comparing hash to stored hash.
 +
# Server attempts addon deletion, checking if the version is valid against the SQL tables.
 +
# Server replies with 'Acknowledgement' or 'Request unsuccessful' message
  
=== Step 5 - Adding remaining server functionality ===
+
Issues and Notes:
  
At this point, the prep for adding the core functionality (Adding, updating, deleting, downloading addons) has been completed. This functionality can thus now be implemented and develop in conjunction with the client library. This means that real-functionality can be tested as it's produced, rather than engaging in creating very fake scenarios Implementation details soon to follow.
+
* Client has to go through multiple 'Yes I'm sure' menus to make sure they want to delete the addon.
  
=== Step 6 - Basic Security and Encryption ===
+
=== 4.6 - Update Password ===
  
The server is now functional, capable of basic addon management. Huzzah! However, it needs securing to ensure no passwords get leaked onto the internet! Firstly, this ,means laying down SSL encryption over the existing network framework (Neev or Asio). This is one point where Neev would be an advantage, as we're guaranteeing modular and reusable code, though at the extra cost of time due to implementing SSL in Neev using Asio, then tweaking existing code to specify it wants an SSL connection.
+
Request the password for an addon be changed.<br />Request format by Trademark [http://hyc.io/wesnoth/umcd.pdf here]
  
Once SSL encryption is active, we can implement logging in with passwords. While implementation details will follow, I'm initially proposing sending the password over the SSL connection, then applying SHA512 and matching against a hash in the database. Further security could be added, such as encrypting the password, then sending the encrypted password over SSL and descrypting it on the other side. I need to research if this would be required.
+
# Client sends a request to the server, this is as detailed above.
 +
# Server changes the password if the new password is stong enough and old password is correct.
 +
# Server replies with an 'Acknowledgement' or 'Request Unsuccessful' packet, depending on whether it was correctly processed.
 +
 
 +
Issues and Notes:
 +
 
 +
* Possibly verify the strength of a password clientside? Or this could be introducing a security flaw if people change it and manually compile their clients, though they would still need the old password.
 +
 
 +
=== 4.7 - Request new Password ===
  
Further implementation details to follow.
+
Request for a new password to be sent to the author's email.<br />Request format by Trademark [http://hyc.io/wesnoth/umcd.pdf here], no changes.
  
=== Step 7 - Refactor Refactor Refactor ===
+
# Client sends a request new password packet.
 +
# Server replies with a reply packet detailing the email to which the password has been sent.
  
The server is now, in theory, ready to see some basic use. Users can upload and download securely, as well as add and delete addons from the server. I will also have gotten, hopefully, a very good feel for the code by this point. That makes it a good time to refactor any bits of code I feel could be improved before continuing on. This will hopefully save time later.
+
Issues and Notes:
  
No further information, as the implementation relies on knowledge I will gain during the project, although an example might be the environment code mentioned on the UMCD proposals page.
+
* This is exactly the same pretty much as Trademark's version, simply with Request Unsuccessful instead of warning if it's an invalid ID.
 +
* Currently emails are stored per-addon. Should this be per-maintainer? Do we need user logins instead of addon logins? Tune in next week to find out!
  
=== Step 8 - Console Administration ===
+
=== 5 - Security and Encryption ===
  
Allow the server host to locally view all registered modders, all available addons, last update, etc. and be able to delete, move, add and otherwise modify stored addons, probably in a way that doesn't modify the addon itself, in order to maintain integrity and try to prevent evil server owners from adding malicious files, or whatever it may be. More details to follow as I think of them. Emphasis at this point is on local administration (i.e, by people with server access).
+
Basic functionality has been implemented at this point. At this point, we need to integrate SSL into the underlying networking library, be it Neev or the existing client/server implementation. This should be (near) transparent from the UMCD's perspective.
  
=== Step 9 - Client Integration ===
+
Once SSL encryption is active, we can implement logging in with passwords. While implementation details will follow, I'm initially proposing sending the password over the SSL connection, then applying SHA512 and matching against a hash in the database. Further security could be added, such as encrypting the password, then sending the encrypted password over SSL and descrypting it on the other side. I need to research if this would be required.
  
The WML based communication protocol should be well tested at this point, given it was implemented in Step-4. The server should also have all the core functionality it needs for a release and it should be stable in terms of its interface with the client. Therefore this is a good time to integrate the client into the Wesnoth client itself ready for release.
+
=== 6 - Client Integration ===
  
I would initially propose maintaining the existing GUI interface and simply retying it to the client library that has been developing seperately. Whether the GUI was changed would depend on the functionality needed at this point in time, though I don't, at the point of writing, predict any changes, except perhaps with how addons are uploaded? A few more details to follow, as usual, though it depends heavily on the library at this point.
+
Integrating the client library into the main Wesnoth client. It's quite a late point to integrate, but hopefully the library should have matured and be stable in terms of interfaces, making this the simple task of integrating the Wesnoth addon GUI in its current form with the library, as we've been developing it seperately along with the test client.
  
=== Step 10 - Documentation Improvements ===
+
=== 7 - Non-code Documentation - How to use, Tutorials, Etc. ===
  
 
Improving the documentation in order to the make the UMCD easy(easier) to use, as well as detailing the files modders will need to create and how they can upload their addons.
 
Improving the documentation in order to the make the UMCD easy(easier) to use, as well as detailing the files modders will need to create and how they can upload their addons.
  
This is a big area for improvement, and will likely involve: Details coming later.
+
I anticipate this mostly being tutorials and explanatory documents unrelated to the code itself or the documentation the code contains. It will li
 
 
=== Step 11 - Testing and Build Scripts ===
 
  
The UMCD currently has a build script embedded within Wesnoth's CMake files. I would propose seperating this file out, then either A) Link to it from the main CMake file, or B) Have it function as a seperate build system.
+
=== 8 - Build Script improvements ===
  
It would also be improved to:
+
Improving the build scripts to:
  
 
* Identify clearly missing dependencies
 
* Identify clearly missing dependencies
 
* Optionally download and build missing dependencies, or a subset of them.
 
* Optionally download and build missing dependencies, or a subset of them.
 
* Configure the database and create the required tables.
 
* Configure the database and create the required tables.
* Build UMCD Tests
 
* Run UMCD tests on request
 
 
* Provide an quick and simply default build, with informative instructions on errors or failures.
 
* Provide an quick and simply default build, with informative instructions on errors or failures.
 
* Run tests on a seperate database to the main DB to avoid polluting it.
 
* Run tests on a seperate database to the main DB to avoid polluting it.
 +
 +
Some of these, for example highlighting missing dependencies, should have already been implemented by this point in the project. Certainly the make files will incorporate testing the server, client and database, as the test cases will have been built up alongside the functionality.
  
 
Ideally the build system will also allow cross platform building, presumably on Cygwin (or equivalent).
 
Ideally the build system will also allow cross platform building, presumably on Cygwin (or equivalent).
  
The testing scripts will also be improved at this point, and the UMCD will undergo a thorough testing to ensure that it is ready for deployment.
+
=== 9 - Console Administration ===
  
=== Step 12 - &quot;Spare Time&quot; ===
+
Allow the server host to locally view all registered modders, all available addons, last update, etc. and be able to delete, move, add and otherwise modify stored addons, probably in a way that doesn't modify the addon itself, in order to maintain integrity and try to prevent evil server owners from adding malicious files, or whatever it may be. More details to follow as I think of them. Emphasis at this point is on local administration (i.e, by people with server access).
 +
 
 +
=== 10 - &quot;Spare Time&quot; ===
  
 
Naturally this isn't time spent doing nothing. I couldn't think of a better name. There will undoubtedly be delays in the project, there are also a number of tasks which are there, but don't necessarily need doing in any particular order, that I don't feel comfortable just adding to the timetable until the project has started. I may also think of new, very important things that need time. This will be a slot to get a lot of the misc things done, that aren't vitally important (All the vitally important stuff has been completed) but would still benefit the project.
 
Naturally this isn't time spent doing nothing. I couldn't think of a better name. There will undoubtedly be delays in the project, there are also a number of tasks which are there, but don't necessarily need doing in any particular order, that I don't feel comfortable just adding to the timetable until the project has started. I may also think of new, very important things that need time. This will be a slot to get a lot of the misc things done, that aren't vitally important (All the vitally important stuff has been completed) but would still benefit the project.
Line 204: Line 356:
  
 
== Other Possibilities/Ideas/Radical Changes ==
 
== Other Possibilities/Ideas/Radical Changes ==
 +
 +
=== Alternative 'Payload' sending ===
 +
 +
In the above proposal, payloads are sent seperately to the UML, and then acknowledged. An alternative might be to guarantee every packet contains a WML header, optionally followed by a payload in the same transmission, much as is done with IP or TCP. Discuss.
  
 
=== Remove UMCD from Wesnoth Repo/lower coupling - Instantiate as a submodule. ===
 
=== Remove UMCD from Wesnoth Repo/lower coupling - Instantiate as a submodule. ===
Line 225: Line 381:
 
* I'm aiming to produce example code for the Neev library so you guys can look at my code, see where I can improve, etc, rather than doing any UMCD work initially. this is as much to learn the library as anything. I'll probably also do some Asio stuff. After that, I might start working on the UMCD early in my spare time! Why not! :)
 
* I'm aiming to produce example code for the Neev library so you guys can look at my code, see where I can improve, etc, rather than doing any UMCD work initially. this is as much to learn the library as anything. I'll probably also do some Asio stuff. After that, I might start working on the UMCD early in my spare time! Why not! :)
 
* Typedefs are useful, especially when given public scope in class, as are 'using', defines and other aliasing tools, but I feel over-aliasing code can almost act as obfuscation, as you need to follow a trail of typeDefs all the way back to find out what the original class if (though some IDEs may do this for you). Ideally I'd like to reduce the use of these (primarily defines, but also any string of two or more typedefs where the name changes really) through refactoring wherever possible. Happy to discuss this with someone more knowledgeable!
 
* Typedefs are useful, especially when given public scope in class, as are 'using', defines and other aliasing tools, but I feel over-aliasing code can almost act as obfuscation, as you need to follow a trail of typeDefs all the way back to find out what the original class if (though some IDEs may do this for you). Ideally I'd like to reduce the use of these (primarily defines, but also any string of two or more typedefs where the name changes really) through refactoring wherever possible. Happy to discuss this with someone more knowledgeable!
 +
 +
== Packet Formats ==
 +
 +
===== Request Unsuccessful =====
 +
 +
<pre>[request_failed]
 +
    msg = MESSAGE
 +
[/request_failed]</pre>
 +
===== Acknowledgement =====
 +
 +
<pre>[acknowledgement]
 +
    msg = MESSAGE
 +
[/acknowledgement]</pre>
  
 
=Questionnaire=
 
=Questionnaire=

Revision as of 22:04, 20 March 2014


This page is related to Summer of Code 2014
See the list of Summer of Code 2014 Ideas



This is a Summer of Code 2014 student page


Description

Spoffy - UMCD Extension Proposal

I intend to complete and extend the User Made Content Daemon produced by Trademark during GSoC 2013. This means implementing the basic functionality (Uploading, downloading addons, etc) in a secure manner. After this, I would continue on to extend it by adding administration tools, refactoring code, adding build scripts, adding cross-platform support (in principle), and other things (probably!). Outlined on this page is an in-depth proposal

More to come as I fill this out.

IRC

spoffy

Design Specification

This is the current design proposal for continuing work with the UMCD. This is likely to be a continous work in progress up until the Summer of Code deadline and possible after it.

DISCLAIMER: Parts of this, primarily with respect to the WML packet details, are built on the formats, or indeed may be identical in some cases, to the formats detailed at his proposal's UMCD pdf

Timetable

This is a rough timetable full of approximations. As progress is made, I intend on adjusting the timetable to reflect likely completion dates.

Date Step Notes Other Info
19/05/14 1. Preliminary Tasks None Uni Exams 19 May - 7th Jun My Exams: 3 Days in this period
29/05/14 2. Refactoring to Neev Optional Step, see description None
04/06/14 3. Client Library None None
07/06/14 4.2 Uploading Addons None None
20/06/14 4.3 Listing Addons None None
27/06/14 4.4 Downloading Addons None None
04/07/14 4.5 Deleting Addons None None
07/07/14 4.6 Update Password None None
10/07/14 4.7 Request New Password None None
13/07/14 5. Encryption Encryption in Net library None
20/07/14 6. Client Integeration Attaching client lib to Wesnoth None
27/07/14 7. Non-Code Documentation Tutorials, how to use. None
29/07/14 8. Improved Build Scripts Improving build scripts further None
03/08/14 9. Console Administration Server admins controlling addons None

NOTE: The Timetable is an approximation. Some scenarios are worst case, some are exact expected length. Hopefully it's an overestimation, and I can accomplish more than is listed.

NOTE2: During my exam period (19th May - 7th June) I will likely be quite slow with producing code. It may be my exams fall at the very beginning, in which case it would not affect SoC at all. On the timetable, I have considered this entire period as 'slow code'.

Details

1 - Preliminary Tasks.

  • Merge master into UMCD, in order to bring it up to date. Check that it compiles, everything is working. If not, fix it. Messy, but it'll save a lot of conflicts later on when we merge UMCD back into master hopefully. Should be a one-off.
  • Redo the database interface, as OTL is out of date and requires SQL2CPP to maintain a single schema copy and maintain type safety in the code. SQL2CPP is apparently hard to maintain and not a suitable long term solution.
    1. (Preferred) Switch to sqlpp11 - A type safe SQL library with MySQL and SQLite connectivity that could be nested as a git submodule.
    2. (Little Change) SOCI, or equivalent - Switch the library from OTL to another non-typesafe database access library such as SOCI and use a connector or ODBC.
    3. (Most Work) Switch to a different library as in 2, but wrap it with factory functions to produce type-safe access objects.
  • Change custom logging code in the UMCD to a logging library such as Boost.Logging, to avoid having to maintain a custom logging library. This just be removing the existing code, and replacing the calls to existing logging functions, hopefully.
  • Replace the boost shared_ptr and unique_ptr with their standard library counterparts, and similar wherever possible. Possibly also change bits to use variadic templates rather than strange boost includes.
  • Extract the UMCD's build scripts from the main Wesnoth scripts and put them in their own location.
  • Give the UMCD its own repository and initialise it as a submodule. See controverisial summary here

2 - Refactoring the existing server for Neev

(Optional Step - Depending upon discussion with Trademark/Mentor) The existing server uses the boost.asio library for networking. Trademark has abstracted much of this into a simple networking library called Neev (available at http://github.com/ptal/Neev, or my fork at Spoffy/Neev).

This factors out much of the boilerplate we have in the UMCD for basic networking into a seperate library, promoting reusability in the long run and allowing it to be developed seperately.

This would involve initialising neev as a submodule of the UMCD

This would ultimately require more dicussion, and can easily be skipped if it's decided to not be required. Refactoring early should make it easier in the long run.

3 - Client Library and Test Client

The client library for Wesnoth needs to be created for communicating with the UMCD. This code would be gradually built up over time into a library that can effectively be slotted into Wesnoth, in place of the current addon code.

In order to facilitate testing of both the library and server, a simple console frontend would be built over the library implementing the functionality expected from wesnoth. This would act the main testing tool for the server.

The majority of this code would go on to be reused in the client. At this point, the client should not take long to build at all, due to the lack of server functionality.

4.1 - Special Messages

These are messages which the server may reply with as a response, instead of the expected reply packet.

Error Packet

Sent when the server could not understand the request, or the request is badly formatted, e.g missing values.
Request format by Trademark here covers this packet

Request Unsuccessful Packet

Sent when the packet was correctly formed, but some part of the data of the packet, say a password or ID, was incorrect.
View here

Acknowledgement Packet

Sent when a non-WML data payload was sent to confirm it has been received, so the communication can move on to the next stage.
View here

4.2 - Uploading to the Server

The initial functionality to be added to the server is uploading, as all other functionality is (arguably) a way to manipulate uploaded content.
Request format by Trademark here

Note: Translation support is lacking in this description, as at the time of writing, I don't have time to learn the inner workings of Wescamp. Note2: I've read enough Translation stuff to realise it likely needs a module of its own in the source. Not sure if we just pull from Git and handle things ourselves, or use Wescamp.py though.

  1. Client sends server upload request - Request contains contents of umc.cfg detailed in the link, rather than it being a file. Also has hash of archive. The request is constructed using a clientside file (similar to .pbl) which also contains a path (relative or absolute) to the addon, meaning the creator can store the files wherever they want.
  2. (Depends if ID is autogenerated or manual) Check if ID matches existing content. If not, upload new content. If it does, check password and update.
  3. Basic collision check - Content can't have the same name as existing content unless the two contents have the same ID.
  4. Server sends status response, either 'Accepted', 'Error', 'Invalid' response.
  5. If 'Accepted', client uploads archive.
  6. The archive arrives.
  7. Archive's hash verified, and a 'Acknowledgement' or 'Unsuccessful' packet is then sent, marking a successful or unsuccessful the upload.
  8. Database stores:
    • Id
    • Name
    • Description
    • Type
    • Version
    • Authors
    • Email
    • Hash of password
    • Archive hash
    • Version
    • Translatable
    • Dependencies
    • Icon path
  9. Uploader IP, date + time of upload, path to addon archive are calculated and stored.
  10. Translation stuff happens (I'll expand this when I have time!)
Updating Content

If we're updating content with a new version, follow the following procedure instead. It starts like this as the initial request is the same.

  1. Check password of the request. If it isn't correct, reply with Request Unsuccessful
  2. Check version of new content > old version. If not, return Request Unsuccessful. They can delete old versions if they need to reupload.
  3. Send status response, "Accepted", "Error" or "Invalid"
  4. Client uploads addon archive.
  5. Store the file in the addon's directory, probably addons/id/version
  6. Update databases. Add a new version, and update the addon's current version to 'point' to the latest version.
  7. Reply with an Acknowledgement packet when the update is complete, or an error if the archive got corrupted in transit.
  8. More translation stuff?

On Upload fail:

Archive is deleted, databases are cleared (if anything has been added, which in this model shouldn't be the case)

Issues and Notes:

  • ID calculated by the server, or assigned by the content's creator?
  • Translation integration - Need more time to learn about Wescamp
  • Protocol modification: No 'state' information, as it's the server's concern and the content doesn't need to know.
  • Protocol modification: Client sends a SHA-2 (or equivalent) hash to the server in the initial request to verify integrity. Better safe than corrupted.
  • umcd.cfg information now sent in the request. This is to avoid having to open the archive, remove umcd.cfg and repackage it due to it containing a pass. Alternatives are having umcd.cfg and the passphrase seperate, though this would mean it still needs umcd.cfg extracting.
  • Icons: Either send it up seperately, or extract from the archive once arrived. Discuss.
  • Alternative way of sending the payload covered here

4.3 - Request List of Addons

This involves requesting a list of addons from the server.
Request and reply format by Trademark here

  1. Client sends a request_umc_list to the server, detailed in the format pdf.
  2. Server checks databases for all the latest versions of addons. Returns a WML file containing, as per format:
    • ID
    • Type
    • Version
    • Size
    • Authors
    • Last Updated Timestamp
    • Downloads
    • Dependencies (And whether they're automatically resolvable)
    • Available Languages
    • Icon File Hash
  3. Server sends an archive of icon files. Either a tar, or a custom format.
  4. Client replies with an Acknowledgement or Error packet.
  5. Client displays this data in the addons menu.

Issues and Notes:

  • Not sure where the quality of the translation is stored. It'd be good to send it, but I need to locate it. Talk to AI?
  • Client might need to send an acknowledgement between list and icons, as I don't think we can guarantee order with async transmissions.
  • Alternative way of sending the payload covered here

4.4 - Download Addon

Request a download of a specific addon from the server.
Request format by Trademark here, minus the language.

  1. Send a request to the server asking download of addon with specific ID.
  2. Server replies with:
    • ID
    • Size
  3. Server sends addon, which is already stored as an archive.
  4. Client acknowledges receipt of addon, sending an Acknowledgement packet or an 'Error' packet if for some reason the data isn't received.
  5. If addons have dependencies which aren't downloaded, offer to the user the option to try and download them (if it has them?) from the addon server this addon was taken from. If yes, launch more download requests for each addon.

Issues and Notes:

  • Not specifying a specific language, as I /believe/ Wescamp translates all of them anyway, and I'd rather just store it permanently as a compressed archive and serve up that, than re-pack the archive every transmission with different languages.
  • Client might need to acknowledge receipt of WML in order to signal the server to send the addon, not necessary with alternative method mentioned below.
  • Alternative way of sending the payload covered here

4.5 - Delete Addon

Request the deletion of a specific addon from the server.
Request format by Trademark here with minor changes detailed below.

  1. Send a request to the server asking deletion for the addon with a specific ID. Optionally contains a Version, specifying deletion of specific version.
  2. Server verifies password, comparing hash to stored hash.
  3. Server attempts addon deletion, checking if the version is valid against the SQL tables.
  4. Server replies with 'Acknowledgement' or 'Request unsuccessful' message

Issues and Notes:

  • Client has to go through multiple 'Yes I'm sure' menus to make sure they want to delete the addon.

4.6 - Update Password

Request the password for an addon be changed.
Request format by Trademark here

  1. Client sends a request to the server, this is as detailed above.
  2. Server changes the password if the new password is stong enough and old password is correct.
  3. Server replies with an 'Acknowledgement' or 'Request Unsuccessful' packet, depending on whether it was correctly processed.

Issues and Notes:

  • Possibly verify the strength of a password clientside? Or this could be introducing a security flaw if people change it and manually compile their clients, though they would still need the old password.

4.7 - Request new Password

Request for a new password to be sent to the author's email.
Request format by Trademark here, no changes.

  1. Client sends a request new password packet.
  2. Server replies with a reply packet detailing the email to which the password has been sent.

Issues and Notes:

  • This is exactly the same pretty much as Trademark's version, simply with Request Unsuccessful instead of warning if it's an invalid ID.
  • Currently emails are stored per-addon. Should this be per-maintainer? Do we need user logins instead of addon logins? Tune in next week to find out!

5 - Security and Encryption

Basic functionality has been implemented at this point. At this point, we need to integrate SSL into the underlying networking library, be it Neev or the existing client/server implementation. This should be (near) transparent from the UMCD's perspective.

Once SSL encryption is active, we can implement logging in with passwords. While implementation details will follow, I'm initially proposing sending the password over the SSL connection, then applying SHA512 and matching against a hash in the database. Further security could be added, such as encrypting the password, then sending the encrypted password over SSL and descrypting it on the other side. I need to research if this would be required.

6 - Client Integration

Integrating the client library into the main Wesnoth client. It's quite a late point to integrate, but hopefully the library should have matured and be stable in terms of interfaces, making this the simple task of integrating the Wesnoth addon GUI in its current form with the library, as we've been developing it seperately along with the test client.

7 - Non-code Documentation - How to use, Tutorials, Etc.

Improving the documentation in order to the make the UMCD easy(easier) to use, as well as detailing the files modders will need to create and how they can upload their addons.

I anticipate this mostly being tutorials and explanatory documents unrelated to the code itself or the documentation the code contains. It will li

8 - Build Script improvements

Improving the build scripts to:

  • Identify clearly missing dependencies
  • Optionally download and build missing dependencies, or a subset of them.
  • Configure the database and create the required tables.
  • Provide an quick and simply default build, with informative instructions on errors or failures.
  • Run tests on a seperate database to the main DB to avoid polluting it.

Some of these, for example highlighting missing dependencies, should have already been implemented by this point in the project. Certainly the make files will incorporate testing the server, client and database, as the test cases will have been built up alongside the functionality.

Ideally the build system will also allow cross platform building, presumably on Cygwin (or equivalent).

9 - Console Administration

Allow the server host to locally view all registered modders, all available addons, last update, etc. and be able to delete, move, add and otherwise modify stored addons, probably in a way that doesn't modify the addon itself, in order to maintain integrity and try to prevent evil server owners from adding malicious files, or whatever it may be. More details to follow as I think of them. Emphasis at this point is on local administration (i.e, by people with server access).

10 - "Spare Time"

Naturally this isn't time spent doing nothing. I couldn't think of a better name. There will undoubtedly be delays in the project, there are also a number of tasks which are there, but don't necessarily need doing in any particular order, that I don't feel comfortable just adding to the timetable until the project has started. I may also think of new, very important things that need time. This will be a slot to get a lot of the misc things done, that aren't vitally important (All the vitally important stuff has been completed) but would still benefit the project.

Ideas for this slot:

  • Fixing the Valgrind memory errors mentioned in the wiki
  • Improving the database with algorithms/SOCI/sql2cpp/multiple database support or refactoring chunks of the code in UMCD and Neev to improve readability. A
  • Lowering UMCD Coupling with Wesnoth
  • Improving the client-side interface if needed

More will likely be added to this as the project goes on.

Other Possibilities/Ideas/Radical Changes

Alternative 'Payload' sending

In the above proposal, payloads are sent seperately to the UML, and then acknowledged. An alternative might be to guarantee every packet contains a WML header, optionally followed by a payload in the same transmission, much as is done with IP or TCP. Discuss.

Remove UMCD from Wesnoth Repo/lower coupling - Instantiate as a submodule.

The UMCD is itself an independant entity, which communicates over the network with other entities (clients). It is not integrated into Wesnoth, but shares some dependencies.

Having the UMCD contained within the Wesnoth repo is, possibly, adding unnecessary coupling between the UMCD and Wesnoth itself. I would propose having the UMCD in its own repository, linked via Git submodules to the main repository (Either with the UMCD a submodule of Wesnoth, or vice versa). This would make the UMCD a standalone entity, and remove the need for it to be constantly rebased alongside the main code, the majority of which is unrealted the the UMCD itself. It would also allow for the UMCD to have its own build script, which makes sense, again for coupling reasons.

Downsides include having the client in a seperate repository to the server, although with submodules this should be transparent to the person building wesnoth, and the use of submodules allow

Implementation details to follow.

Remove existing boost pointers and replacing them with standard C++11 library pointers.

Call me radical, but I believe the standard library should be used wherever possible because.. well, it's just that, it's standard. This may be a minor nitpick, but I think now C++11 has unique_ptr and shared_ptr, they should be used instead of the Boost ones. While we need boost for asio anyway, I feel it's always good to reduce dependency on libraries where they aren't really needed. Note: There may be some demonstrable difference I'm not aware of, in which case, let me know/discuss this with me so I can change this on the proposal.

Misc Notes about the Proposal and Me

  • I like to focus on code readability and maintainabilty. Good code should be able to be read like a book, without needing to dive into docs often at all.
  • I took an iterative approach to this, outlining what needs doing before considering how it would be accomplished. As such, expect more detail to be added, and the timetable to shift around.
  • I'm aiming to produce example code for the Neev library so you guys can look at my code, see where I can improve, etc, rather than doing any UMCD work initially. this is as much to learn the library as anything. I'll probably also do some Asio stuff. After that, I might start working on the UMCD early in my spare time! Why not! :)
  • Typedefs are useful, especially when given public scope in class, as are 'using', defines and other aliasing tools, but I feel over-aliasing code can almost act as obfuscation, as you need to follow a trail of typeDefs all the way back to find out what the original class if (though some IDEs may do this for you). Ideally I'd like to reduce the use of these (primarily defines, but also any string of two or more typedefs where the name changes really) through refactoring wherever possible. Happy to discuss this with someone more knowledgeable!

Packet Formats

Request Unsuccessful
[request_failed]
    msg = MESSAGE
[/request_failed]
Acknowledgement
[acknowledgement]
    msg = MESSAGE
[/acknowledgement]

Questionnaire

1) Basics

1.01) Preamble I may tend to rant on a little bit in this. I hope it gives a better, more honest taste of who I am. If not, then let me know and I can make an abridged version!

1.1) Write a small introduction to yourself. Heyho! I'm Callum, a 1st year Software Engineering student in Southern England! (Well aware this is a public wiki. This will be updated in the actual application!). I'm an avid hacker and open source supporter, going to FOSDEM in 2014 and trying to use as much free software as possible! I also love Hackathons, having gone to 3 between February and March and organising another for late march! All of the code produced in these is available as free software on my Github page (which I encourage people to look at. Especially the Musical Packets hack, it's awesome IMO!).

1.2) State your preferred email address.

spoffeh@gmail.com

1.3) If you have chosen a nick for IRC and Wesnoth forums, what is it?

Spoffy - This will always be my nickname where I can get it! (See my email for what happens when I can't :( )

1.4) Why do you want to participate in summer of code?

I want to hack on more open source projects. While I do enjoy making my own from scratch, it'd be nice to actually chip in and get involved with larger projects. Sadly, the entry barrier to some of these can be quite high, between mailing lists, IRC, coding styles, learning the source, etc, and I rarely seem to have time between the other things I'm hacking to justify it! For me, Google Summer of Code really does help make that initial apparent investment seem more than worthwhile! The money also means I can spend my summer hacking awesome open source things, rather than working at a shop or taking an Internship at somewhere boring and/or propietary. That wouldn't be nearly as cool!

1.5) What are you studying, subject, level and school?

Software Engineer MEng, 1st year. I'll update the school on the actual application. Again, I'm well aware of the internet's scope.

1.6) What country are you from, at what time are you most likely to be able to join IRC?

England! I try to be on IRC whenever possible, but naturally I sleep. I aim for at least 7pm - 11pm GMT each day, and I'm usually on and off at various points between 10am and 12am. Soon (Or perhaps already, if I haven't changed this by the time its submitted) I plan on just rigging one up to run permanently on a remote server to log ALL the messages. :)

1.7) Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.

I have a planned vacation abroad for two weeks around June time, however I plan on taking my laptop and a Pi, with the full intention of continuing coding regardless.

2) Experience

2.1) What programs/software have you worked on before?

I spent 2 years or so prior to University working on a game called Space Station 13 on the BYOND platform. I'm the head programmer for one of the game communities there, although I'm trying to phase out my involvement somewhat due to other commitments (such as SoC and Uni :) ).

Since coming to University, I've played with a variety of projects.. if played is the right word. Since Java is the main language used, I've obviously done a lot of work on Courseworks and extra projects in it, though these are minor things, often with quickly hacked code due to one-week deadlines. Unfortunately, this has also meant University has taken up much of my time, and due to the nature of the projects, left me with little to show for it! (I'll note some exceptions later).

I've also been to a number of hackathons and hacked together projects there, including Musical Packets and a few others (with Musical Packets being one I'm working on refining and looking to continue on and off). These are often made up of very quick prototype code, but are almost always working systems.

2.2) Have you developed software in a team environment before? (As opposed to hacking on something on your own)

As mentioned above, I've been a part of/head of the SS13 programming team for a good two or three years, which has varied between 2 and 5 active members in that time. I've also had group projects at University, and a number of hackathons, all of which have involved teams of two or more!

2.3) Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?

Sadly no, this is my first year! Though it should be an exciting one!

2.4) Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.

As above, I'm involved with Musical Packets (ish, it's only done by 3 people. Does that count as involved?), and SS13, with the latter being phased out. I usually only contribute to these when I have free time, with other things such as Uni taking priority.

2.5) Gaming experience - Are you a gamer?

Very much so! I have a rather large game collection, though I'm struggling to find time to play them between all the hacking!

2.5.1) What type of gamer are you?

I'm tempted to say casual? I have lots of games, but I don't often play them at the moment. Except Tetris. That's always strangely addicting. Can one be a casual hardcore tetris player?

2.5.2) What type of games?

TETRI... oh, other games? Various, I enjoy almost every variety of game. Dating sims are a notable exception... Anyway. Puzzle games are always awesome because of how intellectually challenging they are. I've spent long stints playing Tetris and Puzzle Pirates. I also once spent 12 hours or so optimising a SpaceChem puzzle. I've basically refused to play since, given I know I'll get sucked in for another puzzle. I quite enjoy Turn based strategy as well, although I find many games fail to get it quite right, often requiring super in-depth knowledge of vast tech trees (Space Empires for example.). Wesnoth is one exception to that, more on that later I suspect!

I do also enjoy sandbox games. Anything that lets you stretch yourselve creatively while also posing a challenge. I find Minecraft often fails in the latter, although Space Engineers and other ship-building or colony-building games like Hazeron are often entertaining.

Artemis Bridge Sim is also a game that I rarely get to play, but stands out as being decidedly awesome. How would this even by categorised.

Also, space games. Always.

2.5.3) What type of opponents do you prefer?

I'm guessing AI or Human? Depends on the game. Definitely human in an FPS. Puzzle games, I like to challenge myself, or another human player. AIs tend to feel too strong or too weak. In terms of the type of person I like to play with, those who are competitive, but also equally in it just for fun!

2.5.4) Are you more interested in story or gameplay?

Depends. Usually gameplay, as that also tends to add replayability to a game. Stories have though, when they're good, kept me hooked for an entire game session though. Finding the story/gameplay balance is always difficult though.

2.5.5) Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.

Briefly, I remember playing it a month or so back. I remember playing a few missions and the tutorials and contemplating how balanced and strategy focused the game felt. It's admirable how well it does in that regard. When I find some free time when I'm not hacking, it's definitely on my play again list.

2.6) If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our repository (during the evaluation period or earlier) please state so.

Not as of yet, though I'm attempting to contribute and refine the NEEV library that is intended to replace ASIO in the UMCD. This seems like a more productive use of time, as much as anything.

3) Communication skills

3.1) Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.

Perfectly fluent? I'm a native English speaker, and I love creative writing! Hopefully this questionnaire as well as IRC interactions give a good feel for this.

3.2) What spoken languages are you fluent in?

English... that's about it. I know a little Spanish and French, technically? Oh, I also speak Python.

3.3) Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.

My first role on SS13 was an administrator. It was that path that actually led me to coding. We've also had a number of DOS attacks from players that have been... disagreeable, as well. I'm used to dealing politely with that sort of person. Occasionally, there's a genuine issue behind what they're saying as well, so it's important to get that out.

3.4) Do you give constructive advice?

I do my best to! Though sometimes, I also feel the best advice is to not give much advice. People don't learn if you give them everything, though equally, they sometimes need advice to help them learn. I do my best, but it's a fine balance.

3.5) Do you receive advice well?

Again, I do my best! I like to think I do, and I always welcome ways to improve, though occasionally this does lead to (fun) debates.

3.6) Are you good at sorting useful criticisms from useless ones?

I'd like to think so, at any rate. I'm used to having non-coders try to criticise code or ways of working (For example, saying 'It's easy, it shouldn't take so long!' etc). I'm not sure how to elaborate more here, so onwards!

3.7) How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want

I prefer to dicuss changes as much as possible, though naturally it depends on the scale. If it's something I feel I can sort out myself and doesn't concern people much at all, I'm perfectly happy to go away and come back with a proof of concept or


4) Project

4.1) Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?

I selected the User Made Content Daemon! I want to focus on... all of it? I'm not sure. It's an interesting project that looks fun, but more under that for 'Why'

4.2) If you have invented your own project, please describe the project and the scope.

[Left Empty]

4.3) Why did you choose this project?

Two major reasons. One: It's a standalone project. It's not tied into Wesnoth itself, which means a much smaller code base to learn and more autonomy when contributing, as well as control/flexibility with where it goes. This really, really appeals to me.

Two: Networking. This is one of my primary interests in terms of Software, as well as overall software architecture (Modular, flexible designs, etc). I saw the opportunity to engage is a project that is significantly about network communication I jumped at it. It was for a game too! I love developing for games! As much because of the community as anything, I always find game communities to be passionate and interesting. The project here is certainly cool enough I'm only making a single application to focus on it!

4.4) Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".

Naturally. Wait, hey, this isn't a question! It'll be below, along with other project details.

4.5) Include as much technical detail about your implementation as you can

This will also be below. It's naturally deserving of its own section!

4.6) What do you expect to gain from this project?

A fully working UMCD. Experience. An epic time. Maybe not in that order. Maybe 'A fully working UMCD' seems like a weird thing to put, but I always gain satisfaction from a cool, working hack. The UMCD fits that category.

4.7) What would make you stay in the Wesnoth community after the conclusion of SOC?

Firstly, and this is important to me, that lack of a 'need' to contribute. I like to be able to choose when and where I do, and I think Wesnoth offers that. Secondly, if I don't finish the UMCD. Maybe finish is the wrong word, but I hate incomplete things. I like to have things functioning, and functioning well. Thirdly, the people. If I get to know awesome people here, naturally I'll want to stick around!


5) Practical considerations

5.1) Are you familiar with any of the following tools or languages?

Git (used for all commits)
Git is amazing. Yeah, I used it all the time, and have all my University work under Git version control.

I also use it on all my open source hacks.

C++ (language used for all the normal source code)
I've never done a large project with C++, but I know the language and I intend of adding example code and practicing between now and summer of code.
STL, Boost, Sdl (C++ libraries used by Wesnoth)
I'm fairly familiar with the STL, certainly with abstract data structures in general. I know a bit about Boost Asio (Though this is a focus of mine with respect to working with C++ at the moment). I'm not familiar with Sdl at all.
Python (optional, mainly used for tools)
I'm fluent in Python, it's my main Hackathon language. And such a nice language to write in too.
build environments (eg cmake/scons)
I know a bit about cmake, enough to make basic make files. This is again something I'm actively working on improving at the moment. Scons I know little to nothing of.
WML (the wesnoth specific scenario language)
I know nothing of WML really, except that from the examples I've seen it looks like XML with square brackets, or at least very similar.
Lua (used in combination with WML to create scenarios)
Fluent in Lua, at least I was last time I checked. Haven't written in it in a while, but it was also the first language I ever learnt!

5.2) Which tools do you normally use for development? Why do you use them? Git firstly. Its flexibility with respect to branching and checking out different commits, as well as its ability to quickly pull and push to other repositories make it awesome. Also the fact it has a modular client!

Vim is my go-to text editor for smaller projects, with a number of plugins for Syntax checking, etc. I can't do without Vim keybindings, they make navigating the code so much easier and faster, as you never need to touch a mouse!

Usually however, for larger projects, I like an IDE for code completion and real-time error checking. When I start a new project, there's usually a small internal debate on using an IDE vs Vim. I usually settle for an IDE with a vim keybindings plugin. Hence IntelliJ or PyCharm tend to be gotos, as they have an easy to install but powerful vim plugin.

If Linux is a tool, I always use Linux, as command-line linux is incredibly flexible and powerful.. plus free software!

Otherwise, it really does depend on the project.

5.3) What programming languages are you fluent in?

I'm fluent in Python, Java and Lua. Python and Lua tend to be my gotos for most projects. Also DreamMaker, but it's BYOND specific. I'm competant in C++, C, HTML, Javascript and Awk, though I don't use them often. I have a knowledge of D (Which I love, I just need a project to use it on!), Bash and probably a couple of others I've forgotten.

5.4) Would you mind talking with your mentor on telephone / internet phone? Not at all, I'm perfectly happy talking over phone. IP is preferred though, as I like to have my hands free to type.