SoC2012 Ivan Addon Server

This page is related to Summer of Code 2012

See the list of Summer of Code 2012 Ideas

This is a Summer of Code 2012 student page

Description

Ivan Savenko - Addon Server

The goal of this project is to write a new add-on server for Wesnoth from scratch. Original draft in S­­V­­N (latex): http://svn.gna.org/viewcvs/wesnoth/trunk/doc/design/

Will be written in C++, networking part - boost::asio. From existing codebase I'm going to use WML parser (class "config") and forum integration (class "fuh" aka "forum user handler")

IRC

IvanSav

SoC Application

Submitted to google

Detailed proposition

Storage

Main file - umcd.cfg. Read-only, should be passed as 1st parameter on launch:

[settings]
	port = 15002
	threads_count = 4
	storage_path = /var/umcd-1.10/
	...
[/settings]

(Optional) this file may have multiple sections for each major version of Wesnoth. In this case umcd will automatically start separate copy for each section in this file.

All umcd files will be locate in storage_path:

• storage.cfg - stores current state of server (like config of all uploaded mods)
• archives/ - directory with archives of all add-ons (similar to http://files.wesnoth.org/addons/1.9/)
• unpacked/ - unpacked version of mods for validation and wescamp integration (similar to http://files.wesnoth.org/addons/1.9/unpacked/)

PBL specification

• Old directory structure won't be supported - _addon.pbl and _main.cfg must be placed in add-on subdirectory (only 5 add-ons on 1.9 server still use it)
• Add to PBL field "status". Set by author (Complete, Testing, Incomplete, Unusable). User should be able to see this field and select addons with required level of completeness. (Optional) Allow uploading several add-ons with same name but different status so add-on may have "stable" releases along with "development" ones.
  • Complete - addon is complete, has no major bugs.
  • Testing - addon is complete but have some bugs.
  • Incomplete - addon is still work-in-progrees.
  • Unusable - add-on is for testing purposes and should not be installed by regular user.
  • This field can be set by server to "Broken" in case of any issues like missing dependencies.

• Field "uploader" - will be replaced by "uploaders". This is list of valid usernames who can access this add-on.
• Field "authors" - will be displayed as it, no checks for usernames.

Users authentication

This will remove any need to store passphrazes or e-mail in PBL and will make contacting author by administration easier (PM on forum or e-mail).

Unregistered users - It should be possible to keep current passphrazes system which will be used if PBL have "passphraze" field intead of new "uploaders" Uploading by unregistered users should be selectable via server confug file. Reasoning: even basic e-mail confirmation (as proposed by shadowm) is almost identical what we have at forum.wesnoth.org but will require notable amount of work. Most of UMC authors have account anyway. I found ~20 UMC authors (out of ~200) with add-ons on 1.8 server that don't have account with exactly same username, most of which are typos or a bit different username.

Code-wise behaviour will be similar to multiplayer server. UI behaviour - after user uploaded correct PBL he will be asked for username\password (same dialog as in multiplayer server)

Add-ons updating

Server will keep history in this format:

[update]
	"uploader" = username of person who uploaded this version (if non-anonymous upload)
	"timestamp" = unix timestamp of upload
	"ip_address" = ip address of uploader
[/update]

Wesnoth uses separate server for each major version so history size won't be significant.

Wescamp

Proposed behaviour:

1) When add-on is uploaded\updated it will be automatically uploaded to github.com/wescamp if field in WML "translate" set to true. This will be done by external tool (wescamp.py, probably will be modified).

2) Another tool running in background will either

a) check every %TIME% for any updates on github

b) wait for any post receive hooks to trigger. This is better but more complex solution.

If there are any changes in .po files - compile them via msgfmt tool from gettext.

Connect to add-on server, login as admin and send WML message "update_translation" with new .mo files.

NOTE: interaction with with add-on server via network won't create any notable lag in case of communications with localhost and easies one to implement. Such solution won't remove possibility of uploading translations remotely.

Dependencies

format is mostly described in draft.

• "requires" - add-on must be installed before this one
• "conflicts" - only one add-on of these two can be installed.
• "recommends" - add-on have non-essential content that will be used by this one

Concurrency

Several days ago I've encontered interesting issue with current campaign server - it is possible to download add-on which is currently being updated by author. This resulted in several GUI issues and finally - crash (no stacktrace unfortunately). This means that new add-on server should support atomic updates to prevent such issues.

Client-side changes

I'll try to keep is as it during GSoC. Change in displayed columns may be the biggest change in UI (e.g I'm going to replace download counts with date of last update)

Timeline

Note: Most possibly I'll be free for 1-2 weeks in May - still don't have my exams schedule. Will shift timeline as soon as I will know it. Current "definitely free" date is May, 28

Summer of code timeline

Type	Time	Description
Docs	June, week 1-2	Finish protocol specification draft.
Server	June, week 1	Implement server-side storage.
Server	June, week 2-3	Compressing and sending files via network. Add-on upload\download support.
Server, Client	June, week 3-4	Integration with existing database from forums. Will try to re-use current login to multiplayer server dialog.
Server, Client	July, week 1	Add-on updating support. Finish any remaining tasks. SoC mid-term evaluation.
Server, Wescamp	July, week 2	Implement hooks on server, tool for uploading add-ons on github (based on wescamp.py)
Wescamp	July, week 3-4	Implement tool for receiving and compiling translations from github (based on wescamp.py)
Client	August, week 1	Dependencies support. Prerequisites support is already present in client.
General	August, week 2-3	Finish any remaining tasks. SoC "pencils down" date.

Possible post-SoC tasks

• Any tasks not finished during SoC

Server-side improvements

• wmlint support - nice to have but not reliable.

Client-side improvements

• Send add-on icons to client, probably with some caching. Reasoning: Currently some add-ons use icons from resources of not-yet-installed add-on. Estimated length: 1 week.
• Merge "Remove Add-on" dialog into "Add-ons Manager", add filter "Installed add-ons". Reasoning: not intuitive, for example only way to see installed add-ons is to open "Remove Add-on" dialog. Estimated length: 2+ weeks.
• Remove "Connect to Server" dialog. Reasoning: selecting add-on server is (completely?) unused. May be kept in configuration file or as command line parameter for those rare cases. Estimated length: 1 week.
• Improve "Add-ons Manager" dialog. Will describe this later.
• In-game bug reporting by users. Wesnoth will collect required data (Wesnoth version, OS, add-on version) along with savegame and will send it to add-on author via e-mail along with bug description by user. Needs discussion.

Questionnaire

Basics

Write a small introduction to yourself.

My name is Ivan Savenko, I'm from Ukraine, Chernihiv, really interested in programming and computers in general, I am an avid gamer which makes games development even more interesting for me.

State your preferred email address.

Check Google application - I'll post it there

If you have chosen a nick for IRC and Wesnoth forums, what is it?

IvanSav on both IRC and forums

Why do you want to participate in summer of code?

I really love coding and I would like to spend summer on something that will be used in real world in quite big community.

What are you studying, subject, level and school?

I study system programming at Chernihiv State Technological University, 4th year of study.

What country are you from, at what time are you most likely to be able to join IRC?

I'm from Ukraine (GMT+3), can be on IRC from ~11:00 GMT and till ~23:00 GMT.

Do you have other commitments for the summer period ?

I only have exams in May, will be free for all 3 summer months.

Experience

I've been working on several small apps for fun, one of my goals usually were speed, usage of new features like C++11, trying to implement some complex algorithms (like SDL game that uses only runtime-generated images).

What programs/software have you worked on before?

Apart from small university tasks I'm working on VCMI project.

Have you developed software in a team environment before?

I think I can say yes. I've been working in team on VCMI but apart from some discussions on forums work is usually separate.

Have you participated to the Google Summer of Code before?

Nope. This is first time and most possibly the last time I'm trying to take part in GSoC - will be too busy with graduation next year.

Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.

VCMI, open-source engine for Heroes of Might and Magic III. (vcmi.eu) I've joined this project ~2 years ago. It's quite small project which turned out to be really good way to learn more about C++ after studying it in the university. Apart from some minor stuff I've been working on bringing support for 32-bit graphics in originally 256-colors only game and JSON-based configuration system.

Gaming experience - Are you a gamer?

Definitely :)

What type of games?

Either strategy games (Heroes of M&M, Civilization) or RPG's (Baldur's Gate, The Elder Scrolls)

What type of opponents do you prefer?

I'm playing singleplayer only so I prefer AI. Smart one, if possible.

Are you more interested in story or gameplay?

Good story mostly or gameplay where you need to think at least from time to time.

Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.

Can't tell for how long but I've completed all campaigns that were in 1.8. Played single player games only.

If you have contributed any patches to Wesnoth, please list them below.

Proof-of-concept of add-on server: https://gna.org/patch/index.php?3241

Communication skills

Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.

Written English is probably good. Have no problems with understanding others and vice versa.

What spoken languages are you fluent in?

Russian and Ukrainian. Probably English, but I've never spoke with native English speakers so can't say for sure about this one.

Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.

Shouldn't be a problem.

Do you give constructive advice?

So far there were no complains regarding my advices so I think that yes.

Do you receive advice well?

If somebody is trying to help me then why should I complain? If he is right I may learn something useful after all.

Are you good at sorting useful criticisms from useless ones?

I hope so.

How autonomous are you when developing ?

I definitely can code autonomously but I'd rather discuss all points I'm not sure about before working on them. So in general it will be a mix of both. At first - discuss all basics, then make some small proof-of-concept and go back to discussion. It's much more easier for me to explain something with small snippets of code instead of trying to this in words.

Project

Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?

I've took it from the list, Addon server. Considering that this should work as daemon I should foxus on stability and keep it extendable to implement all optional features Wesnoth may need.

Why did you choose this project?

It's (mostly) a standalone app, that have to be fast and stable, should use boost::asio (which I would like to learn). Addon server is something that I may need sooner or later myself.

Include an estimated timeline for your work on the project.

See SoC2012_Ivan_Addon_Server#Timeline

If anything won't be finished by August then I'll continue to work on it after GSoC.

Include as much technical detail about your implementation as you can

See SoC2012_Ivan_Addon_Server#Description

What do you expect to gain from this project?

Experience making actively used application, some networking knowledge. Working addon server that I can use myself apart from using it in Wesnoth.

What would make you stay in the Wesnoth community after the conclusion of SOC?

Basically any work on this project in case if I won't finish something in time or in case of new feature request.

Practical considerations

Are you familiar with any of the following tools or languages?

• Sub­­version - basic commit\checkout via GUI
• C++ and STL - good I think. Have been actively using it for at least 2 years.
• Boost - I've used several libraries (filesystem, functions, algorithms...), but not asio
• SDL - more-less familiar with SDL 1.2
• build environments - used autotools, would like to learn cmake at some point
• WML - Nope.
• Python Lua - No experience with scripting languages, would like to learn Python though

Which tools do you normally use for development? Why do you use them?

• Linux (Ubuntu) - It may not be perfect but it's really easy to use for developing.
• C++ - Have some flaws but it is more-less portable and powerfull without notable speed loss.
• CodeLite - quite small C++ IDE that have all functionality I need.

What programming languages are you fluent in?

I think I'm good in C++, decent in C, know a bit about Java\C# and Python.

Would you mind talking with your mentor on telephone / internet phone?

OK. But my spoken English is most possibly awful so I hope that this won't be necessary :) Will post it in Google application.

Links

• WesCamp
• PblWML