User:Trademark/GSoC 2013/Addon Server: Create a new and shiny one

This page is related to Summer of Code 2013

See the list of Summer of Code 2013 Ideas

This is a Summer of Code 2013 student page

Caution

This proposal is under construction and can be modified any time.

Proposal — Addon Server

Description

This proposal aims to rewrite and improve the addon server with Boost.Asio. The current server becomes old and has few drawbacks:

• The code is monolithic, resumed in a single file which made it hard to extend and maintain.
• The responses are in english only, support for gettext should be added.
• The translation of the addons are not synchronized with Wescamp.
• Native handling of the different types of add-on (campaigns, factions, eras, …).

The hope is to design a new and robust server with the help of Boost.Asio, make it extensible with every layer of code clearly defined. Moreover the wire has been completely reviewed to integrate translatable features. It'll add meta-information on the different types of add-on.

Server configuration

Most of it is described in the patch #3848. You can also find a PDF version here. The following is a summary of the chapter "Server configuration" of the umcd.pdf document.

Options

The file umcd.cfg resumes the main characteristics of the server:

• Port: The TCP/IP port to listen to for incoming requests.
• Threads: The number of working threads to start.
• Dir: The directory where the add-on will be stored.

Wire

The wire has been extensively described in the patch #3848. You can find a PDF version of the wire at this link umcd.pdf. For documentation purpose, the following is a list of the major features.

Special packets

These packets inform the user that abnormal packet has been sent in the request. For example, it warns if a field is missing in the request.

• Warning packet
• Error packet

User request packets

These packets can be sent by users of Wesnoth having any of the following requests:

• Request the campaign list in a favorite language.
• Request UMC download in a specific language.
• Request UMC change password.
• Request a new password.
• Request UMC upload.
• Request UMC delete.
• Request license.

Admin request packets

Should be discussed with mordante before further analysis.

Architecture

Database

Discussion

An add-on database is required for this project. Here are the characteristics of the system we need:

• Storing an addon package along with meta-information.
• Create dependencies between package.
• Easy operation for adding and removing package.

Indeed, such a system has been well-studied and is known as package manager such as RPM or dpkg. They can help us to be aware of some problems such as the dependency hell, however they are system-oriented, and much more complicated than what we actually need.

Option 1: Flat file hierarchy

The flat file hierarchy would consist to store the add-ons and meta-information in one repositories per add-on.

Pros

• Simple.
• Would exactly fit our needs.

Cons

• We would reinvent the wheel.
• Doesn't respect the ACID properties.

Option 2: Relational database

The relational database would consist of a several tables:

• Add-on table: store information and meta-information on the add-on.
• Dependency table: keep relation between add-on.

It's sound weird to have only two tables and to store big file into a relational database however it would have the ACID properties.

Option 3: RDF database

This final option is to use a RDF database. There is many advantages to use such a database with our server:

• We want to store data (add-on) along with meta-information, RDF is mainly designed for it.
• We want to create relation between add-on such as the dependency relation, but also the upgrade relation (add-on 1.1 is younger than add-on 1.2).
• We can easily retrieve the dependencies because it's a graph database.
• Query RDF can easily be done with SPARQL.

The main disadvantage is that RDF is not as much known as relational database, and could be harder to maintain for new developers.

The tools used would be Soprano which is open-source and written in C/C++.

Interface

Firstly, some high levels choices must be made and are independent from the way data are actually stored. Given the wire, we have an idea of the operations on the database and thus we can establish the database interface. Note that all names can change.

 class AddonDatabase
 {
   public:
     virtual config request_campaign_list(const config& request) =0;
     virtual config request_campaign(const config& request) =0;
     virtual config request_password_update(const config& request)=0;
     virtual config request_new_password(const config& request)=0;
     virtual config request_campaign_upload(const config& request)=0;
     virtual config request_campaign_delete(const config& request)=0;
     virtual config request_license(const config& request)=0;
 };

Implementation details

Signal

The signals are handle via std::signal. The default behavior for every signal is:

• Notice all current tasks that a problem occurs and they must stop.
• If some tasks communicate with any user, send a packet describing the interruption.
• Waiting (with timed out) that all threads have been stopped.
• Restart the server.

Specific actions are applied for SIGTERM and SIGINT (in addition of the 3 first):

• Launch a maintenance server which respond to users with an informative message.
• Close the server.

For the other signals, we mail the server owner (in addition of the default behavior described above. If the server has restarted more than 3 times in half a hour, we do the same as with SIGTERM/SIGINT.

IRC

Trademark

Questionnaire

Basics

Information

Email : ptalbot [at] mopong [dot] net

Nickname : Trademark

Study : Master degree in computer science at the University of Lyon 1

Country : France

Availability : Mostly during the evening (4pm to 9pm UTC)

Write a small introduction to yourself.

My name is Pierre Talbot, I currently follow a Master Degree (first year) in computer science in Lyon (France). After being a hardcore gamer for few years during high school, I choose to study computer science. So I begun my studies in Belgium and after 2 years I joined the University of Lyon 1 in France. Now, more than anything I like to program and to learn more on the various fields of computer science.

Why do you want to participate in summer of code?

Because I already participated in the GSoC, I know that is an invaluable program to excel your programming level. Furthermore being paid to learn from experienced people is just an opportunity.

Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.

I will restart school 2 weeks before the ending. During these weeks I will work more during the week-end instead of balancing the works all over the week. But I'll try to finish most of the work before this date. I didn't planned any vacation but if I take some, it will be at most 3 days.

Experience

What programs/software have you worked on before?

I mainly worked on numerous school projects which involves various skills, the relevants one's for this project were:

• FTP-like server in C.
• Network game programming in Java and C.
• MVC web application (team of 4 persons).

I also worked on many C++ projects (data structures, Boost.Check, garbage collector, …) without a networking layer.

Have you developed software in a team environment before? (As opposed to hacking on something on your own)

Yes, the most of my school projects are in team of 2,3 or 4 students. We are using source versionning such as Mercurial, Git. Depending on the project, we use other team tools such as the ones provided by Redmine.

I used SVN during my previous GSoC.

Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?

I participated in 2011 as a student in the Boost organization. I worked on a new library that is not yet accepted into Boost, however I succeed the GSoC. Boost.Check is a library helping users to validate and create check digits of various numbers (such as credit card numbers, ISBN, IBAN, …). One of the concern is to design it to be easily expendable.

Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.

My long term support for an open-source project is Boost.Check. After nearly two years, I still work on it.

I also worked on a CGAL library during a two months research work in my university. I've upgraded the Combinatorial Map library to be compatible with Boost.Graph concepts and specific CGAL concepts. Now the Combinatorial map can work with generic algorithms that request these concepts (such as the Triangulated Surface Mesh Simplification algorithm).

Gaming experience - Are you a gamer?

Not so much since few years, mainly because the time is lacking.

What type of gamer are you?

I was a hardcore gamer during several years. I played a lot for the competition side. Now I mainly play during holidays or free times.

What type of games?

I played strategies games, FPS and MMORPG. I spent a hundreds of hours on Warcraft 3 and other hundreds on Guild Wars.

What type of opponents do you prefer?

I prefer the opponents that are stronger enough to play beautiful games. The one that can help me to excel my skills. However, sometimes I just play for fun with friends, and then, the opponents doesn't matters.

Are you more interested in story or gameplay?

I like both. Games with great story and poor gameplay are not interesting. I prefer a good gameplay than a great story, however I often expect a little more than just the gameplay.

Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.

I mainly played the solo game but not for a long time (few hours).

Patches

This year patch:

• Patch #3848.

2012 patches:

• Bug #19538.
• Patch #3236.
• Patch #3319.

Communication skills

Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.

I can easily understand people, technical documents and text in English. My writing is good enough to get understood by people. My overall level is medium.

What spoken languages are you fluent in?

My native language is the French. I can speak in English if I know well the speaking topic.

Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.

I know how gamers can be when the game doesn't fit their expectations. We will need to ask them feedback about the add-on server. I think I can handle the dissatisfied uploader, I'd try to find compromises and quick solutions.

Do you give constructive advice?

I'm helping on a french programming forum. I try to explain why and how-to instead of just giving a solution.

Do you receive advice well?

As a programmer, you must receive advices well because it's a field where ideas evolve really fast. Exchanging ideas and receiving/giving advices is a day-to-day task. However I prefer receiving advices from experienced people.

Are you good at sorting useful criticisms from useless ones?

The useless are often a kind of judgment and should be easy to sort out.

How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want

Most of the modern programming methodologies show that you can't just discuss and analyse. You must continue the analysis through out all the process, it requires a lot of refactoring. Resuming, a bit of analysis is always required, but you can't loose to much times on it, otherwise nothing will be never done.

Project

Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?

I choose the add-on server.

If you have invented your own project, please describe the project and the scope.

Not yet.

Why did you choose this project?

To learn the Boost.Asio library. I like to have the flexibility to code from scratch a server. I like to think about good software architecture, and this project helps me to do so.

Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".

Please, give a look at the timeline section.

Include as much technical detail about your implementation as you can

The proposal is described in the proposal section.

What do you expect to gain from this project?

Learning a new library, validate a home-made architecture on a world-wide use server. Of course, it's also to earn money from my passion.

What would make you stay in the Wesnoth community after the conclusion of SOC?

A successful project, I'd like to maintain the server after the SOC (or finish/polish it, if needed).

Practical considerations

Are you familiar with any of the following tools or languages?

• Sub­­version/Git: Good knowledge, I already used both for other projects (subversion for the Boost.Check project and git for all my team project).
• C++: Good knowledge even if I learn more each days on the idioms and features of this language.
• STL, Boost, Sdl: I used a lot the STL and Boost but not really the SDL.
• Python: I coded a SAT solver in Python, but I don't know much of this language.
• build environments (eg cmake/scons): Already used cmake, but I haven't a good knowledge of how it works.
• WML: I started to study it the last year for my previous proposal. I know pretty well the common rules.
• Lua: Never used.

Which tools do you normally use for development? Why do you use them?

I tried many tools such as Visual Studio, VIM, Eclipse,… But I'd say that my favorite is Sublime Text 2 for all the nice features it offers. However it's not an IDE, so the compilation/debugging must be made in the console. However the VS debugger is quite exceptional and I wouldn't be against using it only for this feature.

What programming languages are you fluent in?

The language I know the most is the C++. However I often programmed in C and Java too.

Would you mind talking with your mentor on telephone / internet phone?

Sure. I'm not used to speak with people in English but I think I can do it !