Zaroth Multiplayer Improvements

This page is related to Summer of Code 2011

See the list of Summer of Code 2011 Ideas

Introduction

This is a page containing details on implementation of Multiplayer engine improvements by Zaroth.

IRC

zaroth

Questionnary (part regarding project)

4.1) Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?

I chose SoC_Ideas_Multiplayer_Improvements_2011

Description of my proposal on how to tackle this project can be found here: [1]

4.2) If you have invented your own project, please describe the project and the scope.

Not invented, but description can be found here: Planned functionality

4.3) Why did you choose this project?

Since I'm an eager player of Wesnoth multiplayer games, it lies within my best interests to make it as good and enjoyable as possible - and several features are certainly missing. It also allows me to work on network code and networking is one of my main interests in programming.

4.4) Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".

I would like to be able to start and finish coding a little earlier, due to the heavy-exam time on my university in August. However, to compensate that, I will be able to easily devote a lot of time (and love) to the project during May, June, and first half of July. By the end of 3rd week of July I'd like to have all the planned functionality coded and (mostly) tested. So during the second half of July the only thing left is detailed testing, bugfixing and preparing for my exams.

4.5) Include as much technical detail about your implementation as you can.

Implementation details as well as planned functionality and my overall vision can be found here: Zaroth Multiplayer Improvements

4.6) What do you expect to gain from this project?

I expect to learn how a game networking code can be written and see a working example of that.

Last, but definitely not least is a lot of first-rate contact with experienced developers - I hope to pick up a lot of good programming practices from that! I'll certainly be also looking a lot through existing, (hopefully) good code, so it'll be a teaching experience in many ways.

4.7) What would make you stay in the Wesnoth community after the conclusion of SOC?

A better question is what would make me not stay! I am already an eager, even if not very successful, ladder player and I keep meeting friendly and interesting people in Wesnoth community - on the game server, forum and IRC. I see no reasons not to stay so far...

Timeline

As I learn and plan more on the code I'm going to improve, parts of the timeline may be altered/broken down into more details later.

+	Mar 23 - Mar 31	Familiarize myself with important parts of current Wesnoth multiplayer code and Wesnoth way of doing things. Bug Crab_ and fendrin on IRC for tasks to do. Submit patches.
+	Apr 1 - Apr 15	Discuss implementation ideas and details. Try tackling some bigger tasks, either hacking or producing mainline patches. Gain S­­V­­N commit access. Bug Crab_, fendrin and mordante even more ;-)
+	Apr 16 - Apr 25	Easter break. I won't be available during this time.
now	Apr 26 - May 22	Though the mentioned period starts before 23rd of May, which is Google "start coding" date, I want to start to implement parts of my HIGH prioritized ideas. This is due to fact that it's beginning of semester for me (so a more laid-back time) and I hope to make up then for my limited availability in the end of July and whole August (exam preparation/exam writing time for me). Working on tasks from here, prior to milestone 1.
-	May 22	Milestone 1: full functionality of [campaign] is provided in [scenario_metadata], all functionality of [multiplayer], [tutorial] and [test] is ported to [scenario]
-	May 23 - June 19	Working on tasks from here, tasks after milestone 1, but before milestone 2.
-	June 19	Milestone 2: All mainline content is ported to the new syntax and therefore my code can receive heavy testing.
-	June 20 - June 26	Buffer week. Time to release a binary to testers, implement some lower priority tasks, catch up to schedule, fix bugs.
-	June 27 - July 3	Coding surrendering idea. More details may be added later.
-	July 3	Milestone 3: Surrender is working (and roughly tested) for 1v1 games. This can be done by simply losing the surrendering team by now.
-	June 4 - July 17	Coding surrendering idea. More details may be added later.
-	July 17	Milestone 4: Surrender is working for any amount of players. Responsibilities after surrendering are correctly passed within teams.
-	July 18 - July 24	Another buffer week. In this time I won't be too busy with my exam preparation yet, so I will be able to fix more bugs and generally devote more time than later.
-	July 25 - August 14	In this time I will prepare and later have my exams. I may be able to program a bit, but all the critical parts should be in and quite tested by now. Time to test the main idea code, bugfix it if necessary and maybe do some of the lower priority tasks.
-	August 15	"'pencils down' date. Take a week to scrub code, write tests, improve documentation, etc."
-	Afterwards	- continue to maintain my code and try to implement my low/medium priority ideas I didn't manage to code during GSoC

Current issues

Here I'll put comments regarding stuff I'm currently working on, so one doesn't have to check the whole document for changes every time.

Get to know the current code and its problems

Discussion from IRC:

<zaroth> which of the pre-gsoc ideas of the page would you recommend as the first task?
<Crab_> compile wesnoth, find code which deals with multiplayer game creation, and add MULTIPLE debug
  statements to it so you will be able to see what is going on when the game is created.
<Crab_> I suggest using quite heavy logging, involving some new log domains, like "mp/create/foo",
  "mp/create/bar"
<Crab_> so, you'll be able to use --log-debug='mp/create/*' to get all them running at debug level
<fendrin> And learn about your goal. Have a look how difficult levels are handled in singleplayer
  and see the dirty hack that LoW uses to emulate it.
<Crab_> then, launch 2+ wesnothes (can be on same host, can be on different hosts), and see how they
  interact and what they do send to each other when a game is created and sides are selected
<Crab_> basically, I want you to understand the code and add a lot of log info to it, so we'll be able
  to change it and find issues without having to debug
<Crab_> then, submit the logging as a patch
<Crab_> and by 'heavy' logging I mean really verbose things, including logging the entire scenario
  configs to stdout, if explicitly asked for in the command-line switches.
<zaroth> thanks for the advice/assignment! it certainly doesn't sound too hard right now :-)
<Crab_> that's just the beginning, but the code will be familiar afterwards
<Crab_> just to make sure, we're interested in things that start after someone selects a
  'create game' button and that end when the game is started and everyone receives the scenario config.

Summed up above:

• create log domain for MP create window and make use of it
• learn the current LoW hack
• familiarize with code
• "the code" in question is everything from "create game" buton till "every MP side got their scenario WML"

Already done in this direction:

• got some knowledge of MP code workflow as well as adding some debugging there and discovering some redundant code
• read Mordante's GUI2 design document
• programmed basic mp change control dialog (learned a lot about GUI2 thanks to this)

Probable next training (pre-GSoC) tasks

If any developer has any quick-to-code wishes not already listed in EasyCoding, I'd love to hear them, even if they are not from the code areas I'm going to work in.

WML utility function to get vector of sides from any tag

As described in this IRC log:

<Crab_> code a function which will take a WML tag, and return a vector of integers
  which are determined by the contents of side= attribute in that tag
<Crab_> [foo]side=1[/foo]              1
<Crab_> [foo]side=1,2[/foo]              1,2
<Crab_> [foo]side=all[/foo]              1,2,3,4,...,6,7,8       (all the sides in game)
<Crab_> wml tag is represented by config
<Crab_> put that function in game_events.cpp
<Crab_> it will be used in many tags to allow them to act in the same way ...
<Crab_> this is to allow most tags to specify not just side=1, but a list of sides
<Crab_> later, a support for side_filter would be added
<Crab_> that is, if the cfg contains [filter_side] subtag, then take stuff from side= in tag,
  and AND it with the list of sides given by side filter
<Crab_> where empty side= and empty [filter_side] match all
<Crab_> but if they are both empty, a deprecation warning is produced

Summed up above:

• program a C++ function for getting vector of sides from WML
• it is needed in order to allow similar behavior for many WML tags
• [filter_side] is planned, which will make use of it

Improvement of my previous MP campaign difficulty hack

<Crab_> one of good things to do is to modify the map selection dialog to show scenario-specific options
<zaroth> and allow adding them?
<Crab_> yes.
<Crab_> we want to have a special section, i.e. [scenario_information], outside the #ifdef .... #endif guard
<zaroth> by the way, should it be done via #defines or rather WML variables?
<Crab_> WML
<Crab_> so, when we load the scenario for the first time (without difficulty level), we get it from that section
<Crab_> and then we should dynamically add stuff to the GUI
<Crab_> support at least 2 types of stuff. i.e dropdowns and checkboxes
* zaroth has to brush up WML knowledge, then
<zaroth> I modified this section slightly: link
<zaroth> I also stated my doubt about #define/WML variable there
<Crab_> (snippet 1)
<Crab_> it can look like this
<Crab_> alternatively, in a few years, it can be (snippet 2)
<zaroth> should it be within [scenario]?
<zaroth> or like [campaign], outside?
<Crab_> outside
<Crab_> we want to load it before we know the diff level
<zaroth> will be fun :-)
<zaroth> I'll probably extend my existing hack then
<Crab_> yes, it is an extension of your previous hack

Snippet 1:

[scenario_information]
  [parameter]
    type=checkbox
    name="Enable Fun Mode"
    to_variable=fun_mode
  [/parameter]
  [parameter]
    type=dropdown
    name="Attack time"
    to_variable=attack_time
    [option]
      name="Night"
      value="night"
    [/option]
    [option]
      name="Day"
      value="day"
    [/option]
  [/parameter]
[/scenario_information]

Snippet 2:

[scenario_information]
  [lua_code_to_create_the_gui]
     ...
  [/lua_code_to_create_the_gui]
[/scenario_information]

Summed up above:

• create a [scenario_metadata] section outside of [scenario], it will be preloaded
• don't preload the whole [scenario]
• allow defining scenario's own variables and sliders for game creation in the [scenario_metadata]

Planned functionality

Priorities explanation:

• HIGH means basically the same as "must have", will be definitely done during the GSoC
• MEDIUM means that I plan on implementing this feature during the GSoC and will do it unless serious difficulties occur
• LOW is the same as "nice to have", I'll get to implementing these if I get some spare time in the end. However, they're still tasks I'm interested in and I'll probably get around to doing them after the GSoC (I don't have any plans of leaving this community soon).

Better integration of MP campaigns

Priority: HIGH

Described in detail in the original idea. Since it's heavily dependent on SP and MP engines unification, I'll describe them together here.

Planned features

• merge the classes in *_controller.cpp intelligently, the goal is to make SP a special case of MP
• merge [scenario], [multiplayer] and [campaign] tags ( [scenario] is already almost the same as [multiplayer], and [campaign] can be done by e.g. [metadata] section in [scenario]).
  • The reasoning for this change is the fact that a single scenario can be seen simply as a 1-scenario campaign and a single player scenario is an MP scenario for 1 person.
  • It would also allow showing the scenario settings screen between regular campaign scenarios. It would certainly come in handy for campaigns that already allow controlling of allies by a player or by an AI. (and a must_be_human=yes attribute in [side] would prevent the player for setting an AI to control the main campaign side ;-) - but since one can always droid his side anyway, I don't know if it's needed ). Also, a regular campaign scenario (which suddenly introduces a 1-scenario ally or something alike) could be saved at that moment and played together with a friend on the MP server. Possibilities are plenty.
• allow choosing of difficulty in MP campaigns just as it is done in SP
• allow scenario writers to define their own sliders/controls to be shown in the "create game" screen, with WML-defined values (e.g. slider "Earthquake frequency" with possible values "none", "rarely", "often", "very often") (Priority: MEDIUM)
  • should it be done through preprocessor defines (as currently with difficulty) or WML variables? I'm bending toward defines, but it's open for discussion.

Implementation

In my vision, there should be not an unification through merging SP and MP code. Instead, I plan on improving the MP code to the point where SP code is no longer needed. (Obviously, current SP code will provide me with many precious examples on How Things Can Be Done™ and plentiful opportunities for Copypasta©) Then, somewhere after

Implementation of this task, in order to allow gracious transition for Wesnoth, should be split into following steps:

1. Adding [scenario_metadata] tag. See ScenarioMetadataWML for more details.
2. Support everything what [campaign] currently does with this tag.
3. Provide support for [scenario] in multiplayer (very easy and kind of symbolic only).
4. Develop a working example of a simple MP campaign. This will be probably done in parallel to all above tasks, but finished afterwards. In order not to break the poor LoW on the way, a new test campaign will be made. It will probably be hidden by debug_only=yes (see ScenarioMetadataWML for details) to avoid scarring innocent.
   • Milestone 1: full functionality of [campaign] is provided in [scenario_metadata], all functionality of [multiplayer], [tutorial] and [test] is ported to [scenario]
   • Please note that since above changes are unlikely to break anything working currently (at no point support for what was working is dropped), they can happen before GSoC even if a pre-GSoC Khalifate-release is planned.
5. Change all mainline MP maps from [multiplayer] to [scenario]. Continue to support both, but produce depreciation warning whenever the old tags are used (not to break UMC)
6. Write a tool that will allow easy transition from old-style SP campaigns to new-style SP campaigns. If such a tool will prove to be not feasible and trivial to write during GSoC, at least automatize menial tasks (such as changing all the scenarios to the new syntax and adding some standard [scenario_metadata] headers to them, if it will be needed)
7. Port all mainline campaigns to the new syntax and make them show up in MP creation screen. By default, user installing Wesnoth should see nothing but UMC campaigns using old syntax after pressing Campaign button in main menu at this point.
8. Port LoW to the new syntax. This will probably will be done by hand, since LoW is the only mainline MP campaign now and heavily hacked.
9. Produce depreciation warnings whenever [campaign] is detected.
10. If I'm ahead on time, try segregating SP campaigns from MP content in the creation screen. If not, drop it, pretty UIs can be done later.
    • Milestone 2: All mainline content is ported to the new syntax and therefore my code can receive heavy testing, much heavier than I could ever do with my simple test campaign. A release for wider audience would be nice, with important note that things will break in this release and stability of regular development releases will not be maintained. If a release prove to be not possible during GSoC, spend at least one week testing everything and bugfixing. As for the old campaign code, it will still be there, so UMC authors can see their campaigns in the old campaign choice window until they decide to port to new syntax.
11. (if time allows) add cool new things to the [scenario_metadata] tag, such as allowing defining in WML your own sliders/comboboxes in the game creation screen.
12. (after GSoC) Following the regular Wesnoth depreciation schedule, drop support for [campaign] in x next release and unwire the old code from the old main menu. Create a fancy GUI for new MP creation screen, instead of GUI hacks that will probably be used during the summer (I put the engine functionality first).
    • even if I happen to leave the team after the GSoC (which, as can be read in my questionnaire, I don't plan on), it should be easy to unwire the old code for another developer. That's my argument for allowing UMC authors who are using dev releases (as opposed to S­­V­­N trunk) a gracious transition by leaving the old code and campaign dialog there in the release after GSoC.

Difficulties

Summoning YogiHH from his slumber (46 days not on IRC before that discussion) proved to be very, very instructive. He opened my eyes to the few problems I haven't expected before. For interested, full log below:

<zaroth> since already kind of heated discussion is happening in the channel right now (about
  spritesheets), I think it may be better to speak here
<zaroth> I will post important/relevant parts of the log to my wiki page anyway :-)
<YogiHH> ok
<zaroth> did you read some of my ideas?
<YogiHH> yep
<zaroth> http://forums.wesnoth.org/viewtopic.php?f=8&t=33540#p485238 at this post today I tried to
  explain to some UMC authors who were interested what the changes are
<zaroth> this made me actually rethink and organize everything and I realized that not everything 
  in the "new loading flow" is coherent yet
<zaroth> but I'd like to hear your comments about my approach, if you have any
<YogiHH> ok, i will look at that shortly. Let me just tell some general first.
<YogiHH> without wanting to discourage you, i fear that what you set up as high priority is probably
  more than can be handled within GSoC
<YogiHH> which is not really much of a problem, because you can always start with one of the
  features and add more as time allows
<YogiHH> the problem is not so much the coding itself, it is the testing and debugging
<zaroth> I reduced some priorities recently ;-)
<zaroth> for example, I dropped the idea of redoing the MP creation dialog to GUI2 during GSoC
<YogiHH> ah, i see
<zaroth> after my recent experience with trying to get a small gui2 dialog I know what kind of time
  is needed to get a bugless quality with these things
<YogiHH> hehe
<zaroth> it can be done after GSoC, however
<zaroth> so my current plan involves doing actually two main tasks
<YogiHH> i will give you an example: I literally spent days if not weeks just to test scenario
  carry-over for campaigns after implementing a small little change to correct a bug
<YogiHH> just getting recall lists right can be a nightmare in itself
<YogiHH> and of course it behaves different in single player and mp campaigns atm
<zaroth> moving all [campaign] content to the [scenario_metadata] and testing it so it works kind 
  of right
<zaroth> ... what?
* zaroth starts to become scared
* zaroth starts to listen more closely, as well
<zaroth> what kind of problems did you experience with recalls and why?
<zaroth> could you give some background?
<zaroth> second is introducing surrendering to the MP games
<zaroth> although it can be kind of tricky, I think it can be done in ~3-4 weeks
<YogiHH> there used to be like 3 or four different places in the code, that handle loading and
  saving of recall lists (might be different meanwhile, but i don't think so).
<YogiHH> it makes a difference, if you start a campaign from scratch or load a savegame, because
  it is handled by different code
<zaroth> oh my...
* zaroth forgot savegames
<YogiHH> it makes a difference if you play the first scenario, which is more or less completely
  set up by the scenario file or if you play the second, which involves carry-over of information
<zaroth> where is the information carried over?
<zaroth> in the header?
* zaroth goes to grab a campaign save
<zaroth> all right, maybe not too many things at once
<YogiHH> it makes a difference if you play the second scenario and load a savegame instead of 
  starting in the first and entering the second from there, because carry-over information might
  have been not saved correctly
<YogiHH> and of course sp and mp are different, too, atm
<zaroth> do you think it's a sound idea to drop developing sp code at all and only improve and
  clean mp code?
<zaroth> (taking example from sp code, if needed)
<YogiHH> no, carry-over information is essentially recall lists and money, which is to be found
  in the savegame
<YogiHH> drop sp / clean mp: excellent question and i am not completely sure about that
<zaroth> that's been my plan so far
<YogiHH> what i can tell you is, that the original idea of developing the controller-classes
  was to separate single player from multiplayer (they used to be in a monstrous 3000 LOC
  function that included everything altogether)
<zaroth> but our conversation so far did quite a lot to undermine my confidence ;-)
<YogiHH> :)
<YogiHH> that's why there is a inheritance hierarchy:
<YogiHH> the base classes job is to provide the general infrastructure and common resources
<zaroth> yeah, the inheritance bugged me a bit, because it seemed a bit unintuitive (what
  inherited of what) last time I looked, not sure what was that, though...
<YogiHH> the subclasses (like mp_controller) override the methods that they differ in
<YogiHH> and implement functionality, that they only use themselves
<YogiHH> like for example the clock for mp
<zaroth> that's what I noticed as well
<zaroth> that SP code doesn't have a lot to offer
<YogiHH> if you make SP a special case of MP, you will get lots of functionality that is
  probably not useful for SP
<zaroth> except difficulties and nice defines management
<YogiHH> i am not saying that this is the wrong way, just bringing that to your attention
<zaroth> well, that was kind of my point ;-)
<zaroth> for example I really want to allow the MP creation screen to appear between campaign
  scenarios
<zaroth> to change the difficulty, for example
<zaroth> or to invite a friend, if in this scenario there is an ally
<zaroth> which can be user controlled
<zaroth> the MP creation screen (from local games) can be actually quite useful in the SP
<zaroth> especially if it gets some more functionality
<YogiHH> i think inviting friends is a very good idea
<zaroth> that's why I made a forum topic, partly to encourage UMC authors already thinking
  about it
<zaroth> (you may find a bit of zealous MP campaign fun advocating in my post, feel warned ;-) )
<YogiHH> my guts tell me that changing the difficulty level might bring about a gazillion
  problems. I can't put my finger on it, i just don't have a good feeling about this because it
  changes an existing architectural paradigm.
* zaroth feels that he needs to start early
<zaroth> and be ready to scrap his ideas
<zaroth> by the way, I wanted to bring to your attention these two links:
  Zaroth_Multiplayer_Improvements#Implementation
<zaroth> and the topic I linked above
<zaroth> since I wrote them most recently, they are the most coherent and complete things I
  wrote so far probably about my idea
<YogiHH> ok
<zaroth> there is also one issue that came into my mind today during lunch and it occured I
  didn't think about it enough... caching
<zaroth> but let's talk about it after you finished reading :-)
* zaroth heads off for ~10 mins
<YogiHH> yup
<YogiHH> alright, i sent you an email with some documentation. Will have a look at the links now.
<zaroth> I will have a look at the UMLs in turn then 
<YogiHH> regarding Anonymissimus "preprocessor scope": I think he meant having own defines for
  mp campaigns instead of putting it altogether into one giant "multiplayer" define atm. But you
  might better consult him directly.
<zaroth> well, of course I'd do that ;-)
<YogiHH> Apart from that i don't have any objections. You might want to ask fendrin/fabi however 
  about this, he is deeper into that kind of stuff than i am.
<zaroth> I even considered idea of having every scenario like that: [metadata] #ifdef 
  scenario-specific guard [scenario] #endif
<zaroth> and have all the info needed to display a scenario in the list in the metadata
<zaroth> well, if you don't have any objections, can you tell me what flaws / big problems
  you can see?
<zaroth> apart from fighting with getting savegames work correctly
<YogiHH> sounds good to me. As i said, though, i haven't dealt with WML much so far, except from
  the code side of view. So i can't help you much with that.
<YogiHH> But let me check something else about sp and mp differences...
<zaroth> well, I didn't ask you to get WML comments ;-)
<zaroth> I wanted also you to confirm that my flow of what happens after another is more or
  less right
<zaroth> and it would certainly save me some research if you could put it shortly what's the
  difference between these special beginning-of-scenario saves
<zaroth> and all others
<YogiHH> i guess so. I noticed that the scenario parameters must be independent of the difficulty
  level as they are set before choosing the difficulty but i don't think that will be a problem?
<zaroth> I also don't think so, especially since difficulty is preprocessor based and the
  parameters are going to be WML-based
<zaroth> so let's sum up the big problems you see: 1. getting savegames right
<zaroth> 2. danger of running into one-big-class-that-does-all once again
<zaroth> 3. since everything will be mixed now, avoid from the loading of wesnoth becoming 
  totally slow... getting preprocessor caching right therefore
<zaroth> 4. dropping sp may be not a good idea - could you state reasons?
<YogiHH> 5. changing difficulty level in between
<zaroth> right, you also said that, because it's an architecture fundamental
<zaroth> that's actually quite easy to test for me already - I have written a hack for MP,
  which is one-scenario so far
<zaroth> but doesn't parse the whole thing before, just only after the difficulty is chosen 
  and the campaign as well
<zaroth> for me dropping sp code actually will cause to reduce number of places in code where
  things may be done differently
<zaroth> ergo simplify the whole thing
<zaroth> but I may be overlooking something
<YogiHH> sp dropping: this is a rather architectural issue. What made me create those different
  classes was a whole lot of "if (sp){...}" and "if (mp){...}" stuff. I think making sp
  a special case of mp will introduce that again.
<zaroth> could you give an example?
<zaroth> (of where such if could be needed)
<YogiHH> ugh, that's like 5 years ago :)
<YogiHH> i will see if i can find something to illustrate
<YogiHH> but not tonight
<zaroth> because when I thought about it, I couldn't think of a single piece of MP code that
  really would block something in SP
<zaroth> and couldn't be disabled by a attribute= in [scenario_metadata] tag by scenario author
<zaroth> instead of hardcoding if(sp)
<zaroth> providing such an example will certainly make me think again about it :-)
<YogiHH> if on the other hand you make sp and mp behave the same, you should be able to move
  that part of the code to play_controller
<YogiHH> i will try to set up an example
<YogiHH> how familiar are you with the savegame WML structure?
<zaroth> you could say I'm very beginner
<zaroth> I just looked at the first ~100 lines
<YogiHH> ok, here is a crash course
<zaroth> because that was what I needed for writing my MP support hack
<zaroth> and making sure it works the way I wanted
* zaroth sits down and listens
<YogiHH> at the beginning, there is general information like scenario name, difficulty level
 and stuff. This is needed for all saves.
* zaroth remembers that part
<YogiHH> then, if it exists, there is a variable block with WML variables
<YogiHH> then there is the [replay] section, which contains all the replay information.
<YogiHH> followed by the [snapshot] section, which contains information only needed on reload.
<zaroth> (whose idea it was to make every save contain a replay and why it was done so?)
<zaroth> (doesn't it increase the savegame size significantly?)
<YogiHH> 1. no idea 2. yes
<YogiHH> wait a minute, i think i forgot [replay_start], which contains the gamestate at the
  start of the scenario
<YogiHH> this is necessary to make replays work (hence the name)
<YogiHH> and then there is some statistic information
<zaroth> also, what's exactly in [snapshot] ? does it contain the whole [scenario] ?
<YogiHH> basically, yes. The whole gamestate including all units on the field, their state,
  recall lists, time of day etc
<YogiHH> and then at the end there is some statistic information
<zaroth> all right, thanks for the quick course
<YogiHH> the start-of-scenario save is special, as it is the only save that contains no
  snapshot and no replay_start and not replay
<YogiHH> as far as i know it is meant to be able to watch the scenario trailer again (which you
  wouldn't be able to do with a turn 1 save)
<zaroth> now for the interesting part - are the MP and SP saves different and what makes them so?
  player controlled section?
<zaroth> Ah! That actually is *so* simple! I thought some black magic were in these
  start-of-scenario saves
<zaroth> and it is *lack* of information that makes them work
<YogiHH> there are a couple of tags that only mp has (like the clock settings) and i think
  there are also a couple of tags that only sp uses (might be wrong meanwhile on that)
<YogiHH> but the basic savegame structure that i told you about is the same
<YogiHH> if i am not mistaken the mp_settings class adds another [multiplayer] block to the
  savegame, that sp doesn't have
<YogiHH> if there are sp informations that mp doesn't have, they should only be marginal, though
<YogiHH> sp and mp used to have different tags for player information ([side] and [player]),
  but i think euschn removed that in his GSoC project
<zaroth> nice!
<YogiHH> just one more thing: if you look at the documentation keep in mind, that parts of it
  may be outdated
<zaroth> the UML?
<zaroth> oh, and I wanted to ask but forgot - what are the blue comments for?
<zaroth> are these for your experimental fork?
<YogiHH> No, they originated from euschn's former GSoC project. But they are about 2 years old.
<YogiHH> and i wasn't all too active in between
<YogiHH> but they should be pretty accurate still
<zaroth> what was his project?
<YogiHH> SummerOfCodeProposal_Euschn
<YogiHH> i had been mentoring that project ;)
<YogiHH> you might find some useful information there, although it is rather peripheral to what
  you want to do
<zaroth> well, it'll alleviate at least one of my obstacles, i.e. savegame problems
<zaroth> if they occur

Summed up above:

• Main expected difficulties are:
  1. Getting savegames right (few weeks spent on one bug for such an experienced coder as YogiHH probably is sounds scary)
  2. Danger of running into one-big-class-that-does-all once again (as it was before *_controller.?pp classes)
  3. since everything (SP and MP content) will be mixed now, avoid the loading of wesnoth becoming totally slow... getting preprocessor caching right therefore
  4. dropping sp may be not a good idea (there may be some conflicts by MP with SP content I didn't foresee, YogiHH said he would try to come up with an example)
  5. changing difficulty level in between (because it's a fundamental in Wesnoth so far and changing these has never been easy)
• crash course on savegame structure (I won't sum that up, read it yourself if you want to learn ;-) )

Further notes from YogiHH:

<YogiHH> I had a look at the controller classes and it already is pretty much like single
  player is a special case of multiplayer.
<YogiHH> Basically playmp_controller inherits from playsingle_controller and adds its own
  stuff like the clock and network receiving / sending.
<YogiHH> There are still some redundancies between the two and there is potential for
  further merging. I'd say it is definitely worth a look.
<YogiHH> However, i wouldn't make that priority number one, as your basic idea is more or
  less there already.

Subtasks

• learn the current hack Legend of Wesmere MP version uses for introducing different difficulty levels (done)
• get acquainted with the *_controller.cpp files (as in playmp_controller) to get an idea of what has to be done in order to perform the required merge (partially done thanks to fendrin's guided code trip on IRC)
• Write a hack which loads the MP campaigns without the current ugliness (i.e. runs the preprocessor after choosing the difficulty level, not before, which can significantly decrease the loading time) (done)

MP "create game" dialog improvement

Priority: MEDIUM

Improve the MP create game dialog as stated in main idea topic. Explanation for medium priority: I will have to change this dialog a bit anyway when implementing the SP/MP unification high priority idea, but improving it (and especially GUI2 transition) isn't high priority - it will just "happen" on the way as it is needed.

Planned features

• possibly allow categorization of the scenarios to allow easier finding and selection? As in "Official MP scenarios" > "2 players" > "Caves of the basilisk". This would allow easier browsing among maps when a user has many add-ons installed. (priority: LOW)
• (your suggestion)

Subtasks

• introduce debug logging in these screens, so it's easier for a developer to recognize what's going on there (partially done, probably will add more debug statements as I go)
• convert the creation screens to GUI2
  • create a mockup to allow quickly communicating my ideas of any improvements
  • some GUI elements may be missing, so it probably will involve working with mordante in order to get the necessary widgets in
• possibility to randomize players assignment to sides as in this feature request and make it work over the network
• allow players switching sides after joining/hiding their choices

Issues

<Crab_> ok. so, the task is to hack/modify the MP creation screen (where you select
  the map to play) to use the same system
<Crab_> feel free to delete all MP maps except 1
<Crab_> then, modify that 1 map to use the same #ifdef   thing
<Crab_> so, when you parse it without a scenario-specific define, you'd only parse 
 the info you need for the map selection screen
<Crab_> it's ok if some parts of functionality are lost in the process, or if you 
 need to add a meta-info section like [scenario_meta_info] to store the extra stuff
  you need like the number of sides and the available difficulty levels, etc
<Crab_> then, make sure that after the map is selected, the user is asked to select
  a difficulty  level (or add a dropdown for that)
<Crab_> then, reparse the scenarios with the new define, getting the actual scenario
  data for the difficulty level selected
<Crab_> that's all
<Crab_> feel free to do it non-cleanly, it's only a proof-of-concept
<Crab_> I need to see if you'll be able to hack your way through the code to get the 
 result you want to happen
<Crab_> ask questions, if necessary
<Crab_> that's all )
<zaroth> if I understood correctly, main goal is to avoid preloading everything?
<Crab_> yes
<Crab_> only load everything once you know the difficulty level
<Crab_> by the way, by doing that task, you'd add support for difficulty levels for MP )

Summed up above:

• hack MP creation screen to allow choosing difficulty
• don't preload all MP campaign maps before a difficulty and scenario are chosen
• use the existing campaign defines

Surrendering

Priority: HIGH

Add a possibility to "properly" resign/surrender a game and make it work over network. I didn't consider this a high priority task so far, since no developers expressed much of interest about it. But after getting feedback from Noy, I plan on looking in depth into this problem. Some of doubts/issues can be already seen described below in the IRC log and following lists.

<noy> it's a critical fix that would enable a lot of different functions to be added
<noy> it's just never been coded in properly
<noy> and thinking through the gameplay effects is kinda important.
<noy> one issue that I'm concerned about is what happens in a 2v2 where one player quits?
<zaroth> as far as I thought, when a side surrenders and still has teammates, his side is
  automatically passed to his teammate
<noy> okay, that sounds good
<zaroth> when the last player in the team surrenders, the whole team loses
<zaroth> and in case it's 2v2v2 is offered to be droided to host
<noy> that was the other one
<zaroth> (of course there can be a 1 player team ;-) )
<noy> the option there is also to have a pre-game box that might say sides get droided or
  are removed from the map
<noy> it's critical for a 2v2 ladder
<zaroth> mm... removing a team from the map seems like a huge change to a gameplay
<zaroth> there was so far nothing that did total extermination
<noy> well you could just make them neutral third side

Summed up above:

• if a side in 1v1 resigns, it's an automatic win for the other side
  • maybe add an "automatically resign quitting players" option in MP creation dialog? Possibly even with a customizable timer to provide a buffer time to reconnect in case of connection problems. However, for it to be dependable, the timer should be server-sided.
• if a side in 2v2, 3v3, other team games resigns, its control should be automatically passed over to its teammate
  • note that only control of the side is changed, not the units side (as in blue units join red forces) - this would be a gameplay change I don't intend on here.
• if a team in 1v1v1, 2v2v2, FFA, etc. resigns, the host droids it automatically
  • as noy suggested, there could be an MP creation dialog option for the resigned team to disappear - however, this would be a huge gameplay change and isn't Wesnoth following OAB philosophy? I'm reluctant to implement this therefore, but if I hear more voices for it, I may do it

Potential problems

• What will happen if after surrendering a side in a more than 1v1 game (e.g. 1v1v1) an observer wants to take over this side?
  • In case the side was exterminated (noy's idea), it's obviously impossible.
  • But what should happen if it just became neutral (droided?) - in competetive gaming it doesn't feel right, but I think that allowing/disallowing that happening should be left to host's decision.
  • And what about its (also resigned) teammate, if it was 2v2v2? - my opinion is that it should be also passed to the observer taking over and automatically droided.

Subtasks

• Analyze current Wesnoth server code, see how client quits are handled and if any changes will be needed here
• Implement a working 1v1 surrender and build up from there

Control panel

Priority: MEDIUM

Instead of host having only command line interface to :control and :droid, I plan on coding a control panel window, user-friendly and GUI-accessible. A mockup made quickly in Qt Designer (with the player seeing this dialog (Zaroth) as host of the game, that's why not any of the "give control" comboboxes are not greyed out):

I feel that sides with attribute hidden=yes shouldn't be shown in this dialog as well, since one doesn't want to give a GUI to allow changing controller of statues on e.g. Caves of the Basilisk (it'd be just confusing).

Separation of hiding a side in status table and control panel

It could be also nice to allow hiding a side from control panel, but showing it in the status table. It would come in handy for e.g. hiding pure AI sides in survival games or any other MP games against the AI (AFAIK the AI sides are shown in the status tables of the majority of current MP add-ons/scenarios)

(in my opinion it's not very useful, but if you strongly feel that it's needed, please express this in a comment, this will help me with decision on whether to bother or not)

Planned features

The mockup should be quite self-explanatory. If not, let me know, this means I failed at UI designing and have to make it more intuitive ;-)

More features that are not in the mockup:

• as ancestral suggested on IRC, GUI for muting/unmuting players would be also nice
• generally, hosts could use more sophisticated GUI than simple players, it may be worthwhile to rethink the GUI idea (for example, instead Droid checkbox put a button [actions] and after it, a popup appears - kick, ban, mute, unmute...)

Subtasks

• learn basics of GUI2 (done, thanks to the mordante's GUI2 document)
• implement a simple working control panel with UI based on gamestate_inspector (done)
  • abstract some of the control giving code, since as for now it's pretty tightly tied to the console commands
• find a good way to determine AI sides' nick ownership, to allow proper showing of the AI owner in the nick table (it's not so easy to do with current infrastructure)
• get necessary widgets ready (as mordante said, e.g. GUI2 combobox isn't programmed yet)
• since it'll be quite similar to the current status table, I may also convert the current status table to GUI2 (priority: LOW)

Various others

These are small more or less random ideas I'd like to do. In case I get really tired with the main tasks, I can have a small break while working on these. Also, if I get any free time after finishing my main tasks (which may be unlikely), I can fill it with working on these as well.

Changing difficulty mid-campaign

Priority: LOW

Add a possibility to change a difficulty mid-campaign as described in this topic (and make it work over the network ;-) )

Avoid seeing each other's views when playing hotseat

Priority: LOW

Improve hotseat gaming as in this feature request (prevent unwilling cheating by seeing each other fogs of war cleared areas)

It could also involve fixing naming of local players, since it's currently disabled.

Remove statues' turns from replays by introducing a skip_replay attribute

Priority: LOW (but I'll try to do it before GSoC to get some more coding practice)

As you may have noticed, navigating replays using "Play side turn" isn't too pleasant on e.g. Sulla's Ruins and Caves of the Basilisk. Once per turn you get to see the statues' moves, which are not too interesting. This causes confusion and requires additional clicking.

To solve this, I propose introducing a skip_replay= attribute (default: no) in the [side] tag. Sides with this attribute set to "yes" would have their units moved instantly in replays (as in "Quick replays" option). It would also disable possibility to pause at this side's turn, the next side would be played and paused at instead.