Difference between revisions of "SoC Ideas Multiplayer Data Analysis"

Latest revision as of 23:15, 12 March 2014

This page is related to Summer of Code 2014

See the list of Summer of Code 2014 Ideas

This is a Summer of Code 2014 Idea

Description

Improve the automated collection and display of multiplayer game data

Page for the idea: SoC_Ideas_Multiplayer_Data_Analysis

Multiplayer games are currently archived. This includes lots of interesting and potentially useful data, including recruits, deaths, number of turns and other things. The only way to access this information currently is parsing the WML one save file at a time. Additionally, which side(s) won the game is only recorded in some cases. We would like to record which side(s) won or lost (for as many games as possible) and then display useful information about these games for balancing and feedback purposes.

There is 1 submitted student proposal for this idea

Sachith Seneviratne - Multiplayer Data Analysis Proposal

Although Multiplayer games are being archived currently, no data is saved regarding the winner(s). Recording and presenting this information would allow for easier balancing and feedback for content creators. Specifically, this project will add a means by which players can indicate who won, as well as parse the saved data and store in a database as much useful information regarding the games as possible.

See GSoC sachith500 Proposal for more information.

Additional Information

The primary use for this would be for multiplayer balancing, both for developers and for authors of UMC. This breaks down into 3 parts.

Implement some way in the Wesnoth client to select a winning side (when it is not otherwise obvious).
- Alter how multiplayer games end in such a way that they must either declare a winning side or have the players select a "continue later" option. Thought needs to be put into how to handle this to include different types of games (regular duels as well as custom scenarios that might end with special conditions). This will allow much of the game data that is already collected to be used more directly for analysis. Many games will no doubt never finish "successfully" due to network disconnections or savegames that are never reloaded, but if most players simply declare a winning side via some interface at the end of the game we should get many good data points. User Made Content (UMC) would also provide good data if the [end_level] tag is correctly implemented.
Implement a replay data analyzation tool that processes replays and save what it finds out in a database.
- Store in a database as much data useful to analysis as is currently possible. Savegames are stored in WML, which can be parsed by C++, Python and Perl. Some highlights of very useful things:
  - By era, which factions win the most and on which maps.
  - Which side(s) win the most on which maps.
  - Which eras and/or maps see the most games, and their various full-game completion rate.
  - If possible, what difficulty (if applicable to MP campaign or UMC) was selected for the game.
Implement some frontend to query/visualize the data in the database.
- While the first two parts are most important, if time allows it would be great to create a medium to display and filter the data (preferably web-based). Allow for filtering of many different variable with an eye toward being able to expend and add display variables in the future.

FAQ

Q: Where in the code the data saving takes place?

A: src/server/game.cpp: game::save_replay()

Q: How will I open up the archived replays for data analysis?

A: Using a WML parser you'd go through the replay like the game proper and collect data you think is useful (storing it in the database).

Q: Where would the data analyzation tool live?

A: The analyzation tool would probably be run on the server where also wesnothd runs but it's an independent process, since wesnothd just saves the replay data it has no real idea what's in it.

Whom to ask about this

Soliton, happygrue (IRC is preferable)

@@ Line 1: / Line 1: @@
 {{SoC2014Idea}}
-A template for the Summer of Code 2014 Idea Page. Copy this to the new idea page, and edit it accordingly.
 ==Description==
 ===Improve the automated collection and display of multiplayer game data===
-Page for the idea: [[SoC_Multiplayer_Data_Analysis]]
+Page for the idea: [[SoC_Ideas_Multiplayer_Data_Analysis]]
-Multiplayer games are currently archived [http://replays.wesnoth.org/] but there is no data stored regarding who won the game.  We would like to record which team won (for as many games as possible) and then display information about these games for balancing and feedback purposes.
+Multiplayer games are currently [http://replays.wesnoth.org/ archived].  This includes lots of interesting and potentially useful data, including recruits, deaths, number of turns and other things.  The only way to access this information currently is parsing the WML one save file at a time.  Additionally, which side(s) won the game is only recorded in some cases.  We would like to record which side(s) won or lost (for as many games as possible) and then display useful information about these games for balancing and feedback purposes.
 {{#dpl:
@@ Line 13: / Line 12: @@
   |suppresserrors=true
   |noresultsheader=''There are no submitted student proposals for this idea''
-  |category=Summer of Code 2014 Student Page&SoC Ideas IDEA NAME
+  |category=Summer of Code 2014 Student Page&SoC_Ideas_Multiplayer_Data_Analysis
   |notcategory=SoC 2014 Not Submitted To Google
   |include=#Description
@@ Line 23: / Line 22: @@
 ==Additional Information==
-The primary use for this would be for multiplayer balancing both for developers and for authors of UMC.  This breaks down into 3 parts that I can see.
+The primary use for this would be for multiplayer balancing, both for developers and for authors of UMC.  This breaks down into 3 parts.
-) Alter how multiplayer games end in such a way that they must either declare a winning side or have the players select a "continue later" option.  This will allow much of the game data that is already collected to be used more directly for analysis.  Many games will no doubt never finish "successfully" due to network disconnections or savegames that are never reloaded, but if most players simply declare a winning side via some interface at the end of the game we should get many good data points.
+# '''Implement some way in the Wesnoth client to select a winning side (when it is not otherwise obvious).'''
+#*Alter how multiplayer games end in such a way that they must either declare a winning side or have the players select a "continue later" option.  Thought needs to be put into how to handle this to include different types of games (regular duels as well as custom scenarios that might end with special conditions). This will allow much of the game data that is already collected to be used more directly for analysis.  Many games will no doubt never finish "successfully" due to network disconnections or savegames that are never reloaded, but if most players simply declare a winning side via some interface at the end of the game we should get many good data points.  User Made Content (UMC) would also provide good data if the [end_level] tag is correctly implemented.
+# '''Implement a replay data analyzation tool that processes replays and save what it finds out in a database.'''
+#*Store in a database as much data useful to analysis as is currently possible.  Savegames are stored in [[SavefileWML|WML]], which can be parsed by C++, Python and Perl.  Some highlights of very useful things:
+#** By era, which factions win the most and on which maps.
+#** Which side(s) win the most on which maps.
+#** Which eras and/or maps see the most games, and their various full-game completion rate.
+#** If possible, what difficulty (if applicable to MP campaign or UMC) was selected for the game.
+# '''Implement some frontend to query/visualize the data in the database.'''
+#*While the first two parts are most important, if time allows it would be great to create a medium to display and filter the data (preferably web-based).  Allow for filtering of many different variable with an eye toward being able to expend and add display variables in the future.
-) Store in a database as much data useful to analysis as is currently possible.  Savegames are stored in WML [http://wiki.wesnoth.org/SavefileWML], which can be parsed by C++, Python and Perl.  Some highlights of very useful things:
+'''FAQ'''
-  i) By era, which races win the most and on which maps.
-  ii) Which side(s) win the most on which maps.
-  iii) Which eras and/or maps see the most games, and their various full-game completion rate.
-  iv) If possible, what difficulty (if applicable to MP campaign or UMC) was selected for the game.
-) While the first two parts are most important, if time allows it would be great to create a medium to display and filter the data (preferably web-based).  Allow for filtering of many different variable with an eye toward being able to expend and add display variables in the future.
+'''''Q: Where in the code the data saving takes place?'''''
+A: src/server/game.cpp: game::save_replay()
+'''''Q: How will I open up the archived replays for data analysis?'''''
+A: Using a WML parser you'd go through the replay like the game proper and collect data you think is useful (storing it in the database).
+'''''Q: Where would the data analyzation tool live?'''''
+A: The analyzation tool would probably be run on the server where also wesnothd runs but it's an independent process, since wesnothd just saves the replay data it has no real idea what's in it.
 ==Whom to ask about this==
 Soliton, happygrue (IRC is preferable)