The Battle for Wesnoth Wiki - User contributions [en]

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-17T14:29:33Z

Upthorn: /* If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49222] -- corrects a long-standing bug which caused MSVC9 compiled Wesnoth to crash when parsing a [trait] that lacks a description
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49223 Revision 49223] -- exposes suitable_keep() c++ AI helper function to the Lua AI library.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49236 Revision 49236] -- updates suitable_keep() Lua AI handler not to use obsolete code pattern shown in other Lua AI function handlers.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 27th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile (by July 20th)
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from. (By July 27th)
**Milestone: By July 27th Lua scripts can be called in multiple modules with minimal additional overhead
*July 27th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications (By August 2nd)
*# Implement group tag (By August 19th)
*## Extend WML engine to parse and process [group] (By August 7th)
*## Extend Lua interface to support group-based commands (Due by August 14th)
*## Extend Lua interface to allow creation, deletion, and customization of groups (by August 19th)
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent(by August 26th)
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 20th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 20th - July 15th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by July 27th
#* Lua file-loading deduplication functionality -- Due by July 27th
#** Script manager tracks files loaded by wesnoth.dofile and wesnoth.require -- Due by July 20th
#** Script manager compares new requests to files already loaded, and return reference to the loaded file -- Due by July 27th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 2nd
#* Implement grouping -- Due By August 26th
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 7th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 14th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 19th
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extensions==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

===[[StandardUnitFilter]] extension===
In order to address units by group membership, I propose to add a "group" attribute to allowable unit filters. A [filter] containing a group attribute will only return units that are members of the group whose id is specified in the attribute.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-17T14:13:26Z

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49222] -- corrects a long-standing bug which caused MSVC9 compiled Wesnoth to crash when parsing a [trait] that lacks a description
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49223 Revision 49223] -- exposes suitable_keep() c++ AI helper function to the Lua AI library.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49223 Revision 49236] -- updates suitable_keep() Lua AI handler not to use obsolete code pattern shown in other Lua AI function handlers.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 27th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile (by July 20th)
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from. (By July 27th)
**Milestone: By July 27th Lua scripts can be called in multiple modules with minimal additional overhead
*July 27th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications (By August 2nd)
*# Implement group tag (By August 19th)
*## Extend WML engine to parse and process [group] (By August 7th)
*## Extend Lua interface to support group-based commands (Due by August 14th)
*## Extend Lua interface to allow creation, deletion, and customization of groups (by August 19th)
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent(by August 26th)
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 20th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 20th - July 15th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by July 27th
#* Lua file-loading deduplication functionality -- Due by July 27th
#** Script manager tracks files loaded by wesnoth.dofile and wesnoth.require -- Due by July 20th
#** Script manager compares new requests to files already loaded, and return reference to the loaded file -- Due by July 27th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 2nd
#* Implement grouping -- Due By August 26th
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 7th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 14th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 19th
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extensions==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

===[[StandardUnitFilter]] extension===
In order to address units by group membership, I propose to add a "group" attribute to allowable unit filters. A [filter] containing a group attribute will only return units that are members of the group whose id is specified in the attribute.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-16T18:22:44Z

Upthorn: /* Deliverables */

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49222] -- corrects a long-standing bug which caused MSVC9 compiled Wesnoth to crash when parsing a [trait] that lacks a description
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49223] -- exposes suitable_keep() c++ AI helper function to the Lua AI library.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 27th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile (by July 20th)
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from. (By July 27th)
**Milestone: By July 27th Lua scripts can be called in multiple modules with minimal additional overhead
*July 27th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications (By August 2nd)
*# Implement group tag (By August 19th)
*## Extend WML engine to parse and process [group] (By August 7th)
*## Extend Lua interface to support group-based commands (Due by August 14th)
*## Extend Lua interface to allow creation, deletion, and customization of groups (by August 19th)
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent(by August 26th)
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 20th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 20th - July 15th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by July 27th
#* Lua file-loading deduplication functionality -- Due by July 27th
#** Script manager tracks files loaded by wesnoth.dofile and wesnoth.require -- Due by July 20th
#** Script manager compares new requests to files already loaded, and return reference to the loaded file -- Due by July 27th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 2nd
#* Implement grouping -- Due By August 26th
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 7th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 14th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 19th
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extensions==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

===[[StandardUnitFilter]] extension===
In order to address units by group membership, I propose to add a "group" attribute to allowable unit filters. A [filter] containing a group attribute will only return units that are members of the group whose id is specified in the attribute.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-16T18:22:37Z

Upthorn: /* Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then". */

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49222] -- corrects a long-standing bug which caused MSVC9 compiled Wesnoth to crash when parsing a [trait] that lacks a description
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49223] -- exposes suitable_keep() c++ AI helper function to the Lua AI library.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 27th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile (by July 20th)
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from. (By July 27th)
**Milestone: By July 27th Lua scripts can be called in multiple modules with minimal additional overhead
*July 27th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications (By August 2nd)
*# Implement group tag (By August 19th)
*## Extend WML engine to parse and process [group] (By August 7th)
*## Extend Lua interface to support group-based commands (Due by August 14th)
*## Extend Lua interface to allow creation, deletion, and customization of groups (by August 19th)
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent(by August 26th)
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 20th:
*# Implement Lua script manager to track files loaded by wesnoth.require and wesnoth.dofile
*# Ensure that each such external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 20th - July 15th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 10th
#* Implement grouping -- Due By August 22nd
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 12th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 17th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 22nd
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extensions==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

===[[StandardUnitFilter]] extension===
In order to address units by group membership, I propose to add a "group" attribute to allowable unit filters. A [filter] containing a group attribute will only return units that are members of the group whose id is specified in the attribute.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-16T02:35:16Z

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49222] -- corrects a long-standing bug which caused MSVC9 compiled Wesnoth to crash when parsing a [trait] that lacks a description
* [http://svn.gna.org/viewcvs/wesnoth?view=rev&revision=49222 Revision 49223] -- exposes suitable_keep() c++ AI helper function to the Lua AI library.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 25th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements - By July 22nd
*# Ensure that documentation for "source" element on wiki is thorough and complete - By July 25th
**Milestone: By July 25th, WML authors can easily include Lua code that is not embedded directly in the WML file
*July 25th-August 5th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*August 5th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications -- By August 10th
*# Implement group tag -- By August 22nd
*## Extend WML engine to parse and process [group] -- By August 12th
*## Extend Lua interface to support group-based commands -- Due by August 17th
*## Extend Lua interface to allow creation, deletion, and customization of groups -- Due by August 22nd
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 19th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements
*# Ensure that documentation for "source" element on wiki is thorough and complete
*June 19th - June 26th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 26th - July 15: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 10th
#* Implement grouping -- Due By August 22nd
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 12th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 17th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 22nd
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extensions==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

===[[StandardUnitFilter]] extension===
In order to address units by group membership, I propose to add a "group" attribute to allowable unit filters. A [filter] containing a group attribute will only return units that are members of the group whose id is specified in the attribute.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-16T02:27:23Z

Upthorn: /* Proposed WML extension */

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 25th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements - By July 22nd
*# Ensure that documentation for "source" element on wiki is thorough and complete - By July 25th
**Milestone: By July 25th, WML authors can easily include Lua code that is not embedded directly in the WML file
*July 25th-August 5th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*August 5th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications -- By August 10th
*# Implement group tag -- By August 22nd
*## Extend WML engine to parse and process [group] -- By August 12th
*## Extend Lua interface to support group-based commands -- Due by August 17th
*## Extend Lua interface to allow creation, deletion, and customization of groups -- Due by August 22nd
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 19th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements
*# Ensure that documentation for "source" element on wiki is thorough and complete
*June 19th - June 26th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 26th - July 15: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 10th
#* Implement grouping -- Due By August 22nd
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 12th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 17th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 22nd
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extensions==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

===[[StandardUnitFilter]] extension===
In order to address units by group membership, I propose to add a "group" attribute to allowable unit filters. A [filter] containing a group attribute will only return units that are members of the group whose id is specified in the attribute.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-16T02:23:33Z

Upthorn: /* Proposed WML extension */

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 25th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements - By July 22nd
*# Ensure that documentation for "source" element on wiki is thorough and complete - By July 25th
**Milestone: By July 25th, WML authors can easily include Lua code that is not embedded directly in the WML file
*July 25th-August 5th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*August 5th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications -- By August 10th
*# Implement group tag -- By August 22nd
*## Extend WML engine to parse and process [group] -- By August 12th
*## Extend Lua interface to support group-based commands -- Due by August 17th
*## Extend Lua interface to allow creation, deletion, and customization of groups -- Due by August 22nd
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 19th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements
*# Ensure that documentation for "source" element on wiki is thorough and complete
*June 19th - June 26th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 26th - July 15: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 10th
#* Implement grouping -- Due By August 22nd
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 12th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 17th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 22nd
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extension==
===[group] tag===
====Basic syntax====
My current thinking is that a group tag will look like this
[group]
id=some_group_identifier
automatic=no
[filter]
{optional [[StandardUnitFilter]]}
[/filter]
[ai]
{optional [[AiWML|AI specification]]}
[/ai]
[/group]

The id element is the only mandatory element of the group tag. The other tags specify certain automated behaviors that apply to the group. An explanation of each element follows:

=====id=====
A group id will be some locally unique identifier for the group. This is the name that will be used to issue orders to members of the group, or to add or remove units from the group's membership.

=====automatic=====
The element "automatic" is a boolean element that specifies whether the wesnoth game engine should track and update the group's membership according to a supplied [[StandardUnitFilter]]. 
The default value is "no", and a value of "yes" requires that the group have a [[StandardUnitFilter]] associated.

=====[filter]=====
This child tag will specify a [[StandardUnitFilter]] to which the group membership should be applied. 
If automatic is yes, the [filter] tag is mandatory, and group membership will automatically be updated such that all units matching the filter are considered members, and no units not matching the filter are. 
If automatic is no, units matching the filter will be added to the [group] at the time of the group's formation, but any further additions to or removals from the group will require an explicit WML or Lua command.

=====[ai]=====
This optional child tag will specify a supplementary ai to control member units. Groups without an [ai] child tag will be controlled by the overall [side] ai.

===[[UnitWML]] extension===
To facilitate the use of groups, I propose to extend Unit data to include a subtag [ai_groups]
====Basic Structure====
The basic structure of the [ai_groups] tag will be as follows
[ai_groups]
1=some_group_identifier
...
n=yet_another_group_identifier
[/ai_groups]

Each element shall be a number, associated with the id of an extant and valid [group] tag. The number shall be the priority of group membership, such that if two groups are issued conflicting orders, a unit with membership in both gruops will only follow commands issued to the group with the lowest priority number. 
For example, assume an elven archer unit is assigned to a "forest_patrol" group and a "ranged_units" group such that
[ai_groups]
1=forest_patrol
2=ranged_units
[/ai_groups]
A command is issued to forest_patrol for all member units to move along a circular route within a particular patch of forest and a command is then issued to ranged_units to assault the enemy keep.
The elven archer in question will ignore the command issued to ranged_units and continue to walk along its circular patrol. 
If, instead, the command issued to ranged_units was to prefer attacking swordsmen over archers, the elven archer in question would obey both orders, walking along its circular patrol, and attacking swordsmen in its range, but not archers.

===[group] modification actions===
In addition to the wml extensions specified above, I propose a set of actions to allow WML authors to maintain groups manually.
====Basic syntax====
[add_to_group]
unit_id=some_unit
group_id=some_group
priority=1
[/add_to_group]

[add_to_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
priority=1
[/add_to_group]

[remove_from_group]
unit_id=some_unit
group_id=some_group
[/remove_from_group]

[remove_from_group]
[filter]
[[StandardUnitFilter]]
[/filter]
group_id=some_group
[/remove_from_group]

[clear_groups]
unit_id=some_unit
[/clear_groups]

[clear_groups]
[filter]
[[StandardUnitFilter]]
[/filter]
[/clear_groups]

[delete_group]
group_id=some_group
[/delete_group]

[set_group_ai]
group_id=some_group
[ai]
[[AiWML|AI Specification]]
[/ai]
[/set_group_ai]

[clear_group_ai]
group_id=some_group
[/clear_group_ai]

[add_to_group] adds a single unit (specified in unit_id), or all units matching a [filter] to a group (specified in group_id). If the group does not exist it will be created. The optional priority element specifies what priority level to mark the new group memberships, and defaults to 1. The new membership will take priority over existing memberships with equal or lesser priority value. 
For instance, taking the elven_archer from our previous example, if we run
[add_to_group]
unit_id=elven_archer
group_id=test_group
priority=2
[/add_to_group]
the elven_archer's resulting memberships will be
[ai_groups]
1=forest_patrol
2=test_group
3=ranged_units
[/ai_groups]

[remove_from_group] removes a single unit (specified in unit_id), or all units matching a [filter] from a group (specified in group_id). If the group does not exist, nothing happens.

[clear_groups] removes a single unit (specified in unit_id), or all units matching a [filter] from all current groups.
spec
[delete_group] destroys a current group (specified in group_id) and removes all units from its membership. If the group does not exist, nothing happens.

[set_group_ai] changes the supplementary ai running in a current group (specified in group_id) to the one specified in the [ai] child tag. If the group does not exist, it will be created.

[clear_group_ai] removes the supplementary ai running in a current group (specified in group_id) and reverts all members to the overall [side] ai. If the group does not exist, nothing happens.

Additionally these actions shall all be callable from Lua.

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-14T07:32:49Z

Upthorn: updated proposal as per gsoc comment

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Bring Lua AI functionality up to par with C++
*# Expose C++ AI functions to Lua (by July 8th)
*# Provide thorough documentation for newly exposed Lua functions (by July 15th)
**Milestone: By July 15th (Midterm), Lua AI has documented access to all C++ AI functionality.
*July 15 - July 25th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements - By July 22nd
*# Ensure that documentation for "source" element on wiki is thorough and complete - By July 25th
**Milestone: By July 25th, WML authors can easily include Lua code that is not embedded directly in the WML file
*July 25th-August 5th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*August 5th - August 26th: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications -- By August 10th
*# Implement group tag -- By August 22nd
*## Extend WML engine to parse and process [group] -- By August 12th
*## Extend Lua interface to support group-based commands -- Due by August 17th
*## Extend Lua interface to allow creation, deletion, and customization of groups -- Due by August 22nd
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - June 19th: WML enhancement for easy including of Lua scripts
*# Extend Lua-context tags to include "source" elements as alternative to "code" elements
*# Ensure that documentation for "source" element on wiki is thorough and complete
*June 19th - June 26th: Implement Lua script manager to ensure that each external Lua file is loaded into memory only once, regardless of how many modules it is called from.
*June 26th - July 15: Implement unit grouping concept for AIs
*# Fully determine [group] tag specifications
*# Implement group tag
*## Extend WML engine to parse and process [group]
*## Extend Lua interface to support group-based commands
*## Extend Lua interface to allow creation, deletion, and customization of groups
*## Update wiki documentation of GroupWML and Lua to be thorough and coherent
**Milestone: By July 15th (midterm), Lua AI can be scripted by UMC devs with full functionality of C++ AI, can be easily included in WML modules, and can make use of unit-grouping for fine-grained control of AI behavior. Additionally, all of these features are thoroughly documented on the wiki.
*July 15 - August 26: Use community input and iterative design/debugging to address perceived shortcomings in the Lua AI system, documenting solutions as I go.
**Milestone: By August 26th, Lua AI scripting is easy and useful to UMC community standards.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 8th
#* Wiki documentation of all additions to Lua interface -- Due by July 15th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Fully define [group] tag syntax specification-- Due by August 10th
#* Implement grouping -- Due By August 22nd
#** WML engine extensions to parse [group] tags and assign one-time [[StandardUnitFilter]] groups to relevant units. -- Due By August 12th
#** Lua interface extensions to support AI issuing commands by group (including continually updated [[StandardUnitFilter]] groups) -- Due by August 17th
#** Lua interface extensions to allow adding and removing of specific units and unit filters to and from specific groups -- Due by August 22nd
#** Update wiki documentation of GroupWML and Lua to be thorough and coherent -- Due by August 26th

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extension==
Specification in progress

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-13T04:23:28Z

Upthorn: /* Include as much technical detail about your implementation as you can */

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Expose C++ AI functions to Lua
**Milestone: By July 15th (Midterm), Lua AI has access to all C++ AI functionality.
*July 15 - August 15: Design and implement proof-of-concept WML extensions for "Lua AI standard library"
*August 15 - August 26: Refine WML extensions through iterative debugging
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, and Lua AIs can be easily included in WML modules.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - July 15: In cooperation with UMC development community, analyse Lua AI scripting problem space and design solutions to reduce or eliminate difficulties affecting the development of custom Lua AIs.
**Milestone: By July 15th (midterm), have preliminary specifications for a standard library of utility functions and WML extensions for Lua AI scripting.
*July 15 - August 15: In cooperation with UMC development community, refine and implement Lua AI function library and WML extension specifications.
**Milestone: By August 15th, UMC developers are, on average, content with the Lua AI scripting facilities provided to them.
*August 15 - August 26: Assess and correct gaps in documentation.
**Milestone: By August 26th, project is completed and thoroughly documented.

====Include as much technical detail about your implementation as you can====
As in [[SummerOfCodeProposal_JodyNorthup|my prior application]], I have moved this to [[#Project Details|Project Details]] for organization and ease of reading.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 5th
#* Wiki documentation of all additions to Lua interface -- Due by July 12th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Elaboration under progress

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extension==
In progress

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-13T04:23:22Z

Upthorn:

{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}}

=Description=
====Jody Northup - Improve and extend Lua AI scripting facilities====
This is the [[SoC_Ideas_LuaAI_2011|Lua AI improvement]] idea. 
If I am accepted for this project, I will bring the Lua AI scripting functionality up to par with the current C++ AI programming functionality by first exposing all of the C++ AI support functions to Wesnoth's Lua API, and then adding WML utilities to include and handle custom Lua AI code in WML modules. 
I expect the first step to involve an in-depth analysis of the C++ AI functions and an inventory of the gaps in the currently-implemented Lua AI API, followed by tedious Lua API programming to expose each of those functions. I expect that the second step will involve communication with the development community to determine good WML specifications for the use of Lua AI code in UMC modules.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/proposal/review/google/gsoc2011/upthorn/1 SoC Application]

=Questionnaire=
(Largely copied from [[SummerOfCodeProposal_JodyNorthup|my successful application last year]])
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 26 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my fourth year, in terms of time spent I am in my 6th year, and in terms of the length of time remaining until I graduate, I might as well be halfway through my 2nd year.
In another year I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (UTC-7 during the summer), but my sleep schedule is fairy flexible. I am generally available on IRC between around 18:00 and 9:00 UTC, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will have limited availability on Mondays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I have a recurring social engagement all day every Wednesday, but this commitment can be cancelled if there is urgent work to be done. I also have D&D from 18:00 to 0:00 on Friday nights (1:00 to 7:00 Saturdays, UTC).
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===
====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64. Relevantly, my work on gens has included some additions/improvements to its Lua API.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code 2009, [http://wiki.scummvm.org/index.php/API-Truecolor implementing an API] to allow engine modules to negotiate with backend modules to determine color depth.
* I implemented [[PersistenceWML|Wesnoth's persistent data WML extension]] for Summer of Code last year.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for 2009's Summer of Code, as well as Battle for Wesnoth during 2010's.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in 2009's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games] (Note: dead link pending assistance from melange team to update). I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.
I participated in 2010's summer of code as a student with Battle for Wesnoth. My project was to extend WML to support persistent data for UMC developers. I was assigned Crab as a mentor, and had a very positive experience, in which I learned a lot about WML parsing, and where the errors lie in my approach to software development. My project took the entire length of the coding period, and was completed successfully with his guidance.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation in 2009, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet (except to the extent of debugging Persistence WML in multiplayer contexts.)

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.
* I gained commit access to the BfW SVN during (or slightly before) the coding period for last year's Summer of Code. It appears that I still retain this access.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I was unfortunately unable to learn Russian this year, due to scheduling conflicts.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In 2009's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.
During last year's summer of code, the project specifications were almost entirly a collaboration between myself and my mentor, which allowed me to take a much more independent approach in designing my solution, therefore involving much less refactoring and wasted effort.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
Yes: I selected [[http://wiki.wesnoth.org/SoC_Ideas_LuaAI_2011|the Lua AI improvement]] project from your ideas list. I would like to focus on ensuring that UMC authors can easily include custom Lua AI code in their modules.

====Why did you choose this project?====
# AI is an interest of mine that has always felt unapproachable, and this seems like it would provide a good opportunity to learn how AI works in a game.
# Customizability seems like one of Wesnoth's strongest "selling point"s, and if scripted AI routines are second-class citizens, difficult to use and unable to replicate hard-coded functionality, it is a major hole in Wesnoth's support for customization. In other words: it seems like a very useful goal, and I would like to improve wesnoth in a meaningful way.
# I wanted to challenge myself more this year than I have on previous years. Working on or around AI scares me a little bit, and I want to find out if my fears are founded or not.
# I have worked on extending an application's Lua interface before, and am confident that I can accomplish that in a timely manner.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps.
**Milestone: By May 24th, know which C++ functions are not yet exposed.
*May 23 - July 15: Expose C++ AI functions to Lua
**Milestone: By July 15th (Midterm), Lua AI has access to all C++ AI functionality.
*July 15 - August 15: Design and implement proof-of-concept WML extensions for "Lua AI standard library"
*August 15 - August 26: Refine WML extensions through iterative debugging
**Milestone: By August 26th (Final deadline) Lua AI can be scripted by UMC devs with full functionality of C++ AI, and Lua AIs can be easily included in WML modules.

Optimistic:
*Now - May 23: Research the C++ AI functionality and the Lua AI functionality. Assess gaps. Begin correcting.
*May 23 - June 13: Finish exposing C++ AI functions to Lua
**Milestone: By June 13th, Lua AI has access to all C++ AI functionality.
*June 13 - July 15: In cooperation with UMC development community, analyse Lua AI scripting problem space and design solutions to reduce or eliminate difficulties affecting the development of custom Lua AIs.
**Milestone: By July 15th (midterm), have preliminary specifications for a standard library of utility functions and WML extensions for Lua AI scripting.
*July 15 - August 15: In cooperation with UMC development community, refine and implement Lua AI function library and WML extension specifications.
**Milestone: By August 15th, UMC developers are, on average, content with the Lua AI scripting facilities provided to them.
*August 15 - August 26: Assess and correct gaps in documentation.
**Milestone: By August 26th, project is completed and thoroughly documented.

====Include as much technical detail about your implementation as you can====
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward.
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs. I propose to create an "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter.
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters.

====What do you expect to gain from this project?====
A better understanding of AI concepts, experience working with game AI, a better understanding of my capacities as a programmer, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period, and the free time to make a meaningful commitment to wesnoth development.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, my work on ScummVM in 2009, and my work on Wesnoth last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM in 2009. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.
*Lua: I just use Crimson editor, because it has support for Lua syntax highlighting, and Lua is interpreted, not compiled.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided my phone number in the application at google.

=Project Details=
==Overview==
I will improve and extend wesnoth's support for custom Lua AI scripts in two major steps 
Step one involves use of the Lua API to create several Lua interface functions to invoke C++ functions from, and seems fairly straightforward. 
Step two involves the extension of WML and the Lua API in order to better facilitate creation and customization of AIs, and is significantly more complex. 
For step two, I propose a couple of extensions and modifications to WML
===[Lua] and [AI] tag enhancement===
I propose to add a "source" element to [Lua] and [AI] tags as an alternative to the "code" element. The source element will specify the relative path to a Lua file which contains the desired Lua code. 
Files loaded from the "source" element in these tags will be indexed globally so that multiple WML modules can reference the same Lua script file without causing the file to be loaded into memory multiple times.
===[group] tag===
I propose to create a "[group]" tag for scenario authors to specify AI scripts per-group of units. This would be accompanied by a "[groups]" subtag in unitWML, which contains a unit's group memberships in priority order. Additionally a group could be associated with a standard unit filter in order to automatically apply itself to all new units created which match the filter. 
Additionally, the Lua API will be extended to allow Lua scripts to send messages to unit groups, so that groups can be dynamically created and commanded during the execution of a scenario. This will be accomplished by significant additions to the WML parsing engine to teach it about the new unit group concept, creation of an as-yet indeterminate number of Lua or Lua API functions yet to be named, and possibly a group-related extension to Standard Unit Filters. 
Proposed syntax for the [group] tag is provided below, in the [[#Proposed WML extension|proposed WML extension]] section.

==Deliverables==
===Mandatory===
# Full exposure to Lua of C++ AI support functions. -- Due by July 15th
#* C++ AI support functions exposed to Lua interface -- Due by July 5th
#* Wiki documentation of all additions to Lua interface -- Due by July 12th
# Lua loadable non-redundantly into WML from external files -- Due by August 5th
#* Basic functionality of "source" element in [Lua] tags and [AI] tags with engine="Lua" -- Due by July 25th
#** WML properly parses "source" element and passes file contents to Lua engine -- Due by July 22nd
#** Source element syntax and usage is thoroughly documented on -- Due by July 25th
#* Lua file-loading deduplication functionality -- Due by August 5th
#** Script manager to store path of scripts on first load, compare new requests to files already loaded, and return reference to the loaded file -- Due by August 5th
# [group] tag -- Due by August 26th
#* Elaboration under progress

===Optional===
To be determined via discussion with mentors and UMC development community.

==Proposed WML extension==
In progress

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-07T19:09:09Z

Upthorn: /* Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including in

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-06T21:26:25Z

Upthorn: /* IRC */

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-06T21:20:12Z

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-04-03T19:12:11Z

Upthorn:

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-03-31T17:14:07Z

Upthorn:

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-03-31T17:13:02Z

Upthorn:

SummerOfCodeProposal 2011 LuaAI JodyNorthup

2011-03-31T06:01:35Z

Upthorn: Created page with '{{SoC2011Student_2|upthorn|SoC_Ideas_LuaAI_2011}} =Description= ====Jody Northup - Improve and extend Lua AI scripting facilities==== This is the [[SoC_Ideas_LuaAI_2011|Lua AI i…'

PersistenceWML

2010-08-16T11:13:25Z

Upthorn: /* Transactions */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: in multiplayer, do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* During multiplayer, Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1, and will force a quit if they occur there.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side
* Persistent Variables will retain their values after wesnoth is closed, until they are cleared or the file containing their namespace is erased or lost.

== Transactions ==
For the purposes of efficiency, and gameplay consistency, persistent variables are usually not written out to file immediately. Instead, they are grouped into transactions which are sets of changes that are either all written at once, or not written at all. 
The policy regarding persistent variables will be written is as follows:
*Transactions will be committed (written to permanent file) whenever the game autosaves, whenever the player saves the game manually, or whenever a scenario is completed in either victory or defeat.
*Transactions will be cancelled (ignored and wiped from memory) whenever the player loads an earlier savegame, or quits without saving.
*A WML author can mark a change as an exception to a transaction, to be committed immediately by itself, by using setting the attribute "immediate" to "yes" in a [set_global_variable] or [clear_global_variable] tag. Such operations will be written immediately, regardless of the transaction status, without committing any waiting transactions.

{| align="center" border="1" style="text-align:center;"

| NOTE: Because of the interaction between persistent variable transactions and user loading/saving of games, it is suggested that any variables a WML author wants to remain consistent for a single playthrough of a scenario should be read in immediately at the beginning of the scenario (side 1 turn 1), and set or cleared only in the victory and/or defeat events.

|}

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
side=1
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
side=1
immediate=no
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
side=1
immediate=no
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with. 
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer. 
This attribute is not used in single player, and may be omitted in single-player only scenarios. 
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

==== immediate ====
This attribute is an optional boolean value that specifies whether the changes made by the [set_global_variable] or [clear_global_variable] should be stored permanently at the moment they occur. 
If it is set to yes, the namespace's permanent storage will be updated with the change that occurred in the marked tag immediately. 
The default value is no. 
This attribute does not apply to [get_global_variable], which always occurs immediately.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-16T11:12:19Z

Upthorn:

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: in multiplayer, do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* During multiplayer, Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1, and will force a quit if they occur there.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side
* Persistent Variables will retain their values after wesnoth is closed, until they are cleared or the file containing their namespace is erased or lost.

== Transactions ==
For the purposes of efficiency, and gameplay consistency, persistent variables are usually not written out to file immediately. Instead, they are grouped into transactions which are sets of changes that are either all written at once, or not written at all. The policy regarding this is as follows.
*Transactions will be committed (written to permanent file) whenever the game autosaves, whenever the player saves the game manually, or whenever a scenario is completed in either victory or defeat.
*Transactions will be cancelled (ignored and wiped from memory) whenever the player loads an earlier savegame, or quits without saving.
*A WML author can mark a change as an exception to a transaction, to be committed immediately by itself, by using setting the attribute "immediate" to "yes" in a [set_global_variable] or [clear_global_variable] tag. Such operations will be written immediately, regardless of the transaction status, without committing any waiting transactions.

{| align="center" border="1" style="text-align:center;"

| NOTE: Because of the interaction between persistent variable transactions and user loading/saving of games, it is suggested that any variables a WML author wants to remain consistent for a single playthrough of a scenario should be read in immediately at the beginning of the scenario (side 1 turn 1), and set or cleared only in the victory and/or defeat events.

|}

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
side=1
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
side=1
immediate=no
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
side=1
immediate=no
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with. 
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer. 
This attribute is not used in single player, and may be omitted in single-player only scenarios. 
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

==== immediate ====
This attribute is an optional boolean value that specifies whether the changes made by the [set_global_variable] or [clear_global_variable] should be stored permanently at the moment they occur. 
If it is set to yes, the namespace's permanent storage will be updated with the change that occurred in the marked tag immediately. 
The default value is no. 
This attribute does not apply to [get_global_variable], which always occurs immediately.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-16T10:43:06Z

Upthorn: /* Overview */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: in multiplayer, do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* During multiplayer, Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1, and will force a quit if they occur there.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side
* Persistent Variables will retain their values after wesnoth is closed, until they are cleared or the file containing their namespace is erased or lost.

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
side=1
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
side=1
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
side=1
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
This attribute is not used in single player, and may be omitted in single-player only scenarios.
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-15T22:42:33Z

Upthorn: /* Overview */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: in multiplayer, do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* During multiplayer, Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1, and will force a quit if they occur there.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
side=1
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
side=1
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
side=1
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
This attribute is not used in single player, and may be omitted in single-player only scenarios.
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-15T22:41:31Z

Upthorn: /* WML Syntax */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
side=1
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
side=1
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
side=1
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
This attribute is not used in single player, and may be omitted in single-player only scenarios.
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

Template:WML Tags

2010-08-15T22:38:40Z

Upthorn:

{| class="gallery" style="width:225px;float: right;border: 1px solid #B48648; color:#B48648; font-size: 7pt;margin-left;10px;"
|-
|
[{{SERVER}}{{localurl:Template:WML Tags|action=edit}} edit ]
'''WML Tags'''

|-
|''A:''
[[AbilitiesWML|abilities]],
[[CampaignWML|about]],
[[AdvancedPreferenceWML|advanced_preference]],
[[UnitTypeWML|advancefrom]],
[[UnitTypeWML|advancement]],
[[StatisticalScenarioWML|advances]],
[[AiWML|ai]],
[[DirectActionsWML#.5Ballow_recruit.5D|allow_recruit]],
[[DirectActionsWML#.5Ballow_undo.5D|allow_undo]],
[[ConditionalActionsWML#Meta_Condition_Tags|and]],
[[InterfaceActionsWML|animate_unit]],
[[AnimationWML|animation]],
[[VariablesWML|array]],
[[UnitTypeWML|attack]],
[[AnimationWML|attack_filter]],
[[StatisticalScenarioWML|attacks]],
[[AiWML|avoid]];
|-
|''B:''
[[UnitTypeWML|base_unit]], [[BinaryPathWML|binary_path]], [[HelpWML|bold]], [[EditorWML|brush]];
|-
|''C:''
[[CampaignWML#The_.5Bcampaign.5D_tag|campaign]],
[[DirectActionsWML#.5Bcapture_village.5D|capture_village]],
[[ConditionalActionsWML#.5Bswitch.5D|case]],
[[ReplayWML|choose]],
[[PersistenceWML|clear_global_variable]],
[[InternalActionsWML|clear_variable]],
[[InterfaceActionsWML|colour_adjust]],
command([[InterfaceActionsWML|action]], [[ReplayWML|replay]]);
|-
|''D:''
[[AbilitiesWML|damage]],
[[StatisticalScenarioWML|deaths]],
[[InterfaceActionsWML|debug_message]],
[[AnimationWML|defend]],
[[StatisticalScenarioWML|defends]],
[[UnitTypeWML|defense]],
[[InterfaceActionsWML|delay]],
[[ReplayWML|destination]],
[[DirectActionsWML#.5Bdisallow_recruit.5D|disallow_recruit]],
[[ConditionalActionsWML#.5Bwhile.5D|do]];
|-
|''E:''
[[EditorWML|editor_group]],
[[EditorWML|editor_music]],
[[EditorWML|editor_times]],
[[EditorWML|editor_tool_hint]],
[[EffectWML|effect]],
[[ConditionalActionsWML#Conditional_Actions|else]],
[[DirectActionsWML#.5Bendlevel.5D|endlevel]],
end_turn ([[DirectActionsWML#.5Bend_turn.5D|action]], [[ReplayWML|replay]]),
[[EraWML|era]],
[[EventWML|event]],
[[ThemeWML|expenses]];
|-
|''F:''
[[EventWML#.5Bfilter.5D|filter]],
[[FilterWML|filter]],
[[AnimationWML|filter_attack]],
[[EventWML#.5Bfilter_attack.5D|filter_attack]],
[[FilterWML|filter_location]],
[[EventWML#.5Bfilter_second.5D|filter_second]],
[[FilterWML|filter_second]],
[[AnimationWML|filter_second_attack]],
[[EventWML#.5Bfilter_second_attack.5D|filter_second_attack]],
[[FilterWML|filter_vision]],
[[StandardUnitFilter|filter_wml]],
[[InternalActionsWML|fire_event]],
[[HelpWML|format]],
[[AnimationWML|frame]];
|-
|''G:''
[[GameConfigWML|game_config]],
[[ScenarioWML|generator]],
[[PersistenceWML|get_global_variable]],
[[DirectActionsWML|gold]],
[[ThemeWML|gold]];
|-
|''H:''
[[ConditionalActionsWML#Condition_Tags|have_location]],
[[ConditionalActionsWML#Condition_Tags|have_unit]],
[[HelpWML|header]],
[[DirectActionsWML#.5Bheal_unit.5D|heal_unit]],
[[UnitsWML|hide_help]],
[[InterfaceActionsWML|hide_unit]];
|-
|''I:''
[[ConditionalActionsWML#.5Bif.5D|if]],
[[TimeWML|illuminated_time]],
[[TerrainGraphicsWML|image]],
[[HelpWML|img]],
[[ThemeWML|income]],
[[ReplayWML|init_side]],
[[InternalActionsWML|insert_tag]],
[[InterfaceActionsWML#.5Binspect.5D|inspect]],
[[HelpWML|italic]],
[[InterfaceActionsWML|item]];
|-
|''J:''
[[HelpWML|jump]],
[[InternalActionsWML|join]];
|-
|''K:''
[[DirectActionsWML#.5Bkill.5D|kill]],
[[StatisticalScenarioWML|killed]];
|-
|''L:''
label ([[InterfaceActionsWML|map]], [[ThemeWML|theme]]),
[[LanguageWML|language]],
[[AiWML|leader_goal]],
[[LuaWML|lua]];
|-
|''M:''
[[ThemeWML|main_map]],
[[ThemeWML|menu]],
[[InterfaceActionsWML|message]],
[[ThemeWML|mini_map]],
[[AnimationWML|missile_frame]],
[[SingleUnitWML|modifications]],
[[DirectActionsWML#.5Bmodify_side.5D|modify_side]],
[[DirectActionsWML#.5Bmodify_turns.5D|modify_turns]],
[[ReplayWML|move]],
[[DirectActionsWML#.5Bmove_unit.5D|move_unit]],
[[InterfaceActionsWML|move_unit_fake]],
[[UnitTypeWML|movement costs]],
[[UnitsWML|movetype]],
[[ScenarioWML|multiplayer]],
[[EraWML|multiplayer_side]],
[[InterfaceActionsWML|music]];
|-
|''N:''
[[ConditionalActionsWML#Meta_Condition_Tags|not]],
[[FilterWML|not]],
[[ThemeWML|num_units]];
|-
|''O:''
[[DirectActionsWML#.5Bobject.5D|object]],
[[InterfaceActionsWML|objectives]],
[[InterfaceActionsWML|objective]],
[[ThemeWML|observers]],
[[InterfaceActionsWML|open_help]],
[[InterfaceActionsWML|option]],
[[ConditionalActionsWML#Meta_Condition_Tags|or]];
|-
|''P:''
[[ThemeWML|panel]], [[IntroWML|part]], [[DirectActionsWML#.5Bpetrify.5D|petrify]], [[DirectActionsWML#.5Bplace_shroud.5D|place_shroud]], [[ThemeWML|position]],
[[InterfaceActionsWML|print]], [[AiWML|protect_location]], [[AiWML|protect_unit]];
|-
|''R:''
[[UnitsWML|race]], [[ReplayWML|random]], recall ([[DirectActionsWML#.5Brecall.5D|action]],
[[ReplayWML|replay]]), [[StatisticalScenarioWML|recalls]],
[[ReplayWML|recruit]], [[StatisticalScenarioWML|recruits]], [[InterfaceActionsWML|redraw]],
[[HelpWML|ref]], [[DirectActionsWML|remove_shroud]], [[InterfaceActionsWML|remove_unit_overlay]],
[[InterfaceActionsWML|removeitem]], [[InterfaceActionsWML|remove_sound_source]],
[[DirectActionsWML#.5Breplace_map.5D|replace_map]], [[DirectActionsWML#.5Breplace_schedule.5D|replace_schedule]], [[SavefileWML|replay]], [[SavefileWML|replay_start]],
[[UnitTypeWML|resistance]], [[ThemeWML|resolution]], [[ReplayWML|results]], [[InternalActionsWML|role]];
|-
|''S:''
[[SavefileWML|save]], [[ScenarioWML|scenario]],
[[InterfaceActionsWML|scroll]], [[InterfaceActionsWML|scroll_to]],
[[InterfaceActionsWML|scroll_to_unit]], [[AnimationWML|secondary_attack_filter]], [[AnimationWML|secondary_unit_filter]], [[HelpWML|section]], [[PersistenceWML|set_global_variable]],
[[InterfaceActionsWML#.5Bset_menu_item.5D_.28SVN_trunk_only.29|set_menu_item]], [[DirectActionsWML#.5Bset_recruit.5D|set_recruit]],
[[InternalActionsWML|set_variable]], [[InternalActionsWML|set_variables]], [[InterfaceActionsWML|show_objectives]],
[[SideWML|side]], [[ThemeWML|side_playing]], [[SavefileWML|snapshot]],
[[InterfaceActionsWML|sound]], [[InterfaceActionsWML|sound_source]], [[ReplayWML|source]], [[EventWML|special_filter]], [[EventWML|special_filter_second]],
[[InternalActionsWML|split]],
[[StatisticalScenarioWML#The_.5Bstatistics.5D_tag|statistics]],
status([[SingleUnitWML|single unit]], [[ThemeWML|theme]]), [[InternalActionsWML|store_gold]], [[InternalActionsWML|store_locations]],
[[InternalActionsWML|store_map_dimensions]],
[[InternalActionsWML|store_side]], [[InternalActionsWML|store_starting_location]], [[InternalActionsWML|store_time_of_day]], [[InternalActionsWML|store_unit]], [[InternalActionsWML|store_villages]] [[IntroWML|story]],
[[ConditionalActionsWML#.5Bswitch.5D|switch]];
|-
|''T:''
[[AiWML|target]],
[[StatisticalScenarioWML#The_.5Bteam.5D_tag|team]],
teleport ([[DirectActionsWML#.5Bteleport.5D|action]], [[AbilitiesWML|ability]]), [[AnimationWML|teleport_anim]],
terrain([[TerrainWML|define]], [[DirectActionsWML#.5Bterrain.5D|create]]), [[TerrainGraphicsWML|terrain_graphics]], [[TerrainMaskWML|terrain_mask]], [[ScenarioWML#Test_scenario|test]],
[[WesCamp|textdomain]], [[InterfaceActionsWML|text_input]], [[ThemeWML|theme]], [[ConditionalActionsWML#.5Bif.5D|then]],
[[TerrainGraphicsWML|tile]], [[TimeWML|time]], time_area ([[DirectActionsWML#.5Btime_area.5D|action]], [[ScenarioWML|scenario]]),
[[ThemeWML|time_of_day]],
[[HelpWML|topic]], [[HelpWML|toplevel]], [[SingleUnitWML|trait]], [[DirectActionsWML#.5Btunnel.5D|tunnel]] [[ThemeWML|turn]], [[ScenarioWML|tutorial]];
|-
|''U:''
[[InterfaceActionsWML|unhide_unit]], [[SingleUnitWML|unit]],
[[ThemeWML|unit_abilities]], [[ThemeWML|unit_alignment]], [[ThemeWML|unit_description]], [[AnimationWML|unit_filter]], [[ThemeWML|unit_hp]], [[ThemeWML|unit_image]], [[ThemeWML|unit_level]], [[ThemeWML|unit_moves]],
[[InterfaceActionsWML|unit_overlay]], [[ThemeWML|unit_profile]], [[ThemeWML|unit_status]],
[[ThemeWML|unit_traits]], [[UnitTypeWML|unit_type]], [[ThemeWML|unit_weapons]], [[ThemeWML|unit_xp]],
[[UnitsWML|units]], [[DirectActionsWML#.5Bunpetrify.5D|unpetrify]], [[DirectActionsWML#.5Bunstore_unit.5D|unstore_unit]], [[ThemeWML|upkeep]];
|-
| ''V:''
[[ConditionalActionsWML#Condition_Tags|variable]],
[[VariablesWML|variables]],
[[SideWML|village]],
[[ThemeWML|villages]];
|-
| ''W:''
[[ConditionalActionsWML#.5Bwhile.5D|while]],
[[InterfaceActionsWML|wml_message]];
|}

<noinclude>An box with all the WML tags, each linking to the page they are described in. This box should be included in each of the WML reference pages.</noinclude>

PersistenceWML

2010-08-15T22:36:33Z

Upthorn: /* WML Syntax */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
<side=1>
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
<side=1>
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
<side=1>
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-15T16:38:39Z

Upthorn: /* side */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="text-align:center; color:red; font-size: larger"

| WARNING: do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is equal to "global", the currently active player's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] and [clear_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-05T06:08:47Z

Upthorn: /* Internal Separation */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

{| align="center" border="1" style="color:red; font-size: larger"

| WARNING: do not use persistent variables in '''prestart''', '''start''', and other non-synchronized events.
|}

* Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo", use namespace="foo.bar"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-08-05T06:05:01Z

Upthorn: /* Overview */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

* Persistent Variables '''''can not''''' be used in events that occur before side 1 turn 1.
* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
NOT YET IMPLEMENTED
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-06-06T04:41:14Z

Upthorn: /* namespace */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of", must follow a namespace and may only be directly followed by a "." or another "^". And is provided so that content creators can easily use macros to eliminate data repitition for the most frequently used namespaces.
**a namespace equal to "{def}^" with the def macro "foo" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar" will access the "foo" namespace.
**a namespace equal to "{def}^" with the def macro "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "{def}^^" with the def macro "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "{def}^.rod" with the def macro of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^^.rod" with the def macro "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "{def}^.quiz" with the def macro "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace beginning with "^" is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
NOT YET IMPLEMENTED
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

SummerOfCodeProposal JodyNorthup

2010-05-26T01:20:48Z

Upthorn: /* Mandatory */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4:
*#Refine solution through iterative design and implementation with continuous discussion with community members.
*#Design and implement persistence data management features while awaiting feedback during discussion periods.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately. There is a persistence data management interface allowing users to backup, clear, and restore their persistence data.
*July 5 - July 12:
*#Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*#Document existing changes on wiki while awaiting feedback during discussion periods.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer, additionally Single Player implementation and persistence data management are documented on the wiki.
*July 13 - July 27:
*#Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
*#Present finalized persistence WML capabilities to UMC developers on forums.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic: TODO: Account for all mandatory goals in optimistic timeline.
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables. -- Completed by Aug 26
#* events and syntax -- Completed by June 2
#** bulletproof singleplayer implementation of WML persistence events -- Completed by July 4
#** multiplayer implementation of WML persistence events -- Completed by July 27
#*** error recovery in MP in case players drop/join. -- Completed by Aug 16
#*** transaction blocks for global variable operations. -- Completed by July 27
#** Wiki documentation of events and syntax -- Completed by July 12
#* present new WML capabilities to UMC authors on wesnoth's forums. -- Completed by July 27
# User ability to manage persistence data -- Completed by July 12
#* Backup -- Completed by July 4
#* Clear -- Completed by July 4
#* Restore -- Completed by July 4
#* User Interface (probably modified from "save game" or "load game" dialog. -- Completed by July 4
#* wiki documentation/manual page -- Completed by July 12

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# Engine or built-in macro support for default namespace
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-05-25T22:31:47Z

Upthorn: /* Mandatory */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4:
*#Refine solution through iterative design and implementation with continuous discussion with community members.
*#Design and implement persistence data management features while awaiting feedback during discussion periods.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately. There is a persistence data management interface allowing users to backup, clear, and restore their persistence data.
*July 5 - July 12:
*#Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*#Document existing changes on wiki while awaiting feedback during discussion periods.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer, additionally Single Player implementation and persistence data management are documented on the wiki.
*July 13 - July 27:
*#Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
*#Present finalized persistence WML capabilities to UMC developers on forums.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic: TODO: Account for all mandatory goals in optimistic timeline.
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables. -- Completed by Aug 26
#* events and syntax -- Completed by June 2
#** bulletproof singleplayer implementation of WML persistence events -- Completed by July 4
#** multiplayer implementation of WML persistence events -- Completed by July 27
#*** error recovery in MP in case players drop/join. -- Completed by Aug 16
#*** transaction blocks for global variable operations. -- Completed by July 27
#** Wiki documentation of events and syntax -- Completed by July 12
#* present new WML capabilities to UMC authors on wesnoth's forums. -- Completed by July 12
# User ability to manage persistence data -- Completed by July 12
#* Backup -- Completed by July 4
#* Clear -- Completed by July 4
#* Restore -- Completed by July 4
#* User Interface (probably modified from "save game" or "load game" dialog. -- Completed by July 4
#* wiki documentation/manual page -- Completed by July 12

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# Engine or built-in macro support for default namespace
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-05-25T22:31:18Z

Upthorn: /* Mandatory */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4:
*#Refine solution through iterative design and implementation with continuous discussion with community members.
*#Design and implement persistence data management features while awaiting feedback during discussion periods.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately. There is a persistence data management interface allowing users to backup, clear, and restore their persistence data.
*July 5 - July 12:
*#Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*#Document existing changes on wiki while awaiting feedback during discussion periods.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer, additionally Single Player implementation and persistence data management are documented on the wiki.
*July 13 - July 27:
*#Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
*#Present finalized persistence WML capabilities to UMC developers on forums.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic: TODO: Account for all mandatory goals in optimistic timeline.
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables.
#* events and syntax -- Completed by June 2
#** bulletproof singleplayer implementation of WML persistence events -- Completed by July 4
#** multiplayer implementation of WML persistence events -- Completed by July 27
#*** error recovery in MP in case players drop/join. -- Completed by Aug 16
#*** transaction blocks for global variable operations. -- Completed by July 27
#** Wiki documentation of events and syntax -- Completed by July 12
#* present new WML capabilities to UMC authors on wesnoth's forums. -- Completed by July 12
# User ability to manage persistence data
#* Backup -- Completed by July 4
#* Clear -- Completed by July 4
#* Restore -- Completed by July 4
#* User Interface (probably modified from "save game" or "load game" dialog. -- Completed by July 4
#* wiki documentation/manual page -- Completed by July 12

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# Engine or built-in macro support for default namespace
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-05-25T22:31:08Z

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4:
*#Refine solution through iterative design and implementation with continuous discussion with community members.
*#Design and implement persistence data management features while awaiting feedback during discussion periods.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately. There is a persistence data management interface allowing users to backup, clear, and restore their persistence data.
*July 5 - July 12:
*#Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*#Document existing changes on wiki while awaiting feedback during discussion periods.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer, additionally Single Player implementation and persistence data management are documented on the wiki.
*July 13 - July 27:
*#Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
*#Present finalized persistence WML capabilities to UMC developers on forums.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic: TODO: Account for all mandatory goals in optimistic timeline.
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables.
#* events and syntax
#** bulletproof singleplayer implementation of WML persistence events
#** multiplayer implementation of WML persistence events
#*** error recovery in MP in case players drop/join.
#*** transaction blocks for global variable operations.
#** Wiki documentation of events and syntax
#* present new WML capabilities to UMC authors on wesnoth's forums.
# User ability to manage persistence data
#* Backup
#* Restore
#* User Interface (probably modified from "save game" or "load game" dialog.
#* wiki documentation/manual page

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# Engine or built-in macro support for default namespace
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-05-25T11:33:06Z

Upthorn: /* Optional */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables.
#* events and syntax
#** bulletproof singleplayer implementation of WML persistence events
#** multiplayer implementation of WML persistence events
#*** error recovery in MP in case players drop/join.
#*** transaction blocks for global variable operations.
#** Wiki documentation of events and syntax
#* present new WML capabilities to UMC authors on wesnoth's forums.
# User ability to manage persistence data
#* Backup
#* Restore
#* User Interface (probably modified from "save game" or "load game" dialog.
#* wiki documentation/manual page

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# Engine or built-in macro support for default namespace
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-05-25T11:31:42Z

Upthorn: Did not enroll in math over the summer.

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables.
#* events and syntax
#** bulletproof singleplayer implementation of WML persistence events
#** multiplayer implementation of WML persistence events
#*** error recovery in MP in case players drop/join.
#*** transaction blocks for global variable operations.
#** Wiki documentation of events and syntax
#* present new WML capabilities to UMC authors on wesnoth's forums.
# User ability to manage persistence data
#* Backup
#* Restore
#* User Interface (probably modified from "save game" or "load game" dialog.
#* wiki documentation/manual page

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

PersistenceWML

2010-05-25T11:27:33Z

Upthorn: /* namespace */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

PLANNED
*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of" and may only occur at the beginning of a namespace, before any "." characters, or namespace names.
**TODO modify description of "^" to reflect recent decisions regarding handling of namespaces.
**a namespace equal to "^" with the default namespace "foo" will access the "foo" namespace.
**a namespace equal to "^" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "^" with the default namespace "foo.bar" will access the "foo" namespace.
**a namespace equal to "^" with the default namespace "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "^^" with the default namespace "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "foo.bar^", "foo^", ".bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.
*In this attribute, the sequence "^." means "sibling of", and may only occur as the last element in a sequence of "^" characters.
**a namespace equal to "^.rod" with the default namespace of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^^.rod" with the default namespace of "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.quiz" with the default namespace of "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.rod" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "foo.bar^.rod", ".bar^.rod", "^^.bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
NOT YET IMPLEMENTED
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-05-25T11:18:04Z

Upthorn: /* namespace */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

PLANNED
*If this attribute is omitted or left empty, the scenario or campaign's default namespace will be used.
*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of" and may only occur at the beginning of a namespace, before any "." characters, or namespace names.
**TODO modify description of "^" to reflect recent decisions regarding handling of namespaces.
**a namespace equal to "^" with the default namespace "foo" will access the "foo" namespace.
**a namespace equal to "^" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "^" with the default namespace "foo.bar" will access the "foo" namespace.
**a namespace equal to "^" with the default namespace "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "^^" with the default namespace "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "foo.bar^", "foo^", ".bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.
*In this attribute, the sequence "^." means "sibling of", and may only occur as the last element in a sequence of "^" characters.
**a namespace equal to "^.rod" with the default namespace of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^^.rod" with the default namespace of "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.quiz" with the default namespace of "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.rod" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "foo.bar^.rod", ".bar^.rod", "^^.bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
NOT YET IMPLEMENTED
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-05-03T02:32:04Z

Upthorn: /* Purpose */

{{DevFeature1.9}}
{{WML Tags}}

NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.

== Purpose ==
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

PLANNED
*If this attribute is omitted or left empty, the scenario or campaign's default namespace will be used.
*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of" and may only occur at the beginning of a namespace, before any "." characters, or namespace names.
**a namespace equal to "^" with the default namespace "foo" will access the "foo" namespace.
**a namespace equal to "^" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "^" with the default namespace "foo.bar" will access the "foo" namespace.
**a namespace equal to "^" with the default namespace "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "^^" with the default namespace "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "foo.bar^", "foo^", ".bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.
*In this attribute, the sequence "^." means "sibling of", and may only occur as the last element in a sequence of "^" characters.
**a namespace equal to "^.rod" with the default namespace of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^^.rod" with the default namespace of "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.quiz" with the default namespace of "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.rod" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "foo.bar^.rod", ".bar^.rod", "^^.bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
NOT YET IMPLEMENTED
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

PersistenceWML

2010-05-03T02:31:34Z

Upthorn: Created page with '{{DevFeature1.9}} {{WML Tags}} == Purpose == NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to…'

{{DevFeature1.9}}
{{WML Tags}}

== Purpose ==
NOTE: This feature is currently under active development for Summer of Code. Information contained in this page is not guaranteed to be accurate, may concern planned future developments, and may change significantly without warning.
Persistent Variables are used to record information that can be accessed across savegames and session instances.

For instance, assume that completing a campaign on hard mode should unlock a special item in subsequent playthroughs of this campaign. With a persistent variable, that can be recorded, and made available to the campaign on later plays.

This can also be used to allow two or more related campaigns to communicate with each other about what has happened in each, allowing the player's choices in one campaign to influence the state of another.

== Overview ==
Persistent Variables work like [[VariablesWML|normal Variables]], with a few differences.

* Persistent Variables may be assigned, retrieved, or cleared, but not directly queried.
* Persistent Variables may only be assigned from an existing variable, not directly from a constant
* In addition to a name, each Persistent Variable must have a namespace
* In multiplayer campaigns, each Persistent Variable also requires a side

== WML Syntax ==
The following WML Tags are provided.
[set_global_variable]
namespace=my_addon
from_local=foo
to_global=my_variable_name
{side=1}
[/set_global_variable]
Assigns a persistent variable with the contents of a standard variable

[get_global_variable]
namespace=my_addon
from_global=my_variable_name
to_local=foo
{side=1}
[/get_global_variable]
Retrieves the contents of a persistent variable and stores them in a standard variable.
and
[clear_global_variable]
namespace=my_addon
global=my_variable_name
{side=1}
[/clear_global_variable]
Clears a persistent variable entirely.
=== Attribute information ===
==== namespace ====
This atribute specifies the name of the namespace that the persistent variable resides in.

PLANNED
*If this attribute is omitted or left empty, the scenario or campaign's default namespace will be used.
*In this attribute, the character "." means "child of",
**A namespace equal to "foo.bar" will access a "bar" namespace inside "foo", creating it if it doesn't already exist, and store the peristent variable inside "bar".
**If this attribute begins with "." it will access a child namespace of the default namespace.
***A namespace equal to ".bar" with the default namespace of "foo" will access a "bar" namespace inside the "foo" namespace.
*In this attribute, the character "^" means "parent of" and may only occur at the beginning of a namespace, before any "." characters, or namespace names.
**a namespace equal to "^" with the default namespace "foo" will access the "foo" namespace.
**a namespace equal to "^" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "^" with the default namespace "foo.bar" will access the "foo" namespace.
**a namespace equal to "^" with the default namespace "foo.bar.test" will access the "bar" namespace. inside the "foo" namespace.
**a namespace equal to "^^" with the default namespace "foo.bar.test" will access the "foo" namespace.
**a namespace equal to "foo.bar^", "foo^", ".bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.
*In this attribute, the sequence "^." means "sibling of", and may only occur as the last element in a sequence of "^" characters.
**a namespace equal to "^.rod" with the default namespace of "foo.bar" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^^.rod" with the default namespace of "foo.bar.test" will access the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.quiz" with the default namespace of "foo.bar.test" will access the "quiz" namespace within the "rod" namespace within the "foo" namespace.
**a namespace equal to "^.rod" with no default namespace defined is invalid and will generate a WML error, because it is impossible to determine the parent of nothing.
**a namespace equal to "foo.bar^.rod", ".bar^.rod", "^^.bar^", or the like is invalid and will generate a WML syntax error, because "^" may only be used before the namespace name begins.

==== *global ====
These attributes (from_global, to_global, and global) specify the name of the persistent variable to be worked with.
The name of the persistent variable must be a valid variable name, as defined in [[VariablesWML]]

==== *local ====
These attributes (from_local, to_local) specify the name of the standard variable to be worked with.

==== side ====
This attribute specifies which player client's persistence data should read from or written to in multiplayer.
It is not used in single player.
If this attribute is set equal to "host", the game's current host's data will be used.
If this attribute is empty, omitted, or equal to "global", the game's current host's data will be read in [get_global_variable] operations, and all player's data will be written to in [set_global_variable] operations.

== About Namespaces ==
Namespaces exist in order to prevent variable collisions from occuring when two or more unrelated campaigns use the same name for some of their persistent variables.
A namespace is simply a name that identifies one set of variables. A namespace may contain only alphabetic characters, digits, and underscores.

=== Internal Separation ===
NOT YET IMPLEMENTED
Just as variables can contain other variables, namespaces may contain other namespaces. The syntax for this works just like container variables: to refer to a namespace "bar" contained within a namespace "foo"

=== Uniqueness ===
Because situations may arise where persistent variables from campaigns that are not installed concurrently have to coexist, a namespace should be unique per-context. This means that two unrelated campaigns with the same name should use different namespaces, but two or more related campaigns with different names may use the same namespace.

The following guidelines are provided in order to facilitate this:
==== Single Standalone Campaign ====
For a single campaign which is not intended to interact with other campaigns, the format "(author_handle)_(campaign_name)" is suggested for namespaces.
As an example, if an author going by "upthorn" in the community were to create a campaign named "foo", the suggested namespace would be "upthorn_foo".

In cases where the author wishes to remain anonymous, or there are many contributing authors (more than say, two or three), the alternate format "(campaign_name)_(month)(year)" is suggested.
Using the above example, assuming that upthorn began work on the campaign on the day this wiki page was created, the resulting namespace would be "foo_may2010"

==== Multiple Related Campaigns ====
For multiple related campaigns, there are a few different suggested namespace options depending on order of development and the intended level of interaction.

===== Developed or Conceived Concurrently =====
If a campaign is intended from the beginning to be part of a larger world of related campaigns, it is recommended that a namespace be created for the overall world, using the format "(author_handle)_(world_name)" for a named author, or "(world_name)_(month)(year)" for an unnamed author.
As an example, if an author going by "upthorn" were to begin development on a "hypothetical" world the day this page was created, this would result in a namespace of either "upthorn_hypothetical" or "hypothetical_may2010"

If a campaign in this world is intended to have a high level of interaction, it is recommended that the campaign simply uses the world's namespace as default, and stores any information that it needs to be kept separate in a child namespace of the format "(campaign_name)".
Building on the above example, if upthorn made a "foo" campaign in the "hypothetical" world, with high level of intended interactivity with the world, it would use "upthorn_hypothetical" as its default namespace, perhaps occasionally storing information in "upthorn_hypothetical.foo".

If a campaign in this world is intended to have a medium level of interaction, it is recommended that the campaign creates a child namespace of the format "(campaign_name)" in the world's overall namespace, and uses it as default, accessing the parent namespace when it needs to get information from the overall world.
Continuing with the above examples, if upthorn made a "bar" campaign in the "hypothetical" world, with mid level of intended interactivity with the world, it would use "upthorn_hypothetical.bar" as its default namespace, and somewhat frequently access information in "upthorn_hypothetical", by using the namespace "^" or explicitly naming "upthorn_hypothetical".

If a campaign in this world is intended to have a low level of interaction, it is recommended that the campaign uses an entirely separate namespace, of the format "(world_namespace)_(campaign)" as its default, and accesses the overall world's namespace when it needs to communicate with the world.
Continuing with the above examples, if upthorn made a "fnord" campaign in the "hypothetical" world, with a low level of intended interactivity with the world, it would use "upthorn_hypothetical_fnord" as its default namespace, and occasionally access information in "upthorn_hypothetical", by explicitly naming "upthorn_hypothetical".

===== Developed Sequentially =====
In cases where the first campaign is intended as a standalone, but some time after its release the same author, or other authors decide to make other campaigns in the same overall continuity, the first campaign's namespace cannot be changed.

If campaigns are intended to have a high level of interaction, it is recommended that the later campaign uses the namespace of the prior campaign, and makes a child namespace for itself 'if' it needs to save any of its variables separately.
As an example, if an author going by "upthorn" developed a campaign "foo" and later developed a related campaign "bar", the "bar" campaign would use "upthorn_foo" as its default namespace, and perhaps occasionally store information in "upthorn_foo.bar"

If campaigns are intended to have a medium level of interaction, it is recommended that the later campaign makes its default namespace a child inside the namespace of the prior campaign that it is most heavily realted to, and accesses the parent namespace when it needs to work with variables from the prior campaign(s).
Working from the above example, upthorn's "bar" campaign would use "upthorn_foo.bar" as its default namespace, and somewhat frequently access the "foo" campaign's information in "upthorn_foo", either by setting the namespace attribute to "upthorn_foo" explicitly, or by using "^" to get "upthorn_foo.bar"'s parent.

If campaigns are intended to have a low level of interaction, it is recommended that the later campaign(s) has its own separate namespace using the "Single Standalone Campaign" guideline format, and explicitly name the prior campaign(s)'s namespace(s) for the few times it needs to access them.
Working from the above examples, upthorn's "bar" campaign would use "upthorn_bar" as its default namespace, and occasionally access the "foo" campaign's information in "upthorn_foo", by explicitly naming "upthorn_foo" as the namespace.

SummerOfCodeProposal JodyNorthup

2010-04-17T11:18:05Z

Upthorn: /* Mandatory */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables.
#* events and syntax
#** bulletproof singleplayer implementation of WML persistence events
#** multiplayer implementation of WML persistence events
#*** error recovery in MP in case players drop/join.
#*** transaction blocks for global variable operations.
#** Wiki documentation of events and syntax
#* present new WML capabilities to UMC authors on wesnoth's forums.
# User ability to manage persistence data
#* Backup
#* Restore
#* User Interface (probably modified from "save game" or "load game" dialog.
#* wiki documentation/manual page

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-04-17T11:15:23Z

Upthorn: /* Project Details */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==Deliverables==
===Mandatory===
# WML extension to provide events for persistent variables.
#* set_global_variable
#* get_global_variable
#* clear_global_variable
#* Wiki documentation of events and syntax
#* bulletproof singleplayer implementation of WML persistence events
#* multiplayer implementation of WML persistence events
#** error recovery in MP in case players drop/join.
#** transaction blocks for global variable operations.
#* present new WML capabilities to UMC authors on wesnoth's forums.
# User ability to manage persistence data
#* Backup
#* Restore
#* User Interface (probably modified from "save game" or "load game" dialog.
#* wiki documentation/manual page

===Optional===
# gamemaster mode multiplayer
#* UI
#** button, checkbox, or list option somewhere in the multiplayer interface
#** ???
#* Backend
#** Separate gamemaster persistence from standard SP and MP persistence
#*** SQL storage backend?
#*** player profiles?
#* Wiki documentation/manual page
# Player profiles
#*UI
#**a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
#**a select profile dialog
#***a list of profiles that currently exist
#***an option to create a new profile
#**dialog allowing the user to specify a profile name
#***option to migrate previous (non-profiled) data into new profile
#****dialog allowing player to select which savegames to migrate
#*Backend implementation
#**Append profile name as subdirectory to user_data directory path.
#**Save selected profile to config on exit (or profile selection)
#**Load selected profile from config on startup
#*Wiki documentation/manual page
# SQL backend for global variable storage
# First massively multiplayer campaign
#* Collaborate with UMC authors and playtesters from Wesnoth's MP community
#* ???

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

SummerOfCodeProposal JodyNorthup

2010-04-13T08:52:27Z

Upthorn: /* Basic Syntax */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]
and
[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T20:06:42Z

Upthorn: /* Uniqueness */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

====Importance of Uniqueness====
Multiple addons can possibly have the same name if at least one does not come from the addon server. So the addon name alone is insufficient. It is important that a namespace is globally unique, because even if it is not possible for two addons with the same name to be installed and operating at the same time, a player may wish to retain persistence data when uninstalling an addon.
If a player uninstalls an addon, retains persistence data, and then installs a different addon with the same name, that addon should probably not be loading the old one's persistence data, and almost certainly should not be overwriting it.

Consider this cautionary tale:
=====Example A =====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo and plank::foo both use "foo" as the namespace for global variables, because it is the name of the addon, and two addons with the same name cannot be installed at the same time.
Adam, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Adam knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Adam fully intends to return to bar::foo at some point, and so leaves the persistence data intact. Furthermore, he does not realize that both addons use the namespace "foo", or does not understand how the namespace is relevant to the file choice, and does not think to back up his persistence data from bar::foo.
Adam installs plank::foo, and starts a game with it.
plank::foo unintentionally loads the persistence data from bar::foo, and because some of plank::foo's persistent variables coincidentally have the same names as some of bar::foo's persistent variables, right from the beginning, Adam gets confusing messages referring to events in plank::foo that haven't happened yet.
Adam decides he does not like plank::foo, and so uninstalls it and reinstalls bar::foo. Adam starts a game with bar::foo, and discovers that, because of corruption from plank::foo, he has lost access to several of the unlockable units.

=====Example B=====
Addon "foo" by author "bar" (henceforth referred to as "bar::foo") has persistence data.
Addon "foo" by author "plank" (henceforth referred to as "plank::foo") also has persistence data.
bar::foo uses the namespace "foo_bar", because it is recommended that wml authors use their own handle in addition to the addon title, to avoid collision.
plank::foo violates spec and simply uses "foo".
Brian, a user, has played bar::foo extensively, likes it well, and has unlocked a number of special units in it, but has heard that plank::foo is well made, and would like to try it out.
Brian knows that two addons with the same name cannot be installed happily at the same time, and so he uninstalls bar::foo. Brian fully intends to return to bar::foo at some point, and so leaves the persistence data intact.
Brian installs plank::foo, and starts a game with it.
Brian discovers that plank::foo is a lovingly rendered campaign with a cohesive story and an engaging metanarrative thread tha reveals itself on subsequent playthroughs.
After many happy hours with plank::foo, Brian has done all there is to do in it, and seen all there is to see in it, and decides to return to bar::foo to finish rounding out his unlockables/achievements.
Brian uninstalls plank::foo and reinstalls bar::foo, and starts a game with bar::foo. Brian sees that all the data is intact and happily resumes his progress from just where he left it.

===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T18:32:09Z

Upthorn: /* Standard Multiplayer */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.
===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" during [get_global_variable] will be loaded from the client of the initial host player. If the initial host player has been dropped from the connection, they will read from the current host. Persistent variables with "side=global" during [set_global_variable] will be stored to all players. Persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side.

===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T14:01:42Z

Upthorn: /* Namespace Format */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
===Uniqueness===
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle), to maximize uniqueness.
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.
===Filesystem===
Because (the first section) of a namespace will be the cfg file name, all characters in a namespace must be filename safe. That is, they cannot include '*', '?', ' ', '/', '\', '<', '>', '|', or '"'. Additionally, '.' will be interpreted specially, so WML authors should be aware of that.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T13:53:24Z

Upthorn: /* Namespace Format */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
Tentatively, I am suggesting that namespaces be of the format (addon-name)_(author-handle)
For example, (judging by info in its _main.cfg) the "Love to Death" UMC campaign would use the namespace "Love_to_Death_Geos_PP2_etc".
However, this is extremely open to suggestion and revision.

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T13:26:40Z

Upthorn: /* Persistence Config Format */

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
TODO: Determine suggested format for namespace to minimize chances of collision. 

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. I believe it would be useful to provide facilities for further structure, allowing [variables] blocks to be enclosed within further tags.
====Example====
[variables]
scalar="value"
[some_container]
someint="42"
somestring="m-theory"
[/container]
[/variables]
[arbitrary_namespace]
[variables]
thing="boring"
[/variables]
[/arbitrary_namespace]
[pirates]
[ninjas]
[variables]
ninjapirate="robotzombie"
[/variables]
[/ninjas]
[/pirates]
====Proposed Syntax====
In order to allow WML authors to structure their persistence file in an arbitrarily defined manner, I suggest an extension to the parsing of the namespace attribute in global_variable tags: . should be used to denote sub-namespace access in the same way that, in variables, . denotes access to member variables of a container. This will allow WML authors to keep their variables organized in specific ways that make sense to them, as well as allowing for reuse of variable names in different contexts.
=====An example=====
[set_global_variable]
from_local=boring
to_global=thing
namespace="filename.arbitrary_namespace"
side=1
[/set_global_variable]
=====Special Namespaces=====
Obviously, "variables" and any other automatically generated tags would have to be reserved, and therefore invalid as sections in the namespace attribute. However, we could also provide keyword namespaces that have specific effects on their contained variables. For instance, as a an example the [single_player] namespace could be provided to specify variables that are not to be used during multiplayer.

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T12:30:19Z

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
TODO: Determine suggested format for namespace to minimize chances of collision. 

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. Further structure may be defined in the future. It is likely that the [variables][/variables] block(s) will be nested within some other tags to specify what aspect of the addon the persistent data pertains to.
===Example===
[campaign]
id="somecampaign"
...
[scenario]
id="scenario1"
...
[variables]
...
[/variables]
[/scenario]
[scenario]
id="scenario2"
...
[variables]
...
[/variables]
[/scenario]
...
[/campaign]
===User Defined vs Automatic===
Should the persistence configuration structure be user-definable, or automatically generated? If it is user definable, what syntax should be used to denote blocks and block names?
If it is automatically generated, how does wesnoth determine what block any given variable should go in?

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T06:41:43Z

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619 Patch 1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

and/or design and implement solutions for other contexts of multiplayer games. (E.g. dedicated server mode, game-master mode, peer-to-peer or 'normal' mode)

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
TODO: Determine suggested format for namespace to minimize chances of collision. 

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. Further structure may be defined in the future. It is likely that the [variables][/variables] block(s) will be nested within some other tags to specify what aspect of the addon the persistent data pertains to.
===Example===
[campaign]
id="somecampaign"
...
[scenario]
id="scenario1"
...
[variables]
...
[/variables]
[/scenario]
[scenario]
id="scenario2"
...
[variables]
...
[/variables]
[/scenario]
...
[/campaign]
===User Defined vs Automatic===
Should the persistence configuration structure be user-definable, or automatically generated? If it is user definable, what syntax should be used to denote blocks and block names?
If it is automatically generated, how does wesnoth determine what block any given variable should go in?

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T06:07:49Z

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====

Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, preliminary specification for persistence handling is complete for single player purposes.
*May 24 - June 2: Implement prototype solution for single player context.
**Milestone: By June 2, SP mode handles persistence data accurately to preliminary spec.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By July 4th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
**Milestone: By July 12th (Midterm), there is a complete specification addressing all concerns for standard multiplayer.
*July 13 - July 27: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By July 27th, persistence data works and syncs in standard MP mode to preliminary spec.
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By August 16th, multiplayer handling of persistence data is satisfactory to majority of players and developers.

Optimistic:
*Now - May 23: Design and implement preliminary specification while researching the internal workings of scenario/campaign design, and discussing with mentors and UMC developers to determine needs.
**Milestone: By May 23rd, SP mode handles persistence data accurately to preliminary spec.
*May 24 - June 7: Refine solution through iterative design and implementation with continuous discussion with community members.
**Milestone: By June 7th, specification for persistence handling in SP contexts is finalized and implemented accurately.
*June 8 - June 13: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*June 14 - June 28: Implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading.
**Milestone: By June 28th, persistence data works and syncs in standard MP mode to preliminary spec.
*June 29 - July 12: Refine solution through iterative design and implementation with continued discussion.
**Milestone: By July 12th (Midterm), specification for standard MP handling of persistence data is finalized and implemented.
*July 13 - July 27: Discuss with player, software dev, and UMC dev community to determine details about needs for gamemaster mode.
*July 28 - August 11: Implement proof of concept gamemaster mode.
**Milestone: By August 11th, game master mode is functioning.
*August 11 - August 16: Refine game master mode iteratively via mentor, player, and UMC dev feedback.
**Milestone: By August 16th (final), game master mode is satisfactory to player needs.

and/or design and implement solutions for other contexts of multiplayer games. (E.g. dedicated server mode, game-master mode, peer-to-peer or 'normal' mode)

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
TODO: Determine suggested format for namespace to minimize chances of collision. 

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. Further structure may be defined in the future. It is likely that the [variables][/variables] block(s) will be nested within some other tags to specify what aspect of the addon the persistent data pertains to.
===Example===
[campaign]
id="somecampaign"
...
[scenario]
id="scenario1"
...
[variables]
...
[/variables]
[/scenario]
[scenario]
id="scenario2"
...
[variables]
...
[/variables]
[/scenario]
...
[/campaign]
===User Defined vs Automatic===
Should the persistence configuration structure be user-definable, or automatically generated? If it is user definable, what syntax should be used to denote blocks and block names?
If it is automatically generated, how does wesnoth determine what block any given variable should go in?

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup

SummerOfCodeProposal JodyNorthup

2010-04-12T04:16:47Z

{{SoC2010Student_2|upthorn|SoC Ideas Persistent Gameworld}}
[[Category:SoC Ideas Persistent Gameworld]]

=Description=
====Jody Northup - Implement gameworld data persistence====
This is the [[SoC_Ideas_Persistent_Gameworld|Persistent Gameworld]] idea. 
If accepted, I would communicate extensively with the community to determine a solution that provides gameworld persistence with the greatest ease of access and implementation for the established community of world and campaign builders for Wesnoth. 
Currently, I expect that this will involve extending WML, and creation of some system of communication about the extended WML for multiplayer games.

=IRC=
Upthorn, Upth

=SoC Application=
[http://socghop.appspot.com/gsoc/student_proposal/review/google/gsoc2010/upthorn/t127026899399 Upthorn - Data saving for persistent game worlds]

=Questionnaire=
===Basics===

====Write a small introduction to yourself.====
My name is Jody Northup. I am a 25 year old college student from the United States. Under normal circumstances, I should have graduated by now, but there have been a number of obstacles on my route to academic success.

I have been programming since I was introduced to C++ in the summer of 1999.

====State your preferred email address.====
upthorn@gmail.com

====If you have chosen a nick for IRC and Wesnoth forums, what is it?====
*irc: Upthorn
*irc alternate: Upth
*Wesnoth forums: Upthorn
*gna.org: Upthorn

====Why do you want to participate in summer of code?====
It is a good opportunity to gain experience in aspects of programming that aren't covered in classes, it is a great way to contribute to the open source community on projects that I believe in or even make use of myself, being able to contribute usefully to something gives me a great sense of accomplishment, and it is much more fun than any other summer job opportunity I know of.

====What are you studying, subject, level and school?====
I'm an undergrad with an undeclared major at Sacramento City College. In terms of credit earned, I am in my third year, in terms of time spent I am in my 5th year, and in terms of the length of time remaining until I graduate, I might as well just be starting my 2nd year.
In another year or two I hope to transfer to the University of Maryland, where I plan to double-major in Computer Science and Linguistics, getting a simultaneous degree in both if possible.

====What country are you from, at what time are you most likely to be able to join IRC?====
I live on the west coast of the United States (GMT-7 during the summer), but my sleep schedule is very flexible. I am generally available on between around 19:00 and 11:00 GMT, but I can change this during the coding period if it is inconvenient to my mentor. Until my class period ends, I will be completely unavailable on Wednesdays.

====Do you have other commitments for the summer period ? Do you plan to take any vacations ? If yes, when.====
I may be unavailable from 23:00 Wednesday to 19:00 or later on Thursdays (GMT), but this commitment can be cancelled if there is urgent work to be done.
Additionally, I would like to take up an additional part-time job during the summer if I can fit one into the schedule, but there are no definite plans around this at the moment.
I forgot to mention this earlier, but I plan to enroll in a math class during summer session, I do not yet know what schedule it will be, but it should not interfere greatly.

===Experience===

====What programs/software have you worked on before?====
* A variety of programming projects for high school and college classes
* I have hacked around quite a bit on open source emulators. Primarily the [http://code.google.com/p/gens-rerecording/ rerecording branch of gens] (see [http://tasvideos.org/WelcomeToTASVideos.html#How TASVideos.org introduction] and these [http://tasvideos.org/forum/viewtopic.php?p=77913#779136 forum] [http://tasvideos.org/forum/viewtopic.php?t=7465 threads] for more information, with some work also being done on snes9x and mupen64.
* I have done a great deal of feature programming and debugging work on the [http://info.sonicretro.org/Sonic_Megamix Sonic 1 MegaMix] romhacking project for the sega genesis.
* I have done some simple reverse engineering work to examine a simple archive file format and help create a translation patch for the Japanese indie game Bunny Must Die.
* I have done some minor testing and debugging work on a project to encapsulate windows games in a deterministic environment, allowing input to be recorded and replayed without concern for RNG input from the system clock, or other sources.
* I worked on ScummVM for Summer of Code last year, implementing an API to allow engine modules to negotiate with backend modules to determine color depth.

====Have you developed software in a team environment before? (As opposed to hacking on something on your own)====
Yes, I frequently collaborate with other developers in my development of gens, and Sonic 1 Megamix, as well as having worked in pair and group projects for classes. And, of course, I contributed to ScummVM for last year's Summer of Code.

====Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?====
I participated in last year's summer of code as a student for ScummVM. My project was [http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/scummvm/t124024841147 providing support for RGB color games]. I was successful, but had a great snag towards the end as I completed my project as proposed before the midterm, went on a pre-arranged 2-week vacation, and had a great deal of difficulty with the project I was assigned to work on for the remainder of the summer of code period.

====Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.====
I have been the project maintainer and coordinator of the Gens-rerecording project since about mid-2006, having produced many new features, created a google code project page, and organized the development of many more during that time.

====Gaming experience - Are you a gamer?====
Yes, I enjoy games, although lately I haven't been playing them as much as I used to.

====What type of gamer are you?====
I am an obsessive perfectionist when it comes to gaming. I always try again and again until I get the best possible outcome, and I've collected all the items. Sometimes the frustration from this will drive me to save-scum, or edit my saves outright.

====What type of games?====
I love all types of games, from real time strategy (command & conquer, starcraft, total annihilation) and turn-based strategy (wesnoth, civilization) to RPGs (Planescape: Torment, Skies of Arcadia), adventures (King's Quest, Space Quest, The Void/Тургор), platformers (Sonic the Hedgehog, Mario) and racing games (Virtua Racing, Burnout) and even to First person shooters (Deus Ex, Unreal Tournament) and simulations (Escape Velocity, Dwarf Fortress)

====What type of opponents do you prefer?====
I prefer opponents that are on the same level or slightly better than me (so, generally, ones that are not very good).

====Are you more interested in story or gameplay?====
I cannot choose between story and gameplay. In the best games, there is no separation between the two, but either can hold up a good game on its own.

====Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.====
Yes, I got hooked on wesnoth during my vacation last summer, and the Heir to the Throne campaign became one of the greatest difficulties I had in accomplishing additional work for ScummVM.
I am not very good at strategy games, so I have not even attempted multiplayer yet.

====If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for GSoC. If you have gained commit access to our SVN (during the evaluation period or earlier) please state so.====
* I did submit a patch to correct the VC9 project files, but then I realized it was a duplicate of one from three weeks earlier.
* [https://gna.org/patch/index.php?1612 Patch 1612] -- implements [http://www.wesnoth.org/forum/viewtopic.php?f=15&t=15880 Sapient's proposed changes to allied healing].
* [https://gna.org/patch/?1619] -- provides prototype implementation of [[SoC Ideas Persistent Gameworld#WML_Syntax|Crab's proposed WML syntax]] for saving/loading persistence data.

===3) Communication skills===

====Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.====
I am a native English speaker, and my proficiency with written English is better than average.

====What spoken languages are you fluent in?====
I am a fluent native speaker of English, can communicate alright in Japanese, and am at a basic level in Mandarin and Korean. I hope to learn Russian next year.

====Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.====
I do not know about wesnoth players specifically, but I am fairly good at interacting with players of other games. I tend to be friendly and open, and I understand that the other players' insults and other such things are just for fun and don't mean anything.

====Do you give constructive advice?====
I do give constructive advice whenever I can, although sometimes I think I cause offense by mistake. I always try to think my criticism through so that there is a clear indication of how the aspect could be improved.

====Do you receive advice well?====
I believe so, as long as it is clear and helpful.

====Are you good at sorting useful criticisms from useless ones?====
I think I am pretty good at sorting helpful criticism from [http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING bikeshed painting], yes.

====How autonomous are you when developing ? Would you rather discuss intensively changes and not start coding until you know what you want to do or would you rather code a proof of concept to "see how it turn out", taking the risk of having it thrown away if it doesn't match what the project want====
I like to keep a good balance between discussing changes and coding proof of concepts. In last year's summer of code, I had to interact extensively with the development community in order to design a useful API for them, but at times when it wasn't clear what the community preference was, I would make a proof of concept to demonstrate the features and drawbacks of a given method.
Some of the work would be thrown out, but I did my best to maximize the amount of work that could be reused on a different method.

===Project===

====Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?====
I did select a project from your list. Specifically, I selected [[SoC_Ideas_Persistent_Gameworld|the persistent gameworld]] project.
I would like to concentrate on the necessary modifications to WML to make it work in single player, and add multiplayer support as time permits.

====Why did you choose this project?====
It seemed like something that I am farely likely to be able to accomplish at my current skill level, but is large enough that I shouldn't have to worry about completing it too early as happened last summer.

====Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".====
Pessimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design.
*May 24 - June 2: Discuss with mentors and UMC developers to determine precise needs of world persistence project
*June 3 - July 12: Design a solution that addresses needs of UMC developers
*July 13 - August 16: Implement solution for saving and loading in single player context.

Optimistic:
*Now - May 23: Research the internal workings of WML, and scenario/campaign design, discuss with mentors and UMC developers to determine needs.
*May 24 - June 2: Design and implement proof of concept solution for this in single player context.
*June 3 - July 4: Refine solution through iterative design and implementation with continuous discussion with community members.
*July 5 - July 12: Discuss with player, software dev, and UMC dev community to determine details about needs for world persistence in multiplayer context.
*July 13 - July 27: Design and implement proof of concept solution for addressing multiplayer needs in persistence data saving/loading
*July 28 - August 16: Refine solution through iterative design and implementation with continued discussion and/or design and implement solutions for other contexts of multiplayer games. (E.g. dedicated server mode, game-master mode, peer-to-peer or 'normal' mode)

====Include as much technical detail about your implementation as you can====
I have relocated this content to [[#Project Details|Project Details]] for ease of organization.

====What do you expect to gain from this project?====
Experience with custom scripting language implementations, understanding of how to process XML-style languages efficiently, a sense of having been able to do something useful for someone besides myself, and a nifty t-shirt from google.

====What would make you stay in the Wesnoth community after the conclusion of SOC?====
My interest in the project itself, and any relationships I might forge with community members during the coding period.

===Practical considerations===

====Are you familiar with any of the following tools or languages?====

* Subversion (used for all commits)
Yes, I have used it extensively for gens development, megamix development, and my work on ScummVM last summer.
* C++ (language used for all the normal source code)
Yes, I have been working with C++ for nearly 11 years now.
* STL, Boost, Sdl (C++ libraries used by Wesnoth)
I am fairly familiar with the STL, and worked almost exclusively with SDL on ScummVM last summer. I am new to Boost, but in the past I have been able to catch on to these things fairly quickly with a little bit of practice.
* Python (optional, mainly used for tools)
I have worked with python somewhat, contributing a few minor bugfixes to the Civilization 4 mod "Fall From Heaven II"
* build environments (eg cmake/autotools/scons)
I do not have any experience with cmake, autotools, or scons.
* WML (the wesnoth specific scenario language)
I have some minor experience with WML, from editing my own wesnoth savegames last summer.
* Lua (used in combination with WML to create scenarios)
I have experience both in scripting Lua, and in working with the Lua C API to provide hooks and bindings.

====Which tools do you normally use for development? Why do you use them?====
*C/C++: I generally use Microsoft Visual C/C++, because I am familiar with it from classes, and because it provides a very useful debugger that allows me to trace problems to their source quite quickly. I have used Dev-C++ in the past, though.
*Java: Eclipse, because it was recommended to me and I do not know anything about the others, and also because it provides a very useful debugger facility.

====What programming languages are you fluent in?====
C, C++, Lua, Java, Javascript.

====Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail. If you are willing to do so, please do list a phone number (including international code) so that we are able to contact you. You should probably *only* add this number in the application for you submit to google since the info in the wiki is available in public. We will *not* make any use of your number unless some case of "there is no way to contact you" does arise!====
I have provided a telephone contact in [http://socghop.appspot.com/gsoc/student_proposal/show/google/gsoc2010/upthorn/t127026899399 my SoC application.]

=Project Details=
==Overview==
I will extend WML to provide for events that can save persistent data about what has occured in a scenario or campaign. This data will be accessible to the scenario and campaign in all future plays from the same machine (or related ones that ask for it by name), even if all normal savegames have been deleted. If I have time remaining once this is completed, I will attempt to extend this functionality into mult-player contexts in a meaningful way. 

==WML Syntax==
===Basic Syntax===
Crab has suggested
[get_global_variable]
side=1
namespace=my_addon
from_global=my_variable_name
to_local=foo
[/get_global_variable]

[set_global_variable]
side=1
namespace=my_addon
from_local=foo
to_global=my_variable_name
[/set_global_variable]

[clear_global_variable]
side=1
namespace=my_addon
global=my_variable_name
[/clear_global_variable]

====Additions====
=====Keyword: global=====
side=global
Allows wml author to specify that the variable is not associated with a specific side, and is to be loaded from host during multiplayer.

=====Default namespace=====
namespace=my_addon
May be placed in addon cfg files to specify a default namespace for global variables set by this campaign. This namespace will be used if a "namespace=" line is not specified in [*_global_variable] tags.

===General Thoughts===
Crab's syntax for WML events seems reasonable, so I will be using it as the base on which to build. Currently, to set a persistent variable with a constant value, it is recommended to make a temporary variable with that value, and then set the global variable from that. However, syntax may be defined later to directly set a global variable with a constant value. 
In general, any deviation I make from crab's suggested syntax will be done in order to make persistent "global variables" work more like regular WML variables from the standpoint of a WML author.
==Namespace Format==
TODO: Determine suggested format for namespace to minimize chances of collision. 

==Storage==
===Single Player===
Persistent data will be stored in "(user data directory)/persist/(namespace).cfg".

===Standard Multiplayer===
Persistent data will be stored locally to each player. Persistent variables with "side=global" will be stored and loaded from the client of the initial host player, and persistent variables that are side-specific will be stored and loaded from the client of the player that has taken the specific side. 
===Dedicated Game Master Multiplayer===
In the case of a dedicated game master, persistent variables from all sides will be stored and loaded from the game master. However, due to the nature of game master mode, it is unlikely that the standard single-player storage location will be sufficient.
TODO: Define storage location for game master.

==Persistence Config Format==
===Overview===
Internally, persistence cfg data will primarily be collections a [variables][/variables] block surrounding valid [[VariablesWML#Kinds_of_Variables|variable definitions]]. Further structure may be defined in the future. It is likely that the [variables][/variables] block(s) will be nested within some other tags to specify what aspect of the addon the persistent data pertains to.
===Example===
[campaign]
id="somecampaign"
...
[scenario]
id="scenario1"
...
[variables]
...
[/variables]
[/scenario]
[scenario]
id="scenario2"
...
[variables]
...
[/variables]
[/scenario]
...
[/campaign]
===User Defined vs Automatic===
Should the persistence configuration structure be user-definable, or automatically generated? If it is user definable, what syntax should be used to denote blocks and block names?
If it is automatically generated, how does wesnoth determine what block any given variable should go in?

==Handling Of Eras==
Era add-ons will also be allowed to save persistence data, but it has not yet been determined how era persistence and scenario persistence will interact.

See below for detailed consideration of this:
==Addon interaction==
If a user runs a campaign that saves persistence data with an era active, which of the following options should be the result?
===Continue as normal===
The campaign doesn't know or care that the era is active, and loads and saves persistence data as usual.

This is the easiest to implement, but seems like it has potentially undesirable implications for players and UMC developers.
===Disable persistence functionality===
Campaigns have their persistence functionality disabled while the era is active.

This option preserves data integrity, but seems overly limiting. Additionally, this seems to shut eras out of the whole world persistence scene. Not a good solution.
===Second set of data===
Campaign loads and saves a different second set of persistence data.

This preserves data integrity while maintaining flexibility, but comes with a whole host of additional considerations:

====On a campaign's first run under the active era, what should happen?====
#Campaign copies existing persistence data into second set.
#Campaign saves and loads an entirely new version of persistence data for use with this era.
My gut instinct says that 2 is the better option, but I can easily imagine a significant set of players being upset with either decision. Perhaps we should prompt the player and allow them to decide.

====Where do we store the second set of data?====
=====(user_data_directory)\persist\(era_id)\(namespace).cfg=====
This is a simple solution, but it may cause tons of clutter in the user's persistence data directory.
=====The same persistence .cfg file as normal, but enclosed in an [era][/era] block.=====
I like this option better, but it brings its own set of questions -- what about persistence data that the era wants to save, does it get enclosed into [campaign][/campaign] blocks? If so, how do we know what files to put [era][/era] in and what ones to put [campaign][/campaign]?
I propose that persistence .cfg files don't care what type of addon made them, and may have any number of [campaign][/campaign] blocks in them. If an era is active, it should make an [era][/era] block at the level between [campaign][/campaign] and [scenario][/scenario].

====What if era is irrelevant to the purpose of a persistent variable?====
For example, a variable that just keeps track of how many times this campaign has been played total?
=====User specifies interaction=====
Provide some method allowing campaign authors to specify whether any given variable should be duplicated when eras are active.
This would require syntax to be defined for [*_global_variable] blocks.
I suggest an optional "era=" line, with the following effects:

era=all
There is only one copy made of this variable, all eras use it.

era=any
This is the default if the line is not provided. A copy of this variable is made for each era that this campaign is used with.

Additionally, this can be extended to allow for
era="era_name","era_name2"
which would allow campaign authors to specify global variables that only exist in certain eras, thereby allowing an easy path for UMC developers to build in era/campaign interactions.

Of course, there are plenty of other options we could take to allow user-specified era/campaign interaction, and this option would practically necessitate the addition of a "campaign=" for era addons to use.
=====Don't worry about it, it's not that important=====
Allow this one instance of eras interacting poorly with campaign persistence data, since it isn't ever going to be that big a deal if an era-campaign config keeps a separate play count from the campaign alone. And players can always edit the persistence files without much difficulty if it troubles them.
==Optional Subtasks==
===User profile===
Add user profiles, so that multiple users can share the same wesnoth installation without having to share persistence data and savegames.

For this we will need a
====UI====
*a menu option to select profile (perhaps a "tab" in preferences, along with graphics, sound, etc)
*a select profile dialog
**a list of profiles that currently exist
**an option to create a new profile
*dialog allowing the user to specify a profile name
**option to migrate previous data into new profile
====backend====
*Append profile name as subdirectory to user_data directory path.
*Save selected profile to config on exit (or profile selection)
*Load selected profile from config on startup