WML Engine Rewrite

Following an irc discussion with the game developers, I have decided to change my project. See the original wiki page, Map Editor Ilor, for my questionaire answers.

Wesnoth Markup Language (WML) engine overhaul.

Abstract

WML is a core element of Wesnoth. This custom markup language is used for various data files, save games, and network handling. It s a text format, internally stored in the form of "config" objects, and used throughout the entire Wesnoth code for many different purposes. This current representation is, however, rather inefficient, both in terms of memory consumption and processing time. Additionally, some parts of game code dealing with processing WML nodes are written in a crude way that is inefficient and difficult to maintain, notably the main WML event processing function. The aim of this project is to provide a replacement implementation of the WML engine, with better speed and lower memory consumption, as well as refactoring many of the main WML-related functions.

Description

The inefficiency of WML has been a major issue in wesnothd, the dedicated Wesnoth server. An alternative, stripped down implementation called simple_wml has been introduced there recently by Sirp, and is several times faster and uses less memory. However, while it shows how inefficient the current implementation is, simple_wml is not a suitable replacement for config because it lacks a great deal of existing functionality, and is difficult to use. The replacement WML implementation will have to be mostly compatible with the current config class, to avoid making a huge amount of changes throughout the entire codebase using WML. Some will be needed anyway, but if possible they should be simple enough to automate or semi-automate.

There are some design decisions to be made regarding technical details of internal WML representation. It should be made easy to replace some crucial bits of the implementation so that different approaches can be compared side by side later, when the project is nearing completion.

Timeline

The project milestones would be as follows:

• Refactor the WML commands handling code. Design can take place before the actual beginning of GSoC, the implementation itself should be relatively easy if somewhat time-consuming. I expect this part to be done and tested around half of June.
• Design and implement a method of benchmarking WML performance speed- and memory-wise. This will be important as improved efficiency is a main goal of the project, and since the actual game performance is influenced by many factors, a standalone benchmark will be very useful. Additionally, the benchmarking code can serve as a reasonably comprehensive test suite for the new WML implementation. This part should not take much time, and more tests can always be added later. Most likely the development of the benchmark will be parallel to the next item.
• Design and implement the new internal representation of WML documents and nodes. This is the main part of the project, and will take most of the time. A working, possibly incomplete and not yet optimized version should be ready for the mid-term evaluation, i.e. second week of June
• The final part of the project will involve further optimizations of the code, performing benchmark tests and polishing the interfaces. Then the code will be wired into the Wesnoth codebase, which should be possible to do as automatically as possible. Some final tests of the entire game with the new WML code will be done to see what gains have been made. By the time GSoC ends, the new WML implementation should replace or be ready to replace the current config implementation.