Maintenance tools
The Wesnoth source code distribution includes a couple of tools intended to help authors maintain campaigns, faction & unit packs, and other WML resources. These are:
- wmlscope
- a cross-reference lister, useful for finding unresolved macro and resource-file references.
- wmllint
- a utility for sanity-checking WML syntax and porting your old WML to the current version of WML.
- wmlindent
- a utility for reindenting WML to a uniform style.
You will need a Python 2 interpreter on your system to use these tools. Linux, *BSD, and Mac OS/X should already have Python 2 installed; for Windows it's a free download from http://www.python.org. You will also need to know how to run command-line tools on your system.
If you're working with ubuntu you might have to install the package wesnoth-1.10-tools (or the convenient version).
sudo apt-get install wesnoth-1.10-tools
All three tools will require you to supply a directory list. This is a set of directories containing the WML files you want to work on.
This page is intended as ducementation for users. A developer's-eye discussion of the design constraints on these tools, and their limitations, can be found here [1].
Note to Windows Users: This means you have to run it from the Command Line. The command line may be reached by hitting Start, then Run, then "cmd" or "command" depending on your version of Windows.
Example uses:
python wmllint path\to\files python wmlindent path\to\files
Another example:
"C:\Program Files\Python2.4\python.exe" data\tools\wmllint --dryrun data\core data\{multiplayer,themes} data\campaigns
(You have to specify the full directory path to the executable if you don't have your environment variables set up correctly). The first thing you type is the path to your python executable, followed by a space. The second thing you type is the path to the desired script to run, followed by a space. The third thing you type is the path to the folder (or file) to be processed.
A convenient way of running wmllint on Linux (Debian Lenny) and Windows (Xp) in comparison, Linux:
Assuming we're working with wesnoth 1.9 or more advanced versions (1.10 in this case).
python2 /usr/share/games/wesnoth/1.10/data/tools/wmllint --dryrun /usr/share/games/wesnoth/1.10/data/core ~/.local/share/wesnoth/1.10/data/add-ons/A_Simple_Campaign 1>wmllint-run.log 2>wmllint-err.log
I have these commands inside of a file named
wmllint_dryrun_ASC.sh
and execute it by opening a shell (=terminal, console, command window, bash,...), navigating into the directory with that file and typing
bash wmllint_dryrun_ASC.sh
The python2 command should be automatically known on Debian. The path to the script tells the python interpreter what to execute. --dryrun: A wmllint option, see below. The path to the core files is needed to let wmllint know about e.g. defined core units, followed by the path to the add-on that shall be checked; the last two commands cause the result of the wmllint usage to be written into those files in the same directory as the script. Windows, this is logically exactly the same as the Linux shell script above, so if you are on a Mac you can probably conclude how you need to adapt the paths:
E:\Python26\python.exe E:\Programme\Wesnoth_1.8_svn\data\tools\wmllint --dryrun E:\Programme\Wesnoth_1.8_svn\data\core E:\Programme\Wesnoth_1.8_svn\userdata\data\add-ons\A_Simple_Campaign 1>wmllint-run.log 2>wmllint-err.log
This is the content of a .txt file, whose extension I rename to .bat and double-click onto it. Opening a command window is not needed this way. Since python isn't natively installed on windows and I don't have environment variables set, the full path to python.exe is given. If your directories contain spaces it may help to include the path in quotes:
"C:\Programs\Battle for Wesnoth 1.8\data\tools\wmllint"
Remember that you do not need to enter all of the commands/paths at once. If it doesn't work, start with only "python" or "C:\Python26\python.exe" or the like and interpret the error messages that you get. If you get an "unknown command", python isn't installed or environment variables aren't set correctly. After that, you can add the later commands one by one.
wmlscope
The main use for wmlscope is to find WML macro references without definitions and references to resource files (sounds and images) that don't exist. These are difficult to spot from in-game because they usually result in silence or a missing image rather than actual broken game logic. They may happen because of typos in your WML, or because the name of a macro or the location of a resource file changed between versions of the game.
wmlscope also checks macro invocations for consistency. It will complain if a macro is called with the wrong number of arguments. In most cases it can deduce information about the type of the literal expected to be passed to a given macro argument by looking at the name of the formal.
| Type | Meanining | Formals requiring this type | Literals of this type |
|---|---|---|---|
| side | a single side number | SIDE, *_SIDE, SIDE[0-9] | a numeric or "global" |
| numeric | a numeric integer literal | SIDE, X, Y, RED, GREEN, BLUE, TURN, PROB, LAYER, TIME, *_SIDE, *NUMBER, *AMOUNT, *COST, *RADIUS, *_X, *_Y, *_INCREMENT, *_FACTOR, *_TIME, *_SIZE, DURATION | \-?[0-9]+ |
| percentage | a percentage | *PERCENTAGE | a numeric or 0\.[0-9]+ |
| position | a single x,y coordinate | POSITION, *_POSITION, BASE | -?[0-9]+,-?[0-9]+ |
| span | a set of coordinates or coordinate ranges | *_SPAN | a numeric, position or ([0-9]+\-[0-9]+,?|[0-9]+,?)+ |
| alliance | a set of side numbers | SIDES, *_SIDES | a span, or the empty string |
| range | an attack range | RANGE | "melee" or "ranged" |
| alignment | an alignment keyword | ALIGN | "lawful" or "neutral" or "chaotic" |
| types | a set of unit types | TYPES | a shortname, name, or anything that contains spaces and matches no other type |
| terrain_pattern | a set of terrain codes to filter | ADJACENT*, TERRAINLIST*, *TERRAIN_PATTERN, RESTRICTING | a terrain_code or name |
| terrain_code | a single terrain code, perhaps with overlay | TERRAIN*, *TERRAIN | a shortname or (\*|[A-Z][a-z]+)\^([A-Z][a-z\\|/]+\Z)? |
| shortname | a terrain code or a short, capitalized variable name | [A-Z][a-z][a-z]? | |
| name | a name or identifier | NAME, VAR, IMAGESTEM, ID, FLAG, *_NAME, *_ID, NAMESPACE, BUILDER, *_VAR | anything without spaces that matches no other type |
| optional_string | a string value (may be empty) | ID_STRING, NAME_STRING, DESCRIPTION, IPF | a string, or the empty string |
| string | a nonempty string not matching any of the preceding types | STRING, TYPE, TEXT, *_STRING, *_TYPE, *_TEXT | a shortname, a name, a stringliteral, or anything that contains spaces and matches no other type |
| stringliteral | a string in doublequotes or a translated string | ".*" or _.* but not _[a-z].* | |
| image | an image path, perhaps with image path functions | *IMAGE, PROFILE | [A-Za-z0-9{}.][A-Za-z0-9_/+{}.-]*\.(png|jpg)(?=(~.*)?) |
| sound | a music or sound filename | MUSIC, SOUND | string ending with ".wav" or ".ogg" |
| filter | WML filter | FILTER | any non-quoted string containing "=" |
| WML | arbitrary WML fragment | WML, *_WML | any non-quoted string containing "=", or the empty string |
| affix | a prefix, suffix, or infix for a variable name | AFFIX, *AFFIX, POSTFIX, ROTATION | a shortname or name, or the empty string |
| any | anything | *VALUE, [ARS][0-9] | anything |
If the actual argument is a macro call {.*}, then it matches any formal Otherwise, if the formal has an identifiable type, wmlscope will complain if the actual literal does not match it.
The argument type check only works in macro calls that fit on a single line.
wmlscope has many options for changing the reports it generates; the more advanced ones are intended for Wesnoth developers. Invocations for the most commonly useful reports it generates are included in data/tools/Makefile of the source distribution. Here are some of those reports:
- make unresolved
- Report on unresolved macro calls and resource references; also report macro argument-type mismatches. (This is what you are most likely to want to do).
- make all
- Report all macro and resource file references, not just unresolved ones.
- make collisions
- Report on duplicate resource files.
For more advanced users, or those who want to understand what the canned Makefile invocations are doing, here is a summary of wmlscope's options. Some of the more advanced options will require you to understand Python regular expressions.
- -h, --help
- Emit a help message and quit
- -c, --crossreference
- Report resolved macro references (implies -w 1)
- -C, --collisions
- Report duplicate resource files
- -d, --deflist
- Make definition list. (This one is for campaign server maintainers.)
- -e regexp, --exclude regexp
- Ignore files matching the specified regular expression.
- -f dir, --from dir
- Report only on macros defined under dir
- -l, --listfiles
- List files that will be processed
- -r ddd, --refcount=ddd
- Report only on macros with references in exactly ddd files.
- -u, --unresolved
- Report unresolved macro references
- -w, --warnlevel
- Set to 1 to warn of duplicate macro definitions
- --force-used reg
- Ignore reference count 0 on names matching regexp
- --extracthelp
- Extract help from macro definition comments.
wmllint
wmllint is a tool for migrating your WML to the current version. It handles two problems:
- Resource files and macro names may change between versions of the game. wmllint knows about these changes and will tweak your WML to fit where it can.
- Between 1.2.x and 1.3.1 the terrain-coding system used in map files underwent a major change. It changed again in a minor way between 1.3.1 and 1.3.2. wmllint will translate your maps for you, unless you use custom terrains in which case you will have to do it by hand.
wmllint also performs various sanity-checking operations, reporting:
- unbalanced tags
- strings that need a translation mark and do not have them
- strings that have a translation mark and should not
- translatable strings containing macro references
- filter references by description= (id= in 1.5) not matched by an actual unit
- abilities or traits without matching special notes, or vice-versa
- consistency between recruit= and recruitment_pattern= instances
- double space after punctuation in translatable strings.
- unknown races or movement types in units
wmllint takes a directory-path argument specifying the WML directories to work on. It will modify any cfg and map files under those directories that need to be changed. Here is a summary of its options:
- -h, --help
- Emit a help message and quit.
- -d, --dryrun
- List changes but don't perform them.
- -v, --verbose
- Set verbosity; more details below.
- -c, --clean
- Clean up -bak files.
- -D, --diff
- Show diffs between unconverted and unconverted files.
- -r, --revert
- Revert the conversion from the -bak files.
- -n, --nolift
- Suppress lifting, do sanity checks only
The verbosity option works like this:
- -v
- lists changes.
- -v -v
- warns of maps already converted.
- -v -v -v
- names each file before it's processed.
- -v -v -v -v
- shows verbose parse details (developers only).
The recommended procedure is this:
- Run it with --dryrun first to see what it will do.
- If the messages look good, run without --dryrun; the old content will be left in backup files with a -bak extension.
- Eyeball the changes with the --diff option.
- Use wmlscope, with a directory path including the Wesnoth mainline WML, to check that you have no unresolved references.
- Test the conversion.
- Use either --clean to remove the -bak files or --revert to undo the conversion.
Additionally, wmllint tries to locate a spell checker on your system and spell-checks storyline and message strings. It will work automatically with either aspell, myspell, or ispell provided you have the enchant.py Python library installed.
wmlindent
Call with no arguments to filter WML on standard input to reindented WML on standard output. If arguments are specified, they are taken to be files to be re-indented in place; interrupting will be safe, as each reindenting will be done to a copy that is atomically renamed when it's done. This code never modifies anything but blank lines and leading and trailing whitespace on non-blank lines.
The indent unit is four spaces. Absence of an option to change this is deliberate; the purpose of this tool is to prevent style wars, not encourage them.
If you don't apply this tool to your own WML, the mainline-campaign maintainers will do it when and if your code is accepted into the tree.
Note: This tool does not include a parser. It will produce bad results on WML that is syntactically unbalanced. Unbalanced double quotes that aren't part of a multiline literal will also confuse it. You will receive warnings if there's an indent open at end of file or if a closer occurs with indent already zero; these two conditions strongly suggest unbalanced WML.