CompatibilityStandards

As a piece of software matures, there are often new designs, paradigms, and idioms developed which are superior to old ones. This creates an inherent conflict between the need for progress and the need for compatibility. Wesnoth is no exception. This document describes Wesnoth's approach toward resolving that conflict in a way which is most beneficial to both goals, as well as the rationale behind this approach.

Policy

This policy defines how the creation of new Wesnoth APIs and the deprecation and removal of old ones are to be handled. For the purposes of this document "API" means "any technical channel by which a content creator interacts with the game engine". This includes, but is not limited to: preprocessor macros, WML tags, WFL functions, IPFs, and the Lua API. Note that this policy applies only to software APIs. Core content such as sprites, portraits, animations, lore, etc. are to be updated freely, without consideration for stylistic or literary conflicts that such content changes may present to add-ons.

When to deprecate

Any time a superior API is introduced which has complete feature-parity with an existing API, the old one should be immediately deprecated. Note that in most cases, in order to be considered to have "complete feature-parity", the API should be available in the same language or area of the code as the obsolete one was. For example, the introduction of a more powerful API in Lua which can accomplish a superset of the functionality which had previously been available with a certain WML tag would not obsolete that WML tag. Exceptions can be made to this rule in cases where it is clear that the feature does not make a lot of sense in its current language or area, and was merely there for legacy reasons (such as the proper place for it not having been introduced yet at the time of its creation). Such exceptions should be determined by developer consensus.

How to deprecate

Each deprecated feature should make use of an appropriate deprecation function call for that language or subsystem to ensure that the appropriate deprecation notice is printed to the log output as well as displayed in-game if running in debug mode. In-game deprecation messages should ONLY appear if debug mode is active. Deprecation notices should, ideally, point the user in the direction of the replacement API that they should be using instead. Documentation on deprecated features should be updated to reflect the deprecated nature of the given API, and should likewise have appropriate signposting toward the preferred replacement.

Every effort should be made to create the simplest possible wrappers which will translate from an obsolete API to the updated one. Such wrappers should ideally be organized into their own compatibility file or module, and set up in such a way that the internals of the updated API will not affect how the old calls get wrapped to the new one. Essentially, the idea is to create a set-it-and-forget-it compatibility wrapper which will continue to work regardless of updates made to the newer API.

Deprecation levels - When to remove deprecated features

While creating simple compatibility wrappers should be possible at least 90% of the time, it would be unreasonable to assume that this approach will be viable in absolutely every case. Wesnoth's compatibility policy therefore allows for three different levels of deprecation, depending on the severity of the change being made and the cost of maintaining backwards compatibility:

1. Deprecated indefinitely — This deprecation level is the one which should be aimed for the vast majority of the time. Obsolete APIs which can be implemented in terms of a simple minimal-maintenance wrapper should be deprecated, but left in the code indefinitely. At such time as an obsolete API presents itself as an active obstacle toward further development, it can be slated for removal in the next release, or even removed at will immediately, so long as it has already survived a minimum deprecation period of at least one stable release version. The functions for generating deprecation messages for this level should take a parameter for the first version in which the given feature was deprecated, and their output should reflect whether that period has passed or not, with gentler warnings for those which have not ("this feature is a candidate for removal and may be removed as early as the next version") and harsher warnings for those which have ("this feature is obsolete and may be removed without warning at any time").
2. Deprecated for removal — This deprecation level should be used for features which cannot be maintained as simple wrappers, but which could be maintained as separate, coexisting code. In order to prevent the problem of double-maintenance and code bloat, this level has a built-in lifespan cap of one stable release version following its initial deprecation, after which it will be removed. There may also be cases where a level 1 deprecated feature presents itself as a hindrance to future development, but can be temporarily maintained in this manner as well. In such cases, it is ideal to move the deprecated feature from level 1 to level 2 rather than removing it outright. Deprecation messages for this level should reflect the severity of the deprecation ("this feature will be removed in the next version and requires immediate maintenance").
3. Removed without deprecation — This level should be used EXTREMELY rarely, and only in cases where it is ABSOLUTELY NECESSARY. Occasionally, an update to a feature will change the underlying architecture in such a fundamental way that the old paradigm cannot coexist with the new one no matter how much redundant code one would create. While this kind of scenario is extremely rare, and every effort should be made to find creative solutions to avoid it, there are occasionally cases where it truly is impossible to maintain both methods even in the short term. This level should only be used with broad developer consensus, after the majority of active developers familiar with the feature in question have given at least some thought to trying to deprecate gracefully and failed.

Post-removal

Features which have been removed (subsequent to any of the deprecation levels) should ideally maintain a skeleton of their former interface for at least one stable release version following their removal. This skeleton should print a more detailed error message than the usual "invalid tag/function/object/attribute/whatever" which would result from their complete removal, and the messages should include information on when (and possibly why) the feature was deprecated and where to look for its replacement.

Rationale

The above policy is the result of a large amount of thought, discussion, and debate. Following is a brief outline of the considerations and goals on both sides of the problem, why there is an inherent conflict between them, some failed approaches to resolving the conflict, and how the final policy ultimately maximizes the pursuit of both goals.

The Problem - The paradox of progress

Invariably, as development on any project moves forward, developers will realize that there are better, cleaner, or more elegant ways to structure things than they had been previously. These changes can be to improve efficiency, make an API more intuitive, keep code better organized, make common tasks more straightforward, or any number of other positive things. These changes can also make the development of additional features much more viable. In short, progress is good.

On the other hand, a content-heavy program such as Wesnoth relies on the ability of content creators to efficiently create, maintain, and update their content. Too many changes all at once will force creators of existing content to spend obscene amounts time updating their creations just to keeping them up-to-date and in working order. This can lead to a high amount of frustration, a drop in motivation, and a decline in content being created. In short, progress is bad.

Backwards Compatibility - benefits and drawbacks

Most of the time, older paradigms can still be maintained in a manner in which they coexist with the newer ones. This allows for existing content to continue functioning, while at the same time allowing and encouraging new content to be created using the newer methods. However, maintaining such code can be problematic in the following ways:

1. There may eventually be architectural changes a developer would want to make where the old method's square peg no longer fits, even forcibly, into the new method's round hole.
2. Having compatibility code hanging around may, depending on how it is implemented, mean that any updates made to the feature, module, or subsystem in question would have to made in both the new, cleaner design, and the older, poorly structured one, adding more work for developers.

The Naive Approach - Deprecate and remove everything old

One approach to balancing old and new is to deprecate the old and slate its eventual removal after either a certain amount of time has passed or a certain number of subsequent versions have been released. This sounds good in theory, but in practice, there will be many changes which are minor or cosmetic in nature and which will add up. Things like replacing macros with WML tags, updating the name of an API call or order of parameters to be more consistent with other similar functions, or switching from a functional to an object-oriented structure are very good for organizational purposes, but will result in a large amount of maintenance required on the part of content creators to keep existing code operational, and for very little real gain. This approach invariably leads to the situation where so much is being changed from one version to the next that creators turn into maintainers, forced to spend nearly all of their time trying to stay ahead of the update curve in an attempt to keep their existing content working, and leaving them very little time and motivation to create new content. And of course, by the time they're finished painstakingly updating their existing code for every little change made for the current release, whoops, there's a new release with a whole slew of new changes that need accounting for. It simply becomes unmanageable.

The Naive Approach - Deprecate and remove only when necessary

The opposite aproach would be to keep all existing paradigms until they actively interfere with a new architecture or create a double-maintenance problem. While this approach does cut down on the maintenance burden by ensuring that content using an older design continues to work, it hinders progress by the fact that the moment at which it first becomes clear that an architectural or double-maintenance problem will occur is exactly the same moment at which keeping the old structure around becomes problematic. Beginning a deprecation cycle at that point and then having to "wait out" the old paradigm will cause an unacceptable delay in development.

The Middle Ground - Deprecate everything old, remove only when necessary

Most of the time, older APIs can be implemented in terms of their newer, cleaner counterparts through the use of things like simple wrappers, parse-translators which re-write the older paradigm's code in terms of the new one, or other relatively low-maintenance "set-it-and-forget-it" approaches. These simple wrappers are not really detrimental to making progress, don't require updating when the new APIs internals are changed, and can usually be organized into their own files and/or modules so that they don't clutter the cleaner code. Many such wrappers will never truly present either of the backwards compatibility drawbacks mentioned above. As such, there is really no detriment to keeping them around indefinitely. However, occasionally, a new idea or approach will be put forth that updates the newer paradigm in such a way that the older one can no longer cleanly wrap to it. This usually happens, as inspiration is wont to do, suddenly, unexpectedly, and without warning. As such developers need the flexibility to be able to remove outdated code as freely as possible when the situation requires. Therefore, the ideal solution would be to deprecate any API which has newer, feature-complete ways to do it, while leaving it in the codebase until such time as its presence becomes a hindrance. Essentially, deprecation need not necessarily mean "this WILL be removed" so much as "this is now a candidate for removal". By separating the concepts of deprecation and removal, both goals can be better served.

The Net Result - Mutual benefit

By deprecating old methods while keeping them around for as long as feasible, the work for content creators to update for a newer release is drastically reduced. They need only update a bare minimum in order to get their content working again, while still being encouraged to update the rest. While it is true that ideally, a content creator would update all deprecated code to use the updated APIs, the human element can't be ignored here. It is extremely frustrating to find that your existing code has thousands of places which need updating before any of it works at all. It is far more pleasant for to be able to do a small amount of tweaking to get something back into working order and then spend time gradually correcting the "uglier" or "less desirable" code while still having something functional to work off of. Counter-intuitive though it may be, indefinite deprecation actually does a better job of encouraging users to update their code than deprecation with a hard removal time does, by avoiding what would otherwise be a massive drain on motivation. Likewise, when things are deprecated as early as possible, developers have the freedom to remove things at will when the need arises, which will better maintain their motivation as well.

More Complex Cases - One size does not fit all

There may, however, be cases where the obsolete API cannot be implemented cleanly in terms of the updated one and must be maintained separately, or, in extremely rare cases, cannot coexist at all. There needs to be some leeway for such cases as well. In the former case, it therefore makes sense to allow for a feature to be deprecated pending removal after the shortest reasonable deprecation period. In the latter case, there is obviously no choice but to make the change and remove the old immediately. Developers should, naturally, be encouraged to find creative solutions to avoid such cases, but it is inevitable that there will eventually be a few cases where no graceful transition procedure can be found. These more aggressive forms of deprecation should only be done with developer consensus, and only after all options of creating a backwards-compatible transition have been exhausted.

Graphics - The exception that proves the rule

The one area where backwards compatibility should NOT be a factor is graphical changes. Any change to the terrain graphics, unit sprites, or portrait images has the potential to result in visually-incompatible custom content. Core content creators cannot be expected to maintain a deprecated visual style alongside a more modern one, as any approach toward doing so would add an unreasonable amount of bloat and overhead, and make it very difficult for core graphics to be updated without having to do double the work. In addition, add-ons will continue to function even with such visual incompatibilities present, they just won't look right. As such, it can be said that stylistic incompatibilities fall more within the realm of content than of code, and it is not unreasonable to expect a content creator to... well... create content. Expecting the graphical style to remain backwards-compatible would be just as unreasonable as expecting stories, help descriptions, or other forms of lore to never change because they may introduce plot holes into add-on stories. Essentially, since the goal of the software portion of Wesnoth is, at its heart, merely to facilitate the creation and advancement of this kind of content, core content needs to be free to develop unhindered by considerations for add-on content.