Revision as of 19:32, 26 April 2007

WIP

Addon server

After collecting the info in this thread http://www.wesnoth.org/forum/viewtopic.php?t=15664 this is a first proposal to all changes for the addon server. Personally I'm most interested to do the server side changes and not the client side changes so I hope to find a victem for that part ;)

The changes are splitted in 3 parts, the general changes, the changes specific to improve the handling of translations and last the automatic test for some broken content.

General changes

Client side changes

For the client side some extra fields will be available to show, some are set by the author and some by the admin/devs. Also note I think it might be a good idea to give some non-devs dev access to the server, that would be people the devs think should have that access. I don't want to do the voting system at the moment since that will be quite some work (due to possible abuses) and adds little benefit. (The main reason for starting this project is to allow asyncronious updates of the translations.)

icon        like current icon
name        like current name
title       like current title
description like current description (but hopefully shown in the client)
image       larger image to show (as shown in the addon menu when you select
            an addon)
type*       type of the content see below (the client might want to split this
            displayed info per type)
author      name of the author
version     version number of the addon (as set by the author)
date        timestamp of last upload, this way 2 updates on one day can be seen
            as different. (ideally the client should notice a newer version by
            this value and give a visible clue)
size        download size of the addon excluding languages. Languages show
            this value separately.
length      number of scenarios, mainly for campaigns
difficulty  difficulties available as comma separated list
version     string with versions it works with eg
            - 1.2.* all 1.2 versions
            - 1.2.3+ versions equal or newer as 1.2.3
            - 1.2.0 - 1.2.2 versions greater or equal to 1.2.0 and smaller
              or equal to 1.2.2
dependency  other content this addon depends on, comma separated list of names
            (This might allow the client to allow automatic downloads of
            dependencies)
status      The status is split in 3 parts
            - author status, the development status of the addon**
            - dev status, the status the devs give to the addon***
            - macroscope status, the status macroscope gives****

* type is one of the following
SP campaign
MP campaign
MP map(s)
era
faction
mod
other

** author status is one of the following
under construction      addon is still in its early stages and is incomplete
alpha                   addon is pretty much complete
beta                    addon is complete but needs more polishing/debugging
complete                addon is complete and hopes to get a star soon

*** dev status is one of the following
Unknow      default status
broken      the addon doesn't work as expected
buggy       the addon works almost as expected but has some bugs
works       the addon works as expected but uses deprecated WML
certified   the addon works and triggers no deprecated WML messages
            (I don't like the word certified, so a better name is welcomed)
star        same as certified but the devs enjoy to playing with this addon

**** Macroscope status
Macroscope is a tool which generates an automatic status, described in more
detail later, it has 2 possible statusses
broken      a red cross, the addon is broken due to unresolved references
not broken  a green V, the addon has no unresolved references (this does
            not mean the addon can't be broken in other ways.)

Server

On the server is some more info stored

email   email adress of author mandatory so it's possible to notify about status
        changes

Translations

The goal is to seperate translations from the addon so it's easier to allow translators and the author update independently. Also the author no longer needs to work with Wescamp the update is automatically send to Wescamp. (If the author wants that of course.)

Note the specifics about the Wescamp are not discussed here only the general changes. The specifics will be discussed when the time is there abd probably on irc.

client side changes

In the client a user can select 0 or more languages (s)he's interested in, this is a general preference it's used at the client as a hint. No need to show 30 languages with their status if the user isn't interested in it. The untranslated addon is always downloaded directly so English speakers might not be in translations at all.

The client can show the following info about the translation date date of the last update size the size of the download revision revision number of the last update (Since the info is pulled from

           Wescamp the updater can use this info to test for changes. The
           client can use this info to determine whether the downloaded
           translation is the most recent version and notify the user if not.)

status the percentage of strings translated. (Since fuzzy strings are

           shown in English the count as not translated.)

Upload

The pbl file will get a entry to tell whether an addon should or shouldn't be translated. If translated it will need the svn path to Wescamp for that addon. If marked for translation the addon is copied to Wescamp overwriting the stuff there. After the upload the version at Wescamp is directly updated.

This part will be updated in a later phase for now only the marking for translation is important, the other details will depend on the implementation.

Updating translations

The will be a cronjob on the addon server that will check all translations in Wescamp whether there's a newer revision as the last copied revision if so the newer revision will be copied to the addon server. (It's undetermined whether this will be a real cron job or a cron like facility in the addon server.)

Macroscope

Esr wrote a tool, named macroscope, which can validate whether WML references are broken or not (it can do more but for the server only the broken references are used) Broken references are - calls to non-existing macros - references to non-existing images/sound/music files

Macroscope will run on the server, the problems with running on the clients are - Python is required - It's possible to upload broken content by modifing macroscope - it's required anyway due to new versions

Upload

On upload an addon is checked whether it has no broken references. If it's broken it's not allowed on the server, the output of Macroscope is send back to the uploader so (s)he can see what the problems are.

After the upload is successful there will be a preprocesed file made which can be used by the dependent addons.

NOTE this means if addon B depends on macro A in addon A and A uploads a new version without macro A then addon B is broken. A can't be blamed that B uses some of it's stuff so the upload of A will not be blocked. This means the server needs to "remember" the reverse dependencies and test them after a new upload, so B gets marked broken.

New version

If a new version of Wesnoth is released a proprocessed file uploaded and all current available content is checked. The macroscope status is set in the addon, upon change of status the author is notified per email (if possible)

Since stable versions should be compatible this shouldn't change the status. For the development version it can be that a content gets broken.

Terrain system

The goal is to convert the old single letter terrain system to a new multi letter system. The internal changes to the engine are almost done.

These are the proposals and explanations for the new system, once these are final and implanted these texts will move into the wiki.

Version: proposal 2.3
Last updated: --SkeletonCrew 13:52, 29 November 2006 (CET)

History:
proposal 1.1

added the encounter terrains

proposal 1.2

added the link to the new proposals of Mog and Eleazar

proposal 1.3

added reminder to add fog/shroud to the editor
added idea to maybe add support to add tiles at the top and left

proposal 2.0

removed editor changes from proposal and moved to own topic
removed the discussion points and wrote the conclusion
rewrote User:SkeletonCrew#terrain_strings
rewrote the letter proposal, with the input given so far

proposal 2.1

Eleazar changed some terrain definitions and fixed some errors I made in them
added section about starting positions and the new definition
added extra keeps and changed the way they're going to work

proposal 2.2

added AnimationWML section

proposal 2.3

changed encountered terrain section

terrain

Every terrain type gets a new field named string this will be the new string to use to refer to the terrain. The old char field will stay for the existing terrains, this is needed to convert older maps to be used with the new engine.

This string will exist of 2 to 4 characters, the allowed characters are all letters (the engine is case sensitive), all numbers and the symbols / | \. Note leading numbers are not allowed in the string, they might give confusion with starting locations. Other characters are reserved for special uses in the engine.

The limit of 4 characters is not the upper limit of the engine, but the upper level for easy parsing. The internal format uses 4 bytes and every character is 1 byte. This way it's very easy to convert the characters to the internal format. If more letters are required it will require some modifications to the engine.

I'm considering to allow 1 character terrains as well since I've some trouble to find a second letter everywhere.

Every terrain will also get a new field named theme this is explained in the editor section.

scenario map

The map will use the new strings instead of the old letters. For every tile a string will be used instead of one letter, these strings are separated with a comma. Between the string and the comma one or more spaces are allowed.

Instead of spaces tabs are allowed, but keep in mind what looks good in your text editor, might look horrible in the editor of somebody else. The official files should use spaces.

The WML for map will use the same format. The letter for map WML will use a single terrain string without comma's.

Terraingraphics

The need two modifications the map format and the type format.

map

This format will change a bit more than the other items. The old format is described here TerrainGraphicsWML. According to mog the letters in the map are never used, only the anchors are used. The new format will only accept anchors. The special characters . and * will stay the same.

This map file will no longer be space separated but comma separatedand. Odd lines will start with a leading comma instead of 2 spaces. Spaces are allowed for padding

Also here tabs are allowed instead of spaces, see previous section.

For example this

will become

map="
,1
1,1
,2
1,1
,1"

or

map="
, 1
1 , 1
, 2
1 , 1
, 1"

type

The type will be a list of terrains, comma separated. Spaces are allowed but then the items need to be between brackets. (A WML rule.) Also after special modifiers a comma is required (at the moment only the ! is a special modifier, but that might change in the future.)

Mog had an idea which will also be implanted, the * can be used in a terrain string, V* would match all terrain strings starting with the letter 'V'. Vh* would match all terrain strings starting with 'Vh' eg Vh, Vha, Vhaa but not VH or vH.

terrain files

Eleazar would like to move the terrains in subfolders, this seems to be possible with the current system. The required modifications can be found in the following changelog http://svn.gna.org/viewcvs/wesnoth?rev=14069&view=rev

I've no idea what the best possible directory layout will be. I'll ask Eleazar to do this change when I'm ready for it. At the moment I've some patches in my queue which will conflict with these changes.

terrain strings

The terrain strings will consist of 2 till 4 letters which indicate the terrain. Numbers are not allowed except for starting positions. See next section. The following special characters can be used /|\ these should be used for directional terrains, which are only bridges at the moment.

The terrain letter _ is reserved for interal special terrains this will alway be used as leading character and not used inside strings.

For readability terrains should start with a capital letter followed by one or more lowercase letters (with a maximum of 3 lower case letters). The official terrains must follow these rules, the custom terrains should follow these rules.

There will be some letters reserved for user made content these letters are Y, y, Z, z. This means every terrain with one or more of these letters inside are custom terrains. This way it's easier to integrate a new terrain without having to change the terrain-graphics.cfg.

Since UTBS is mainline the custom terrain used there will be moved to the new system.

starting positions

The new system will allow to define a starting position and terrain in the map file.

The new format is "nnwtttt"

n is a number and must be at least 1 position, this number defines the starting position, for alignment the engine assumes a maximum of 2 positions
w is one space, needed for readability and easy parsing
t is the terrain string

For backwards compability the "wtttt" part can be ommitted and the terrain will automatically become _K but support won't last forever (see User:SkeletonCrew#Specials).

At the moment Wesnoth supports 9 starting positions, this proposal allows for more positions but the engine can't use the extra positions. This limit will be removed in the near future.

aliases

The aliases will also be a comma separated list, where the same discussion about the + and - can be started as in the types.

terrains sorted per group

This list shows the new proposal for the terrains, which takes in account the idea's of Mog and Eleazar. The italic strings are reservations for terrains. The only terrain which isn't backwards compatible is '*'. The other terrains will be converted.

Some letter defined in the old system don't have a terrain defined yet. These are still listed with their reserved letter. Just in case the terrain gets defined before this system is up and running.

I think about merging some theme's if the number of items is to small. The following merges will probably be done

hill, mountain
water, swamp
castle, keep

I still want to keep the themes, if for example the number mountain and hill terrains increases enough to justify their own page the splitting will be trivial.

The letters "$.;:<>_` are listed but unassigned in the old list and thus not supported in the new system.

Snow terrains

String	Old letter	Description	Theme
Ai	i	ice	snow
Aa	S	tundra "Arctic"	snow

Bridges

Since bridges are directional all strings here will be appended with either /|\ to show their direction. In this list these signs are replaced with a *.

String	Old letter	Description	Theme
Brx*		Bridge of stones over chasm for aboveground	bridge
Bxu*		Bridge over chasm underground	bridge
Bsw*		wooden bridge over swamp NOTE bws Brigde wood swamp	bridge
Bwr*		Stone Bridge over shallow water NOTE bss Bridge stone swamp not better?	bridge
Bww*	/\|\	wooden bridge bridge, over wadeable, wooden NOTE Brigde wood or Bridge wood water not better?	bridge, common

Castles

String	Old letter	Description	Theme
Ce	n	encampment (castle)	castle, common
Ch	C	human castle	castle, common
Chr	N	ruined castle	castle
Chs	q	human ruin (swamp)	castle
Chw	Q	human sunken ruin	castle
Co	O	will be an Orc Castle one day	castle
Cud	o	dwarven castle "castle underground, dwarf"	castle, underground

desert

NOTE still need to add the UTBS terrains

String	Old letter	Description	Theme
Dd	I	desert	desert
Do	P	desert oasis	desert
Ds	d	sand, (old desert)	desert

Forest

String	Old letter	Description	Theme
Fa	F	forest (snow) "forest arctic"	forrest, snow
Fb		Sinister Forest "Forest, Bad"	forrest
Fet	?	great Elven tree	forrest
Ff	f	forrest	forrest, common
Ft	T	forrest tropical	forrest

Grass

String	Old letter	Description	Theme
Gb		dying "Grass brown/bad"	grass
Gf		Farmer's Fields	grass
Gg	g	grass	grass, common
Gs	G	savanna (grass)	grass

hill

String	Old letter	Description	Theme
Ha	H	hills (snow) "hills arctic"	hill, snow
Hd	J	desert hills (dunes)	hill, desert
Hh	h	hills	hill, common

Keep

The keep definition is now done with terrain transition WML, since we're no longer out of letters, this will be done manually. the _K is for backwards compability but this will remove at some point see specials. The _K uses the castle transition WML the new ones don't. This would also allow to mix items, eg een Kh keep in a Ce castle.

String	Old letter	Description	Theme
_K	K	keep (castle)	keep, common
Ke		encampment (keep)	keep, common
Kh		keep castle	keep, common
Khr		ruined keep	keep
Khs		human ruin keep(swamp)	keep
Khw		human sunken ruin keep	keep
Ko		will be an Orc keep one day	keep
Kud		dwarven keep "keep underground, dwarf"	keep, underground

Mountain

String	Old letter	Description	Theme
Ma	j	snowy mountains	mountain, snow
Md	M	desert mountains	mountain, desert
Mm	m	mountain	mountain, common

(mostly) Impassable

String	Old letter	Description	Theme
Qx		chasm for above ground	misc
Qxu	X	chasm for underground	misc, underground
Qxw		aboveground chasm with water pouring over the edge	misc
Ql	l	lava	misc
Qr		stone wall	misc

road

String	Old letter	Description	Theme
Rc		Carpeted Floor	road
Rd	E	desert road	road, desert
Re	r	dirt road "earth"	road, common
Rr		stone road "rock"	road
Rw		Wood floor (interior)	road

swamp

String	Old letter	Description	Theme
Sb		fetid swamp "Swamp Bad"	swamp
Ss	w	swamp	swamp

underground

String	Old letter	Description	Theme
Uf	]	mushroom grove "underground forest"	underground
Uh	[	rockbound cave (functionally "underground hill")	underground
Uu	u	normal cave	underground
Uuc		normal cave, carved	underground
Uui	'	illuminated cave	underground

village

String	Old letter	Description	Theme
Vda	B	desert village (adobe)	village, desert
Vdt	U	desert village (test)	village, desert
Veg	t	elven village "village elven grassland"	village
Ves	e	elven (snow) village	village, snow
Vha	V	village, snow snow village (tundra, village)	village, snow
Vhg	v	human village (village)	village, common
Vhh	a	human hill village	village, hill, common
Vhha	A	human (snow) hill village	village, snow, hill
Vhm	b	human mountain village	village, mountain
Vht	L	tropical village (savanna, village) --nothing foresty about this village	village
Vu	D	underground village (cave, village)	village, underground
Vud	p	dwarven village	village, underground
Vwm	Z	mermen village (shallow water)	village, water
Vs	Y	swamp village (swamp, village)	village, swamp

water

String	Old letter	Description	Theme
Wo	s	deep water "water, ocean"	water, common
Ww	c	shallow water, "water, wade-able"	water, common
Wwf	k	river ford "water wadeable, ford"	water
Wwu		shallow underground water	water, underground

Impassable

String	Old letter	Description	Theme
Xm	&	impassable moutains	impassable, mountain
Xrw		Impassable stone "rock" wall (interior)	impassable
Xu	W	cavewall	impassable, underground
Xuc		cavewall, carved	impassable, underground
Xww		Impassable wooden wall (interior)	impassable

Specials

These terrains are hardcoded in the game and translated automatically. The old UMC terrains are here for backwards compability this compability will only be guaranteed till the next major release after 1.2.(That would be 1.4 or 2.0.) As stated before the * will no longer be supported.

String	Old letter	Description	Theme
_f	~	fog	misc
_s	(space character)	void/shroud	misc
_za	^	reserved for UMC	misc
_zb	%	reserved for UMC	misc
_zc	@	reserved for UMC	misc
_zx	x	reserved for UMC	misc
_zy	y	reserved for UMC	misc
_zz	z	reserved for UMC	misc

encounter terrains

This list maintains the encountered terrains by the user this list is used in the help function to decide what to show and what not. This is now a comma seperated list. The new version will also be a comma separated list but with the new terrain strings instead of letters. Backwards compability will be dumped, the new list will be called encountered_terrains_list and the user will be required to rediscover every terrain. (The conversion is harder as expected and the benefit too small)

This way the 1.2 branch will use the old list and the 1.3 branch the new list.

AnimationWML

These also need an update here the * will also be allowed. The current system look at the aliasses (which is not documentated) the new system will look at the string and as said before allow the * to match.

editor

The editor also will need an overhaul due to the changes, since the terrain consists of a maximum of 4 characters I propose it writes the terrain as a string of 6 characters; the terrains string followed by a comma and padded with spaces. eg the following terrain string

Vh,Vha,Vhaa,Vha,Vh

will be written as

123456123456123456123456123456 (numbers won't be written.)
Vh,   Vha,  Vhaa, Vha,  Vh

Discussion: Should the editor be able to write the old format terrain or not.

custom terrains

Zookeeper told me there's no option to add custom terrains at the moment. I want to add a button to add a custom terrain, once clicked it will prompt for the terrain string and insert the tile to the list of available tiles and allow to use this tile. The tile will be 'painted' with the terrain string. When loading a map with unknown strings these will be added to the list of terrains. The only problem will be the location of the terrains but I'll worry about that later.

There should also be an option to add fog/shroud tiles to the map. See http://www.wesnoth.org/forum/viewtopic.php?p=192326#192326

Discussion: Would this solve the unkown tile problem?

themes

In my opinion the list with terrains to browse is pretty long and this will only grow in the future. I want to add an extra grid of buttons with theme's the buttons will have an icon for their theme. When a button is clicked the list of terrains is filtered per theme. eg when the theme villages is selected only village tiles are shown. Every terrain gets one or more themes for example a snow forest can be in the theme forest and the theme snow, so with either theme this tile is shown. There will be 2 special theme's favorites and map. Favorites are your favorite tiles saved in preferences and can contain unknown terrains. Map are all tiles already used on this map, including the unknown.

Discussion: Is this handy?

resize map

maybe there should be an option to add tiles to the left and top instead of only to the right and bottom. See http://www.wesnoth.org/forum/viewtopic.php?p=192326#192326

Small maps

It would be handy to be able to create maps smaller than 20 x 20 for WML usage. IMO as soon as the map goes below this threshold a warning should be shown, that the map might nog work properly as main scenario map. See http://www.wesnoth.org/forum/viewtopic.php?p=192435#192435

Todo

Things I plan to do after I finished my WIP's.

team sides

After the terrain system has been modified it's possible to define more teams, EdB asked this in irc at 20061027.

It would be nice to support 12 sides in multiplayer with their own tcolour. The engine should support more sides for the AI to use. These will use and scenario WML defined colour. It's not sure whether WML supports this if not the support should be made. EdB will look for an possible algo to define nice auto colours.

terrain archtypes

Eleazar had an idea for terrain archtypes in the terrain thread. It looks sane and would be nice to implant. http://www.wesnoth.org/forum/viewtopic.php?p=191410#191410

Misc

Various things I want to look at at some time.

version checks for WML, needed or not?
modulo WML, needed or not? Done, might cleanup similair code the same way.
continue to work on the Python AI

Branch terrain

Items marked with the text (SVN terrain only) are features only available in the latest development version of the terrain branch. These items will be merged in the trunk once finished. The branch can be found here. http://svn.gna.org/svn/wesnoth/branches/mordante_terrain

(SVN terrain only)

Back references AnimationWML#Progressive_parameters FilterWML TerrainLettersWML TerrainWML BuildingMaps TerrainGraphicsWML BuildingFactions TerrainMaskWML InternalActionsWML

Template:DevFeature EditorGroupWML MapGeneratorWML

@@ Line 92: / Line 92: @@
 On the server is some more info stored
+<pre>
 email   email adress of author mandatory so it's possible to notify about status
          changes
+</pre>
 === Translations ===

Difference between revisions of "User:SkeletonCrew"

Revision as of 19:32, 26 April 2007

Contents

WIP

Addon server

General changes

Client side changes

Server

Translations

client side changes

Upload

Updating translations

Macroscope

Upload

New version

Terrain system

terrain

scenario map

Terraingraphics

map

type

terrain files

terrain strings

starting positions

aliases

terrains sorted per group

Snow terrains

Bridges

Castles

desert

Forest

Grass

hill

Keep

Mountain

(mostly) Impassable

road

swamp

underground

village

water

Impassable

Specials

encounter terrains

AnimationWML

editor

custom terrains

themes

resize map

Small maps

Todo

team sides

terrain archtypes

Misc

Branch terrain