GSoC 2011 Sytyi

This page is related to Summer of Code 2011

See the list of Summer of Code 2011 Ideas

This is a Summer of Code 2011 student page

Project: SoC_Ideas_Schema_Validation2011

Description

Sytyi - WML Validator

The main idea of WML Validator is grammar based validator on C++ generating USER-FRIENDLY error messages in gcc style.

IRC

Sytyi

Contacts

IRC: Sytyi
E-mail: nsytyi[at]gmail.com
Skype: azcure
ICQ: four-zero-six-one-four-four-one-seven-four

Proposal Summary

I want to write a grammar based validator. One tool will prepare schema file containing all tags, keys and possible values from annotated source. Validator will read schema file on the beginning of the work and then, while parsing, just compare is this value allowed in this place.

Main idea is the hierarchical structure. Data storing allows to check key value, just by comparing with possible values. Also some simple types as string, integer, boolean can be possible too.

There are two problems:

1. Tags with different specific but one name. I think hierarchy will solve this problem but make another one. Each tag will have
2. Generally used tags like [action], [event], [filter]. They will be organized in global list. And list tags will have they mentioned. With this, hierarchy is broken a little, but it is still not graph in the widely understanding.

I also want to include Schema Generator in build system. I suppose, SG to update schema file, not to "delete-and-again" generation. It is possible cause of using hierarchy tag structure and annotations. WIP proof-on-concept SG example here https://gna.org/patch/?2636

Format of annotations and schema file

See WML_Annotation_Format for details.

How It All Work?

After a patch the developer runs SchemaGeneratorTool to regenerate schema file. I want to include the special build target in building system, trying to avoid the whole regenerating, only current updates. Also see ValidationDesign

If somebody want to validate his WML document he runs ValidatorTool, and receives Error Messages like

source-file-name:lineno:column: In file %filename
source-file-name:lineno:column: in  toplevel tag [scenario]
source-file-name:lineno:column: Mandatory key "id" is misssing
source-file-name:lineno:column: in tag [side] 
source-file-name:lineno:column: Mandatory key "side" is misssing  
source-file-name:lineno:column: Unknown value in key "controller"  %wrong_value found, but %value_list possible

Build System

I want to include Schema generating tool to build systems. This is not preference idea, only an add-on part. I want to make schema file refreshable (one source file was changed and only that part of schema will be updated), not "delete-and-again", but there are a few problems. If a tag or a key is removed from the source, how th SGT know about it? I think two ways: one to add source files names to schema file (this can be a little help to developer, trying to fix some bugs in code ), and second: to add special annotation to deleted keys.

Timeline

8.04 - 22.05 Getting into the project time
Improving annotations list. Getting as close as possible to WML documents. Studying how parser works.

23.05 - 30.06 Schema generating tool
I have exams in this period of time, and max number of exams can be 5, so I can't plan much more. And I even have no timeline of exams, it will be available only in June.

1. 23.05 - 15.06 Annotation parsing part.
   
2. 16.06 - 30.06 Schema generating part.
   

1.07 - 15.08 Document validating tool

1. 1.07 - 10.07 Hierarchy structures.
2. 11.07 - 25.07 Document parsing part.
3. 26.07 - 5.08 Validation.
4. 6.08 - 10.08 Data structures refactoring. (If nesessary).
5. 10.08 - 15.08 Errors messages generating part.
6. 15.08 - 20.08 Testing. Fixing bugs.

16.08 - 31.08 Afterwork

1. Testing
   
2. Fixing bugs
   
3. Documentation
   

<If any time left>
Improving build-systems integration, making more error types.

In far future
Maybe a validator for unpreprocessed WML.

Stages

ID	PART OF	NAME	PRIORITY	LAST FINISH TIME	STATUS
0		Proof-of-concept patch	GREATEST	15.04	IN PROCESS
		MILESTONE = "PREPARE"	GREAT	23.05
1	PREPARE	annotate a part of existing code	IMPORTANT	23.05	90%
2	PREPARE	improve annotations list due to nessesarity. view stage 2	VERY IMPORTANT	23.05	90%
3	PREPARE	write documentation and examples of annotations in wiki	BASIC	23.05	90%
		MILESTONE = SCHEMA GENERATOR	GREAT	30.06
1	SG	IMPLEMENTATION working mainstream source to SG	BASIC	20.06	90% NEED REVISION
2	SG	DOCUMENTATION including extended examples of annotations, algorithm describing, and list of used regexes	BASIC	30.06	60%
3	SG	global tags support	BASIC	30.06	100%
		MILESTONE = VALIDATOR	GREAT	15.08
1	VALIDATOR	parsing schema file. Build wml tags tree	BASIC	10.07	100%
2	VALIDATOR	parsing wml file part. Using /src/serialisation/ (Optionally)	BASIC	25.07	NEED DISCUSSION
3	VALIDATOR	checking against tree.	BASIC	5.08	10%
4	VALIDATOR	generating gcc-style errors implementation	BASIC	15.08	5%
5	VALIDATOR	working tool	BASIC	15.08
6	VALIDATOR	documentation about data containers, hierarchy, algorithms and input/output specification.	BASIC	25.08
	SG	BUILD system support	ADDITIONAL		NEED DISCUSSION
	SG	refreshable schema file support	ADDITIONAL	NOT NEEDED
	SG	extend error messages	ADDITIONAL

BASIC PRIORITY means that milestone without that stage just cannot be mentioned The milestones after 30.06 schould be improved with more information, non mainstream milestones and so one. It's a bit hard nowadays. The more work - the more problems I explore.

Patches and commits

No one at this moment of time.

Answers to those questions

Basics

Write a small introduction to yourself.
Hello. My name is Nick Sytyi . I study Computer Engineering at the Chernigiv Technological University. I want to participate in Google Summer of Code and gain expierence in C++ and game development. I like this project and I want to encahnce it.

1.2) State your preferred email address.
nsytyi@gmail.com

1.3) If you have chosen a nick for IRC and Wesnoth forums, what is it?
IRC: Sytyi
Wesnoth forums: none for now Wesnoth wiki: Sytyi
GNA: sytyi

1.4) Why do you want to participate in summer of code?
I would like to try some serious work. I want to grow in C++ development, open source, team work and game development experience. I think the experience I gain here, will help me in future. I hope I come in project for long period of my life, because it is my dream (to develop a game) since when I was child.

1.5) What are you studying, subject, level and school?
Computer Engineering, 3rd year Faculty of Electronic Informational Technologies, Chernigiv Technological University, Ukraine.

1.6) If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for Wesnoth. If you have gained commit access to our S­­V­­N (during the evaluation period or earlier) please state so.

No patches for now. Maybe the situation will change for better.

Experience

good knowledge of C++, Java, computer science concepts, small experience of programming translators, olympiad projects with MCS-51 and AVR stends (I know it is insignificant but it gave me experience in quick, well synchronized C programs).

2.1) What programs/software have you worked on before?
Mostly, I'am working on website development using Java EE. Also, I've done little C++ projects writing simple data manipulating programs.

2.2) Have you developed software in a team environment before? (As opposed to hacking on something on your own)
Yes. Our university has command projects, and now I am a team leader for a simple course work So I was to learn S­­V­­N features.

2.3) Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?
I haven't participated in the GSoC before. This is my first chance to participate.

2.4) Open Source

2.4.1) Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.

I use Ubuntu, Mozilla, Eclipse, Qt , but Wesnoth is the first open source project I am involved with.

2.5) Gaming experience - Are you a gamer?
Yes, I am a gamer )

2.5.1) What type of gamer are you?
Slow strategic gamer.

2.5.2) What type of games?
I prefer turn-based strategy games or RPG with good story.

2.5.3) What type of opponents do you prefer?
I can’t say I play good. So I prefer good opponents which sometimes teach me some tricks. When I’m beaten, I analyze, why this occurred.

2.5.4) Are you more interested in story or gameplay?
Story is more interesting for me. But if gameplay is very booooring ………….

2.5.5) Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer. We do not plan to favor Wesnoth players as such, but some particular projects require a good feeling for the game which is hard to get without having played intensively.
I have heard about Wesnoth first time at 18 March. It interested me, and sometimes, when I need a rest, I play a little.

Communication skills

3.1) Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.
I can read and write English with few mistakes.

3.2) Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.
I just ignore rough statements in address. In a team I takes the second role.

3.3) Do you give constructive advice?
Maybe. Sometimes.

3.4) Do you receive advice well?
Yes

3.5) Are you good at sorting useful criticisms from useless ones?
Yes

Project

4.1) Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?
Yes. It is interesting to make tools for non-programmers content developers. I want to concentrate on grammar of WML. And if I have free time after all of this, maybe a simple WML GUI or including WML validation into existing GUI’s.

4.2) If you have invented your own project, please describe the project and the scope.

4.3) Why did you choose this project?
When I studied Google list of participating organizations, I read about your game first, I played a little, and then I want to enchance project. I am interesting in translations. WML validation seemed to me a perfect task to study the main project and a perfect task for a starter not confident in his abilities.

4.4) Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".
I'll have 4 or 5 exams in June, but I hope to pass 3 of them out by auto.
3-4 days in archaelogic expedition near 15 of August.
Two times 3-4 days on bike down the river in July. I can not say current time now, it depends on the last exam and my friends. More possible it will be after stipendy at 25 of each month

4.5) Include as much technical detail about your implementation as you can
I want to write two tools. One to generate Schema file from the annotated source. and another to validate WML documents using schema. Tags are organized in a hierarchical structure.
4.6) What do you expect to gain from this project?
I expect to gain experience in game development and maybe a continious work with Wesnoth.

4.7) What would make you stay in the Wesnoth community after the conclusion of SOC?
Willing to “upgrade” myself. To know more. To work with interesting people on the interesting tasks.

Practical considerations

5.1) Are you familiar with any of the following tools or languages?

* Sub­­version (used for all commits) Yes.

* C++ (language used for all the normal source code)Yes STL. QT. 

* Python (optional, mainly used for tools) no. 

* build environments (eg cmake/autotools/scons) 'make' - yes, 'scons' no. 'autotools' - no

I don't know LUA and i'm studying WML now.

5.2) Which tools do you normally use for development? Why do you use them?
Eclipse under Windows and Ubuntu. Qt. Notepad.

5.3) What programming languages are you fluent in?
I have a little practice in Java (1 years of experience) and C/C++ (2 years of experience) I know a bit of sh/javascript/pascal/c

5.4) What spoken languages are you fluent in?
I can speak in Ukrainian (native), in Russian (native), English.

5.5) At what hours are you awake and when will you be able to be in IRC (please specify in UTC)
I'm in UTC+3 timezone in summer. I am normally awake from +11 UTC to +02 UTC, and most of that time I'm at home or some anywhere with wi-fi, so I'm able to be in IRC.

5.6) Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail.
Skype : azcure

Grammar

I wrote grammar cause of misunderstood. So it does not influence the main idea, etc. But it may be useful to some people.

Grammar needs some improvements in types of key values, but the main work is done.
If you see any mistake here, please contact me
EBNF form for unpreprocessed WML http://en.wikipedia.org/wiki/EBNF

Document = [ Header ], { StmntList } ;
StmntList = Statement, EOL, { StmntList } ;
Statement = { ( Tag | MacroCall | MacroDefine | TextDomain | PreprocIf ) }  ;
Tag = TagStart { ( Tag | Key | MacroCall | MacroDefine | TextDomain | PreprocIf ) }, TagEnd ;
Key = Name, { Name} ,"=", KeyValue, { KeyValue },  EOL;
MacroCall = "{", Name," ",{ " " }, MacroValue {, MacroValue} , "}"
MacroDefine = DEFINE, Name, MacroVar {, MacroVar} , StmntList, "ENDDEF";
TextDomain = "#textdomain ", Name ;
PreprocIf = (IFBegin, StmntList, [IFElse, StmntList,] IFEnd) | (Undef, MacroName);
TagStart = "[",[ "+" ], TagName , "]" ;
TagEnd = "[/", TagName ,"]" ;
(* Can TagName contain uppercase?  *)
TagName = LowerCaseLetter | "_", { LowerCaseLetter | Special  | "_"};
KeyValue = ( Name | Substitution | Strings );
Substitution = "$",  (Name | Formula) ;
Formula = "(", Name, ")" ;
MacroVar = Name;
IFBegin = "#ifdef " | "#ifhave " | "#ifndef " | "#ifnhave " | "#ifver " | "#ifnver " ;
IFElse = "#else " ;
IFEnd = "#endif" ;
DEFINE = "#define " ;
ENDDEF = "#enddef" ;
Name = Letter, { Letter };
Identifier = Letter, { Letter | Digit | Special };
Strings = ( String | TranslatedString ), { "+", ( String | TranslatedString ) };
String = '"', { ( Letter | Digit | Symbols) },'"';
TranslatedString = "_", String;
Letter = (UpperCaseLetter | LowerCaseletter | "_" );
UpperCaseLetter = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z" ;
LowerCaseLetter = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" ;
Digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
Sign = "+" | "-"  ;
WhiteSpace = " " | "\t" , { " " | "\t" } ;
Special = "-" | ":" ;
EOL = { "\r" },  "\n" , {  "\r"|"\n"  } 

EBNF form for preprocessed WML

Document = [ Header ], { StmntList } ;
StmntList = Statement, EOL, { StmntList } ;
Statement = { ( Tag | TextDomain ) }  ;
Tag = TagStart { ( Tag | Key ) }, TagEnd ;
Key = Name, { Name} ,"=", KeyValue, { KeyValue },  EOL;
TextDomain = "#textdomain ", Name ;
TagStart = "[",[ "+" ], TagName , "]" ;
TagEnd = "[/", TagName ,"]" ;
(* Can TagName contain uppercase?  *)
TagName = LowerCaseLetter | "_", { LowerCaseLetter | Special  | "_"};
KeyValue = ( Name | Substitution | Strings );
Substitution = "$",  (Name | Formula) ;
Formula = "(", Name, ")" ;
Name = Letter, { Letter };
Identifier = Letter, { Letter | Digit | Special };
Strings = ( String | TranslatedString ), { "+", ( String | TranslatedString ) };
String = '"', { ( Letter | Digit | Symbols) },'"';
TranslatedString = "_", String;
Letter = (UpperCaseLetter | LowerCaseletter | "_" );
UpperCaseLetter = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z" ;
LowerCaseLetter = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" ;
Digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
Sign = "+" | "-"  ;
WhiteSpace = " " | "\t" , { " " | "\t" } ;
Special = "-" | ":" ;
EOL = { "\r" },  "\n" , {  "\r"|"\n"  } 

Thanks to Timotei Dolean for providing his works about WML grammar to Eclipse plugin.

SoC Application

- Battle for Wesnoth - Google Summer of Code Application