View source for GSoC 2011 Sytyi

{{SoC2011Student_2|Sytyi|SoC_Ideas_Schema_Validation2011}}
== Description ==
<h4>Sytyi - WML Validator</h4>
The main idea of WML Validator is grammar based validator on C++ generating '''USER-FRIENDLY''' error messages in gcc style.

==IRC==
Sytyi
== Contacts ==
'''IRC:'''  Sytyi<br>
'''E-mail:''' nsytyi[at]gmail.com<br>
'''Skype''': azcure<br>
'''ICQ:''' four-zero-six-one-four-four-one-seven-four<br>

== Proposal Summary ==
I want to write a grammar based validator. One tool will prepare schema file containing all tags, keys and possible values from annotated source.  Validator will read schema file on the beginning of the work and then, while parsing, just compare is this value allowed in this place.

Main idea is the hierarchical structure. Data storing allows to check key value, just by comparing with possible values. Also some simple types as string, integer, boolean can be possible too.

There are two problems:
# Tags with different specific but one name. I think hierarchy will solve this problem but make another one. Each tag will have     
# Generally used tags like [action], [event], [filter]. They will be organized in global list. And list tags will have they mentioned. With this, hierarchy is broken a little, but it is still not graph in the widely understanding.

I also want to include Schema Generator in build system.
I suppose, SG to update schema file, not to "delete-and-again" generation. It is possible cause of using hierarchy tag structure and annotations.
WIP proof-on-concept SG example here https://gna.org/patch/?2636

== Format of annotations and schema file ==
See [[WML_Annotation_Format]] for details.


== How It All Work? ==
After a patch the developer runs SchemaGeneratorTool to regenerate schema file. I want to include the special build target in building system, trying to avoid the whole regenerating, only current updates.
Also see [[ValidationDesign]]
 
If somebody want to validate his WML document he runs ValidatorTool, and receives Error Messages like 
 source-file-name:lineno:column: In file %filename
 source-file-name:lineno:column: in  toplevel tag [scenario]
 source-file-name:lineno:column: Mandatory key "id" is misssing
 source-file-name:lineno:column: in tag [side] 
 source-file-name:lineno:column: Mandatory key "side" is misssing  
 source-file-name:lineno:column: Unknown value in key "controller"  %wrong_value found, but %value_list possible


== Build System ==
I want to include Schema generating tool to build systems. This is not preference idea, only an add-on part. I want to make schema file refreshable (one source file was changed and only that part of schema will be updated), not "delete-and-again", but there are a few problems. If a tag or a key is removed from the source, how th SGT know about it? I think two ways: one to add source files names to schema file (this can be a little help to developer, trying to fix some bugs in code ), and second: to add special annotation to deleted keys.

== Timeline ==
<b>8.04 - 22.05 Getting into the project time</b><br>
Improving annotations list. Getting as close as possible to WML documents. Studying how parser works.


  
<b>23.05 - 30.06 Schema generating tool<br></b>
I have exams in this period of time, and max number of exams can be 5, so I can't plan much more. And I even have no timeline of exams, it will be available only in June.
#23.05 - 15.06 Annotation parsing part.<br>
#16.06 - 30.06 Schema generating part.<br>

<b>1.07 - 15.08 Document validating tool</b><br>
# 1.07 - 10.07  Hierarchy structures. 
# 11.07 - 25.07 Document parsing part.
# 26.07 - 5.08  Validation.
# 6.08 - 10.08  Data structures refactoring. (If nesessary). 
# 10.08 - 15.08 Errors messages generating part. 
# 15.08 - 20.08 Testing. Fixing bugs.


<b>16.08 - 31.08 Afterwork</b><br>
#Testing<br>
#Fixing bugs<br>
#Documentation<br>

<b><If any time left></b><br>
Improving build-systems integration, making more error types.

<b>In far future</b><br>
Maybe a validator for unpreprocessed WML.

== Stages ==

{| border="1"
! ID
! PART OF
! NAME
! PRIORITY
! LAST FINISH TIME
! STATUS
|-
| 0
| 
| Proof-of-concept patch
| GREATEST
| 15.04
| IN PROCESS
|-
| 
|
| MILESTONE = "PREPARE"
| GREAT
| 23.05 
| 
|-
| 1
| PREPARE
| annotate a part of existing code
| IMPORTANT
| 23.05
| 90%
|-
| 2
| PREPARE
| improve annotations list due to nessesarity. view stage 2
| VERY IMPORTANT
| 23.05
| 90%
|-
| 3
| PREPARE
| write documentation and examples of annotations in wiki
| BASIC  
| 23.05
| 90%
|-
| 
| 
| MILESTONE = SCHEMA GENERATOR
| GREAT 
| 30.06
| 
|-
| 1
| SG 
| IMPLEMENTATION working mainstream source to SG
| BASIC  
| 20.06
| 90% NEED REVISION
|-
| 2
| SG
| [[WML_Annotation_Format|DOCUMENTATION]] including extended examples of annotations, algorithm describing, and list of used regexes
| BASIC  
| 30.06
| 60% 
|-
| 3
| SG
| global tags support
| BASIC
| 30.06
| 100%
|-
| 
| 
| MILESTONE = VALIDATOR
| GREAT 
| 15.08
| 
|-
| 1
| VALIDATOR
| parsing schema file. Build wml tags tree
| BASIC
| 10.07
| 100%
|-
| 2
| VALIDATOR
| parsing wml file part. Using /src/serialisation/ (Optionally)
| BASIC
| 25.07
| NEED DISCUSSION 
|-
| 3
| VALIDATOR
| checking against tree. 
| BASIC
| 5.08
| 10%
|-
| 4
| VALIDATOR
| generating gcc-style errors implementation
| BASIC
| 15.08
| 5%
|-
| 5
| VALIDATOR
| working tool 
| BASIC
| 15.08
| 
|-
| 6
| VALIDATOR
| documentation about data containers, hierarchy, algorithms and input/output specification. 
| BASIC
| 25.08
| 
|-
| 
| SG
| BUILD system support
| ADDITIONAL  
| 
| NEED DISCUSSION
|-
| 
| SG
| refreshable schema file support
| ADDITIONAL  
| NOT NEEDED
| 
|-
| 
| SG
| extend error messages
| ADDITIONAL  
| 
| 
|-
|}

BASIC PRIORITY means that milestone without that stage just cannot be mentioned
The milestones after 30.06 schould be improved with more information, non mainstream milestones and so one. It's a bit hard nowadays. The more work - the more problems I explore.

== Patches and commits ==
No one at this moment of time.


==Answers to those questions==
<h3>Basics</h3>

<b>Write a small introduction to yourself.</b><br/>
Hello. My name is Nick Sytyi . I study Computer Engineering at the Chernigiv Technological University. I want to participate in Google Summer of Code and gain expierence in C++ and game development. I like this project and I want to encahnce it.  

<b>1.2) State your preferred email address.</b><br/>
nsytyi@gmail.com 

<b>1.3) If you have chosen a nick for IRC and Wesnoth forums, what is it?</b><br/>
IRC: Sytyi <br/>
Wesnoth forums: none for now
Wesnoth wiki: Sytyi <br/>
GNA: sytyi <br/>

<b>1.4) Why do you want to participate in summer of code?</b><br/>
I would like to try some serious work. I want to grow in C++ development, open source, team work and game development experience.
I think the experience I gain here, will help me in future.
I hope I come in project  for long period of my life, because it is my dream (to develop a game) since when I was child.

<b>1.5) What are you studying, subject, level and school?</b><br/>
Computer Engineering, 3rd year Faculty of Electronic Informational Technologies, Chernigiv Technological University, Ukraine.

<b>1.6) If you have contributed any patches to Wesnoth, please list them below. You can also list patches that have been submitted but not committed yet and patches that have not been specifically written for Wesnoth. If you have gained commit access to our S&shy;&shy;V&shy;&shy;N (during the evaluation period or earlier) please state so.</b><br/>

No patches for now. Maybe the situation will change for better.

<h3>Experience</h3>
good knowledge of C++, Java, computer science concepts, small experience of programming translators, olympiad projects with MCS-51 and AVR stends (I know it is insignificant but it gave me experience in quick, well synchronized C programs).

<b>2.1) What programs/software have you worked on before?</b><br/>
Mostly, I'am working on  website development using Java EE. Also, I've done little C++ projects writing simple data manipulating programs.

<b>2.2) Have you developed software in a team environment before? (As opposed to hacking on something on your own)</b><br/>
Yes. Our university has command projects, and now I am a team leader for a simple course work So I was to learn S&shy;&shy;V&shy;&shy;N features.

<b>2.3) Have you participated to the Google Summer of Code before? As a mentor or a student? In what project? Were you successful? If not, why?</b><br/>
I haven't participated in the GSoC before. This is my first  chance to participate.

<b>2.4) Open Source</b><br/>

<b>2.4.1) Are you already involved with any open source development projects? If yes, please describe the project and the scope of your involvement.</b><br/>

I use Ubuntu, Mozilla, Eclipse, Qt , but Wesnoth is the first open source project I am involved with.

<b>2.5) Gaming experience - Are you a gamer?</b><br/>
Yes, I am a gamer )

<b>2.5.1) What type of gamer are you?</b><br/>
Slow strategic gamer. 

<b>2.5.2) What type of games?</b><br/>
I prefer turn-based strategy games or RPG with good story.

<b>2.5.3) What type of opponents do you prefer?</b><br/>
I can’t say  I play good. So I prefer good opponents which sometimes teach me some tricks. When I’m beaten, I analyze, why this occurred. 

<b>2.5.4) Are you more interested in story or gameplay?</b><br/>
Story is more interesting for me. But if gameplay is very booooring ………….

<b>2.5.5) Have you played Wesnoth? If so, tell us roughly for how long and whether you lean towards single player or multiplayer.
We do not plan to favor Wesnoth players as such, but some particular projects require a good feeling for the game which is hard to get without having played intensively.</b><br/>
I have heard  about Wesnoth first time at 18 March. It interested me, and sometimes, when I need a rest, I play a little.

<h3>Communication skills</h3>

<b>3.1) Though most of our developers are not native English speakers, English is the project's working language. Describe your fluency level in written English.</b><br/>
I can read and write English with few mistakes. 

<b>3.2) Are you good at interacting with other players? Our developer community is friendly, but the player community can be a bit rough.</b><br/>
I just ignore rough statements in address. In a team I takes the second role.

<b>3.3) Do you give constructive advice?</b><br/>
Maybe. Sometimes.

<b>3.4) Do you receive advice well?</b><br/>
Yes

<b>3.5) Are you good at sorting useful criticisms from useless ones?</b><br/>
Yes

<h3>Project</h3>

<b>4.1) Did you select a project from our list? If that is the case, what project did you select? What do you want to especially concentrate on?</b><br/>
Yes. It is interesting to make tools for non-programmers content developers. I want to concentrate on grammar of WML. And if I have free time after all of this, maybe a simple WML GUI or including WML validation into existing GUI’s. 

<b>4.2) If you have invented your own project, please describe the project and the scope.</b><br/>

<b>4.3) Why did you choose this project?</b><br/>
When I studied Google list of participating organizations, I read about your game first, I played a little, and then I want to enchance project. 
I am interesting in translations. WML validation seemed to me a perfect task to study the main project and a perfect task for a starter not confident in his abilities.  

<b>4.4) Include an estimated timeline for your work on the project. Don't forget to mention special things like "I booked holidays between A and B" and "I got an exam at ABC and won't be doing much then".</b><br/>
I'll have 4 or 5 exams in June, but I hope to pass 3 of them out by auto.<br/>
3-4 days in archaelogic expedition near 15 of August.<br/>
Two times 3-4 days on bike down the river in July. I can not say  current time now, it depends on the last exam and my friends. More possible it will be after stipendy at  25 of each month<br/>  

<b>4.5) Include as much technical detail about your implementation as you can</b><br/>
I want to write two tools. One to generate Schema file from the annotated source. and another to validate WML documents using schema. Tags are organized in a hierarchical structure.<br>     
<b>4.6) What do you expect to gain from this project?</b><br/>
I expect to gain experience in game development and maybe a continious work with Wesnoth.

<b>4.7) What would make you stay in the Wesnoth community after the conclusion of SOC?</b><br/>
Willing to “upgrade” myself. To know more. To work with interesting people on the interesting tasks.

<h3>Practical considerations</h3>

<b>5.1) Are you familiar with any of the following tools or languages?</b><br/>

 <b>* Sub&shy;&shy;version (used for all commits) Yes.<br/>
 * C++ (language used for all the normal source code)Yes STL. QT. <br/>
 * Python (optional, mainly used for tools) no. <br/>
 * build environments (eg cmake/autotools/scons) 'make' - yes, 'scons' no. 'autotools' - no<br/>
 I don't know LUA and i'm studying WML now.</b><br/>
<b>5.2) Which tools do you normally use for development? Why do you use them?</b><br/>
Eclipse under Windows and Ubuntu. Qt. Notepad. 

<b>5.3) What programming languages are you fluent in?</b><br/>
I have a little practice in Java (1 years of experience) and C/C++ (2 years of experience)
I know a bit of sh/javascript/pascal/c

<b>5.4) What spoken languages are you fluent in?</b><br/>
I can speak in Ukrainian (native), in Russian (native), English.

<b>5.5) At what hours are you awake and when will you be able to be in IRC (please specify in UTC)</b><br/>
I'm in UTC+3 timezone in summer. I am normally awake from +11 UTC to +02 UTC, and most of that time I'm at home or some anywhere with wi-fi, so I'm able to be in IRC.

<b>5.6) Would you mind talking with your mentor on telephone / internet phone? We would like to have a backup way for communications for the case that somehow emails and IRC do fail.</b><br/>
Skype : azcure
 
== Grammar == 
I wrote grammar cause of misunderstood. So it does not influence the main idea, etc. But it may be useful to some people. 

<b>Grammar needs some improvements in types of key values, but the main work is done.<br> If you see any mistake here, please contact me</b><br>
<b>EBNF form for unpreprocessed WML</b> http://en.wikipedia.org/wiki/EBNF <br>

 Document = [ Header ], { StmntList } ;
 StmntList = Statement, EOL, { StmntList } ;
 Statement = { ( Tag | MacroCall | MacroDefine | TextDomain | PreprocIf ) }  ;
 Tag = TagStart { ( Tag | Key | MacroCall | MacroDefine | TextDomain | PreprocIf ) }, TagEnd ;
 Key = Name, { Name} ,"=", KeyValue, { KeyValue },  EOL;
 MacroCall = "{", Name," ",{ " " }, MacroValue {, MacroValue} , "}"
 MacroDefine = DEFINE, Name, MacroVar {, MacroVar} , StmntList, "ENDDEF";
 TextDomain = "#textdomain ", Name ;
 PreprocIf = (IFBegin, StmntList, [IFElse, StmntList,] IFEnd) | (Undef, MacroName);
 TagStart = "[",[ "+" ], TagName , "]" ;
 TagEnd = "[/", TagName ,"]" ;
 (* Can TagName contain uppercase?  *)
 TagName = LowerCaseLetter | "_", { LowerCaseLetter | Special  | "_"};
 KeyValue = ( Name | Substitution | Strings );
 Substitution = "$",  (Name | Formula) ;
 Formula = "(", Name, ")" ;
 MacroVar = Name;
 IFBegin = "#ifdef " | "#ifhave " | "#ifndef " | "#ifnhave " | "#ifver " | "#ifnver " ;
 IFElse = "#else " ;
 IFEnd = "#endif" ;
 DEFINE = "#define " ;
 ENDDEF = "#enddef" ;
 Name = Letter, { Letter };
 Identifier = Letter, { Letter | Digit | Special };
 Strings = ( String | TranslatedString ), { "+", ( String | TranslatedString ) };
 String = '"', { ( Letter | Digit | Symbols) },'"';
 TranslatedString = "_", String;
 Letter = (UpperCaseLetter | LowerCaseletter | "_" );
 UpperCaseLetter = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z" ;
 LowerCaseLetter = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" ;
 Digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
 Sign = "+" | "-"  ;
 WhiteSpace = " " | "\t" , { " " | "\t" } ;
 Special = "-" | ":" ;
 EOL = { "\r" },  "\n" , {  "\r"|"\n"  } 

<b>EBNF form for preprocessed WML</b> <br>

 Document = [ Header ], { StmntList } ;
 StmntList = Statement, EOL, { StmntList } ;
 Statement = { ( Tag | TextDomain ) }  ;
 Tag = TagStart { ( Tag | Key ) }, TagEnd ;
 Key = Name, { Name} ,"=", KeyValue, { KeyValue },  EOL;
 TextDomain = "#textdomain ", Name ;
 TagStart = "[",[ "+" ], TagName , "]" ;
 TagEnd = "[/", TagName ,"]" ;
 (* Can TagName contain uppercase?  *)
 TagName = LowerCaseLetter | "_", { LowerCaseLetter | Special  | "_"};
 KeyValue = ( Name | Substitution | Strings );
 Substitution = "$",  (Name | Formula) ;
 Formula = "(", Name, ")" ;
 Name = Letter, { Letter };
 Identifier = Letter, { Letter | Digit | Special };
 Strings = ( String | TranslatedString ), { "+", ( String | TranslatedString ) };
 String = '"', { ( Letter | Digit | Symbols) },'"';
 TranslatedString = "_", String;
 Letter = (UpperCaseLetter | LowerCaseletter | "_" );
 UpperCaseLetter = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z" ;
 LowerCaseLetter = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z" ;
 Digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ;
 Sign = "+" | "-"  ;
 WhiteSpace = " " | "\t" , { " " | "\t" } ;
 Special = "-" | ":" ;
 EOL = { "\r" },  "\n" , {  "\r"|"\n"  } 

[[Category:Summer of Code]]

=SoC Application=
[http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/nicksytyi/1 - Battle for Wesnoth - Google Summer of Code Application]