SummerOfCodeProposal corn

Introduction

Hi, I am Greg Shikhman and I'd like to work on and improving stats.wesnoth.org.

Preferred Email

cornmander@cornmander.com

Nicknames

• IRC - corn
• Wesnoth Forums - cornmander

Why I want to participate

Working on Wesnoth for GSoC would be my first experience working on an large and public open source project. GSoC is a structured way (similar to an internship at a company) for me to get into the development of this project.

Studies

I am a high school senior who will be attending an accredited university this fall. This section will be more definite when I decide where I want to go :)

Patches

https://gna.org/patch/?1153 https://gna.org/patch/?1149

1 or 2 more patches to come related to bug #13094.

Experience

Last summer I had a full time summer internship at Morgan Stanley, the financial company. My primary accomplishment was the creation and maintenance of a statistics interface for their networking hardware. I created a system of scripts that handled statistics acquisition, and I wrote a java servlet that showed these statistics, with nice looking graphs via RRDTool. The tool was put into production shortly after my intership ended.

A smaller task that I also handled was the creation of a real-time packet rewrite/editing script on the network hardware, to support and upgrade legacy software.

Reference is available on request.

As part of a class, I have also created a real time rasterizer, with model loading, animation, and texture mapping. A scripting language for inputting animation sequences was also developed as part of the project. Source code, along with instructions for compiling are available at http://cornmander.com/dragonballx.tar.gz .

I also run my own server at http://cornmander.com . It is a colocated Gentoo Linux box. I have working sysadmin experience.

Programs / Software

• python
• LaTeX
• GNU make
• gdb
• vi(m)
• linux
• subversion
• gcc
• grep, sed, awk, other standard shell hacker tools :)
• bison, flex
• libraries: opengl, sdl, opencv
• languages: c, python, php, java, latex, html, tcl

Team Environment Experience

My rasterizer was created as part of a team project. I managed and handed out tasks to my teammates, and tried to keep them updating their TODOs.

My internship at Morgan Stanley was done under the supervision of my manager. Although I was working on my project alone, I took suggestions and feature requests from the end users.

Open Source

Involvement

I've reported bugs to the Gentoo Linux project, and I am credited for reporting a minor security bug: http://www.derkeiler.com/Mailing-Lists/Full-Disclosure/2007-08/msg00364.html .

I follow the development of ioquake and mplayer, but I haven't had a chance to contribute to either project. I may try to enter the ffmpeg (an integral part of mplayer) GSoC process next year.

Gaming Experience

I have been a PC gamer since late elementary school. I am a hardcore gamer primarily interested in FPS games - this is what sparked my interest in working on cutting edge graphics tech like rasterization. My current addiction is TF2. I also appreciate RPG games. I played Wesnoth for a few weeks with my friends after the Wesnoth 1.5 release, and I will start playing again now.

What type of Gamer

I used to play video games for a few hours per week, but school has cut it down to 2-4 per week. Nevertheless, I'd categorise myself as a hardcore gamer.

Preferred Opponents

I like playing versus friends, or at the very least versus people. AI isn't good enough to adapt and get better as you play, and there is always the joy of rubbing in your victory to the face of a human opponent.

Story or Gameplay?

Gameplay is king. I am used to playing multiplayer games, and the game mechanics are much more important than plot are backstory. Nevertheless, I do enjoy reading a good story or being a part of it in a single player campaign. Co-op support is an interesting idea.

Have I played Wesnoth

I played 1.5 multiplayer for a few weeks, and played through the Heir to the Throne campaign.

Communications

English

I am a native english speaker, and I am extremely fluent in Russian as well, although I can't read or write it.

Player Interaction

I ignore trolls, and I haven't been involved in player communities for a while. I lurk on forums, but I talk on IRC.

Constructive Advice

I take criticism well, but sometimes I am impatient when I don't see the point to a suggestion/requirement.

Receive Advice?

My priority is getting the job done. I don't take criticism personally.

Sorting out Criticism

The people I work with are typically more experienced and smarter than I am. Although I have the urge to question a suggestion, I'll never dismiss a suggestion without giving it some consideration.

Project

Which Project

I want to improve the stats server.

Why

I have existing experience with displaying stats, and the freedom of tools and programming languages should make it easy for me to hit the ground running.

Technical Details

Frontend will be written in python with the turbogears application platform (for the nice templating language support). Databases for stats data and any user/configuration settings for the frontend will be in MySQL. Charting will be done using Google Charting API.

Changes will be made to the wesnoth client so that it keeps an array or a map where a list of dead units is kept for each tile. This data will be sent along with stats so that killgraphs can be generated. A program will also be written to take wesnoth maps, generate PNG files of them (if there isn't already such functionality) and color in each tile according to how many units were killed on it. This will be generated nightly and used for the killgraphs.

The stats interface will scale up to millions of rows by operating on a randomly taken subset of the data grabbed nightly. If the subset turns out not to have data for a particular set of filters, the interface will revert back to the bigger database.

Goals

PRIORITY	SUBPROJECT	RESULT	PROGRESS
MUST	Basic stats.wesnoth.org website	Create a better formatted version of my prototype at http://cornmander.com:9090	Completely rewrote the code for generating graph pages. Everything is driven by a database and new graphs take less than a minute to create. I still need to create a page for adding new graphs.
MUST	Add new stat windows to the website	Created a recruited units window, a dead units window, and a unit by level breakdown in addition to improving the things currently in the prototype. Add date ranges to all the charts on the site.	Added date range selection to all the charts. The table in the database with the unit information still isn't being used. Adding new graphs is quick and easy right now, but only piecharts and linegraphs are supported.
MUST	Improve filter behavior	Make the filters automatically remove choices with 0 entries based on currently selected filters	Not being done. This may be too performance-intensive to accomplish.
MUST	Remove outlier data	Give statistics for the median 80%, leaving out the outlying 10% below and 10% above.	Not implemented yet. I need to think about how to do this.
MUST	Add scalability	Make the statistics presentation scalable to the millions of rows currently on stats.wesnoth.org. I plan to do this by creating a 10k random-row table subset of the data in a nightly cronjob and using this by default for all queries. If a query gives under CONSTANT_NUMBER results, the site will use the main database instead.	Done. I created a script that generates subset tables of 10k, 100k, and 1000k rows. I wrote a wrapper around my SQL queries that tries these tables first and checks the size of the result (using a library of evaluators). If the result is too small, it queries the next largest size table. Queries take less than a second on average.
MUST	Kill maps	Create a section of stats that gives a breakdown by tile on a scenario of the units killed on that tile. The presentation will be a jscript-enhanced map where a user can hover over a tile and see a percentage breakdown of the units killed on that tile. In addition to website coding, significant changes must be made to stat collection.	Not done yet. I will start on this after I finish linegraphs, the addview page, and login support.
MUST	CSS and Design	Integrate the look of the stats page with the rest of wesnoth.org . Make the website user-friendly
MUST	Email subscription to charts	Create a login system so that users can subscribe to particular charts, and receive daily emails for comparison. This will be useful for basic AI regression testing, ex: run 10 different campaigns with an AI during a particular day and compare with the charts of the previous day	Not done yet. All the site functionality should be laid down before I start skinning it.
GOOD	Geographic data	The server already gets data on user IPs by accepting logs from them. Create a display page where a map is shown colored in according to how many users play from each country.
GOOD	Get MP data	Get the same data on scenario completion, platform breakdown, win/loss/quit information from MP matches.
GOOD	GUI or scripting interface for hooking in new stat pages	Create an interface for looking at new stats. User will have a choice of several graph types and what data sets to use for x and y axes.	Done. This is the database driven graph stuff.

What I expect to get from this Project

I want to have done a substantial open source contribution. This project will be my first coordinated team effort and hopefully help me get started on other OSS projects. It will also expose me to the process of maintaining a large, polished product.

Would I stay with Wesnoth after GSoC?

I would maintain my code for Wesnoth. I think that I will continue to contribute code in unrelated areas to the project after I am done the project as well.

Practical considerations

Familiarity with Tools

• Subversion - I know how to use svn, and I run it on my own server as well.
• C++ - I know C and Java very well, but I don't know C++. I think I can learn it over the course of a week or two, however.
• Python - I know python very well.
• Build Environments - My development environment is linux with GNU make. However, I have used MSVS before but I am not familiar with their solution files. I have a pretty good idea of how to use SCons now.

Development Tools

I use vi, xterm, svn, and GDB. I dislike working in IDEs because of the overhead and effort of getting your project set up properly in them.

Fluent Programming Languages

• Java
• C
• Python
• PHP

I also know LaTeX and HTML, markup languages.

Fluent Spoken Languages

I am a native english speaker. I also am fluent in Russian, but can't read or write it.

Hours of Availability

Until the end of June, I am available from 6 PM EST (22:00 UTC) to 3 AM EST (7:00 UTC). I will either be available during the entire day of 9 PM EST (1:00 UTC) to 10 PM EST (21:00 UTC). I expect to put in full work days of 8 hours on this project.

Phone / Internet Phone Conversations

I would prefer written electronic communication like IRC or email, but I am OK with talking over the phone. It's a bit frustrating to explain your code over the phone, however.