Machine Learning Recruiter

This page documents the new machine learning recruiter submitted as a patch for Wesnoth 1.11. We describe how to run it, discuss experiment showing that the ML Recruiter achieves dramatically better performance than the RCA AI recruiter, describe known issues and suggest a development road map.

Note that the ML Recruiter is a work in progress. We welcome feedback on it. Please discuss it on the thread "Machine Learning Recruiter" at http://forums.wesnoth.org/viewtopic.php?f=10&t=36642.

Why include ML Recruiter in Wesnoth?

The ML Recruiter makes use of a small subset of the Waffles Machine Learning toolkit adding 13 pairs of .cpp/.h files to Wesnoth. In addition, the neural nets used by ML Recruiter are serialized as .json files, which is a format Wesnoth has not yet contained. So why is this patch worthwhile?

Why the ML Recruiter will be great for Wesnoth

1. Performance is great. ML Recruiter defeats RCA AI 70 - 71% of the time. Although we don't have data, this should translate into better performance against humans
2. This superior performance is achieved with comparable "fun factor"
   • Variety of units recruited by recommended ML Recruiter is comparable to or better than RCA AI
3. Don't need to eliminate RCA AI recruiter. Campaign designers can choose to use one or the other
4. ML Recruiter should be easier to customize than RCA AI because
   • All core logic is in Lua, which is easier to modify than existing C++
   • Performance "out of the box" on known units likely to be strong
   • When new recruitable units are introduced by campaign designers, it can be trained by running c. 600 games in two hours. The new model is included as a .json file with the campaign data
   • Plug and play architecture of machine learning "features" easily allows minor modifications to mainline recruiter or to campaign-specific recruiters
5. Easy way to adjust campaign difficulty: Adjusting ML AI for more/less randomness makes is easier/harder to defeat
6. Inclusion of ML Recruiter could lead to greater publicity and more contributors to Wesnoth
   1. SeattleDad plans to submit this work as a scientific paper to a conference such as Computational Intelligence in Games
   2. Others might later build on this work by, for instance, trying ML algorithms other than neural nets, adding new features, further generalizing the algorithm, etc.
   3. The machine learning infrastructure is not specific to recruiting and could be repurposed for, for instance, attack planning, weapon selection, and making "retreat and heal" vs. "attack" decisions
   4. All of the above is potentially publishable research, so Wesnoth could attract contributions from computer science graduate students
      1. Note that, having established the basic framework with this patch, future work on machine learning will be much easier

Using ML Recruiter

Playing against the ML Recruiter

1. From the main menu, choose "Multiplayer"
2. Choose "Local Game"
3. Pick a map and adjust settings as desired. ML Recruiter has been trained with the default setting for village gold and support, but it should work fine on other settings
   1. Hit Okay
4. For one side, Choose Player/Type-->Computer Player and then either ML AI (Recommended) or ML AI (Pure)
   1. For the opponent, either play against it yourself (pick your name) or watch it play the default AI (Computer Player-->RCA AI)

Testing the ML Recruiter in batch mode

Testing in batch mode is easy. After applying the ML Recruiter patch, copy utils/ai_test/ai_test2.cfg to the directory in which you want to run the experiment. Then edit the first line of the .cfg file, "path_to_wesnoth_binary" to point to your Wesnoth executable. Then adjust faction1 and faction2 to point to the factions you want to experiment with and point ai_config1 at the ML configuration file you want to try out. Finally, to make everything easier, add the following to your path:

[Wesnoth_Install]/utils/ai_test/

Now you can test Wesnoth in batch as follows:

ai_test2.py ai_test2.cfg

Experiments: ML Recruiter vs. RCA AI Recruiter

http://imagebin.org/223121

Known issues

Bugs

1. Haven't added new Waffles files to Visual C++, so it won't compile under VC++. I need some help with this.
2. ML Recruiter runs fine vs. RCA AI and vs. human, but can't run against itself (ML Recruiter vs. ML Recruiter). This appears to be an issue with Lua in Wesnoth 1.11 and not with ML Recruiter

Current Limitations

1. Only tested on two-player multiplayer games. Doesn't work when there are more than two leaders on the map.
2. Works optimally on the following two-player maps (trained on these)
   1. Weldyn Channel
   2. The Freelands
   3. Den of Onis
   4. Fallenstar Lake
3. Tested on all other two-player maps and runs nearly as well except it crashes on the following:
   • Aethermaw, Hornshark Island, Dark Forecast, Sablestone Delta, Elensefar Courtyard, Silverhead Crossing
4. Currently writes log messages as "print" statements to stdout. I need some advice on this. I've added a new method called "ai_log_message" to core.cpp to allow Lua to write to the "ai/engine/lua" log domain, but would like some advice on whether this is a good idea