Machine Learning Recruiter

This page documents the new machine learning recruiter submitted as a patch for Wesnoth 1.11. We describe how to run it, discuss experiment showing that the ML Recruiter achieves dramatically better performance than the RCA AI recruiter, describe known issues and suggest a development road map.

Note that the ML Recruiter is a work in progress. We welcome feedback on it. Please discuss it on the thread "Machine Learning Recruiter" at http://forums.wesnoth.org/viewtopic.php?f=10&t=36642.

Why include ML Recruiter in Wesnoth?

The ML Recruiter makes use of a small subset of the Waffles Machine Learning toolkit adding 13 pairs of .cpp/.h files to Wesnoth. In addition, the neural nets used by ML Recruiter are serialized as .json files, which is a format Wesnoth has not yet contained. So why is this patch worthwhile?

Why the ML Recruiter will be great for Wesnoth

1. Performance is great. ML Recruiter defeats RCA AI 70 - 71% of the time. Although we don't have data, this should translate into better performance against humans
2. This superior performance is achieved with comparable "fun factor"
   • Variety of units recruited by recommended ML Recruiter is comparable to or better than RCA AI
3. Don't need to eliminate RCA AI recruiter. Campaign designers can choose to use one or the other
4. ML Recruiter should be easier to customize than RCA AI because
   • All core logic is in Lua, which is easier to modify than existing C++
   • Performance "out of the box" on known units likely to be strong
   • When new recruitable units are introduced by campaign designers, it can be trained by running c. 600 games in two hours. The new model is included as a .json file with the campaign data
   • Plug and play architecture of machine learning "features" easily allows minor modifications to mainline recruiter or to campaign-specific recruiters
5. Easy way to adjust campaign difficulty: Adjusting ML AI for more/less randomness makes is easier/harder to defeat
6. Inclusion of ML Recruiter could lead to greater publicity and more contributors to Wesnoth
   1. SeattleDad plans to submit this work as a scientific paper to a conference such as Computational Intelligence in Games
   2. Others might later build on this work by, for instance, trying ML algorithms other than neural nets, adding new features, further generalizing the algorithm, etc.
   3. The machine learning infrastructure is not specific to recruiting and could be repurposed for, for instance, attack planning, weapon selection, and making "retreat and heal" vs. "attack" decisions
   4. All of the above is potentially publishable research, so Wesnoth could attract contributions from computer science graduate students
      1. Note that, having established the basic framework with this patch, future work on machine learning will be much easier

Using ML Recruiter

Playing against the ML Recruiter

1. From the main menu, choose "Multiplayer"
2. Choose "Local Game"
3. Pick a map and adjust settings as desired. ML Recruiter has been trained with the default setting for village gold and support, but it should work fine on other settings
   1. Hit Okay
4. For one side, Choose Player/Type-->Computer Player and then either ML AI (Recommended) or ML AI (Pure)
   1. For the opponent, either play against it yourself (pick your name) or watch it play the default AI (Computer Player-->RCA AI)

Testing the ML Recruiter in batch mode

Testing in batch mode is easy. After applying the ML Recruiter patch, copy utils/ai_test/ai_test2.cfg to the directory in which you want to run the experiment. Then edit the first line of the .cfg file, "path_to_wesnoth_binary" to point to your Wesnoth executable. Then adjust faction1 and faction2 to point to the factions you want to experiment with and point ai_config1 at the ML configuration file you want to try out. Finally, to make everything easier, add the following to your path:

[Wesnoth_Install]/utils/ai_test/

Now you can test Wesnoth in batch as follows:

ai_test2.py ai_test2.cfg

Experiments: ML Recruiter vs. RCA AI Recruiter

Chart showing relative winning percentages

• RCA AI: The Default AI. Wins 50% of the time against itself (of course)
• Random: Recruiting units are chosen completely at random. Wins 45.5% of the time overall
• Recommended ML Recruiter: Units are chosen at random weighted by their relative value. Wins 70% overall.
• Pure ML Recruiter: ML AI always chooses the unit it thinks is best. Wins 71.5% of the time overall, but is might be seen as boring since it can produce armies which are overwhelmingly one or two units.

Chart showing winning percentage of RCA AI, Random, Recommended, and pure ML Recruiters

Winning percentages for Recommended ML Recruiter

$ analyze_log.py *.log -d default_ai_with_recruit_log -m Overall Stats Win % Wins AI 30.0% 1076 "default_ai_with_recruit_log 70.0% 2506 "ml_ai_faction Total: 3582 Drakes vs Undead 37 43 46.2% Drakes vs Northerners 44 75 37.0% Drakes vs Loyalists 69 17 80.2% Drakes vs Knalgan Alliance 52 54 49.1% Drakes vs Drakes 66 40 62.3% Drakes vs Rebels 77 17 81.9% Total Drakes 345 246 58.4%

Knalgan Alliance vs Undead 74 19 79.6% Knalgan Alliance vs Northerners 29 73 28.4% Knalgan Alliance vs Loyalists 93 16 85.3% Knalgan Alliance vs Knalgan Alliance 67 37 64.4% Knalgan Alliance vs Drakes 83 39 68.0% Knalgan Alliance vs Rebels 75 18 80.6% Total Knalgan Alliance 421 202 67.6%

Loyalists vs Undead 25 73 25.5% Loyalists vs Northerners 24 67 26.4% Loyalists vs Loyalists 73 20 78.5% Loyalists vs Knalgan Alliance 57 50 53.3% Loyalists vs Drakes 61 27 69.3% Loyalists vs Rebels 55 48 53.4% Total Loyalists 295 285 50.9%

Northerners vs Undead 91 5 94.8% Northerners vs Northerners 72 26 73.5% Northerners vs Loyalists 91 2 97.8% Northerners vs Knalgan Alliance 83 2 97.6% Northerners vs Drakes 67 12 84.8% Northerners vs Rebels 107 6 94.7% Total Northerners 511 53 90.6%

Rebels vs Undead 81 19 81.0% Rebels vs Northerners 34 64 34.7% Rebels vs Loyalists 97 6 94.2% Rebels vs Knalgan Alliance 92 23 80.0% Rebels vs Drakes 68 32 68.0% Rebels vs Rebels 84 14 85.7% Total Rebels 456 158 74.3%

Undead vs Undead 65 29 69.1% Undead vs Northerners 39 42 48.1% Undead vs Loyalists 108 9 92.3% Undead vs Knalgan Alliance 94 9 91.3% Undead vs Drakes 90 19 82.6% Undead vs Rebels 82 24 77.4% Total Undead 478 132 78.4%

Known issues

Bugs

1. Haven't added new Waffles files to Visual C++, so it won't compile under VC++. I need some help with this.
2. ML Recruiter runs fine vs. RCA AI and vs. human, but can't run against itself (ML Recruiter vs. ML Recruiter). This appears to be an issue with Lua in Wesnoth 1.11 and not with ML Recruiter

Current Limitations

1. Only tested on two-player multiplayer games. Doesn't work when there are more than two leaders on the map.
2. Works optimally on the following two-player maps (trained on these)
   1. Weldyn Channel
   2. The Freelands
   3. Den of Onis
   4. Fallenstar Lake
3. Tested on all other two-player maps and runs nearly as well except it crashes on the following:
   • Aethermaw, Hornshark Island, Dark Forecast, Sablestone Delta, Elensefar Courtyard, Silverhead Crossing
4. Currently writes log messages as "print" statements to stdout. I need some advice on this. I've added a new method called "ai_log_message" to core.cpp to allow Lua to write to the "ai/engine/lua" log domain, but would like some advice on whether this is a good idea