Machine Learning Recruiter

This page documents the new machine learning recruiter submitted as a patch for Wesnoth 1.11. We describe how to run it, discuss experiment showing that the ML Recruiter achieves dramatically better performance than the RCA AI recruiter, describe known issues and suggest a development road map.

Note that the ML Recruiter is a work in progress. We welcome feedback on it. Please discuss it on the thread "Machine Learning Recruiter" at http://forums.wesnoth.org/viewtopic.php?f=10&t=36642.

Why include ML Recruiter in Wesnoth?

The ML Recruiter makes use of a small subset of the Waffles Machine Learning toolkit adding 13 pairs of .cpp/.h files to Wesnoth. In addition, the neural nets used by ML Recruiter are serialized as .json files, which is a format Wesnoth has not yet contained. So why is this patch worthwhile?

Why the ML Recruiter will be great for Wesnoth

1. Performance is great. ML Recruiter defeats RCA AI 70 - 71% of the time. Although we don't have data, this should translate into better performance against humans
2. This superior performance is achieved with comparable "fun factor"
   • Variety of units recruited by recommended ML Recruiter is comparable to or better than RCA AI
3. Don't need to eliminate RCA AI recruiter. Campaign designers can choose to use one or the other
4. ML Recruiter should be easier to customize than RCA AI because
   • All core logic is in Lua, which is easier to modify than existing C++
   • Performance "out of the box" on known units likely to be strong
   • When new recruitable units are introduced by campaign designers, it can be trained by running c. 600 games in two hours. The new model is included as a .json file with the campaign data
   • Plug and play architecture of machine learning "features" easily allows minor modifications to mainline recruiter or to campaign-specific recruiters
5. Easy way to adjust campaign difficulty: Adjusting ML AI for more/less randomness makes is easier/harder to defeat
6. Inclusion of ML Recruiter could lead to greater publicity and more contributors to Wesnoth
   1. SeattleDad plans to submit this work as a scientific paper to a conference such as Computational Intelligence in Games
   2. Others might later build on this work by, for instance, trying ML algorithms other than neural nets, adding new features, further generalizing the algorithm, etc.
   3. The machine learning infrastructure is not specific to recruiting and could be repurposed for, for instance, attack planning, weapon selection, and making "retreat and heal" vs. "attack" decisions
   4. All of the above is potentially publishable research, so Wesnoth could attract contributions from computer science graduate students
      1. Note that, having established the basic framework with this patch, future work on machine learning will be much easier

Using ML Recruiter

Playing against the ML Recruiter

1. From the main menu, choose "Multiplayer"
2. Choose "Local Game"
3. Pick a map and adjust settings as desired. ML Recruiter has been trained with the default setting for village gold and support, but it should work fine on other settings
   1. Hit Okay
4. For one side, Choose Player/Type-->Computer Player and then either ML AI (Recommended) or ML AI (Pure)
   1. For the opponent, either play against it yourself (pick your name) or watch it play the default AI (Computer Player-->RCA AI)

Testing the ML Recruiter in batch mode

Testing in batch mode is easy. After applying the ML Recruiter patch, copy utils/ai_test/ai_test2.cfg to the directory in which you want to run the experiment. Then edit the first line of the .cfg file, "path_to_wesnoth_binary" to point to your Wesnoth executable. Then adjust faction1 and faction2 to point to the factions you want to experiment with and point ai_config1 at the ML configuration file you want to try out. Finally, to make everything easier, add the following to your path:

[Wesnoth_Install]/utils/ai_test/

Now you can test Wesnoth in batch as follows:

ai_test2.py ai_test2.cfg

Experiments: ML Recruiter vs. RCA AI Recruiter

Chart showing relative winning percentages

• RCA AI: The Default AI. Wins 50% of the time against itself (of course)
• Random: Recruiting units are chosen completely at random. Wins 45.5% of the time overall
• Recommended ML Recruiter: Units are chosen at random weighted by their relative value. Wins 70% overall.
• Pure ML Recruiter: ML AI always chooses the unit it thinks is best. Wins 71.5% of the time overall, but is might be seen as boring since it can produce armies which are overwhelmingly one or two units.

Chart showing winning percentage of RCA AI, Random, Recommended, and pure ML Recruiters

Winning percentages for Recommended ML Recruiter

$ analyze_log.py *.log -d default_ai_with_recruit_log -m
Overall Stats 
Win %	Wins	AI
30.0%	1076	"default_ai_with_recruit_log
70.0%	2506	"ml_ai_faction
Total:	3582

Drakes vs Undead                 	37	43	46.2%
Drakes vs Northerners            	44	75	37.0%
Drakes vs Loyalists              	69	17	80.2%
Drakes vs Knalgan Alliance       	52	54	49.1%
Drakes vs Drakes                 	66	40	62.3%
Drakes vs Rebels                 	77	17	81.9%
Total Drakes                     	345	246	58.4%

Knalgan Alliance vs Undead       	74	19	79.6%
Knalgan Alliance vs Northerners  	29	73	28.4%
Knalgan Alliance vs Loyalists    	93	16	85.3%
Knalgan Alliance vs Knalgan Alliance	67	37	64.4%
Knalgan Alliance vs Drakes       	83	39	68.0%
Knalgan Alliance vs Rebels       	75	18	80.6%
Total Knalgan Alliance           	421	202	67.6%

Loyalists vs Undead              	25	73	25.5%
Loyalists vs Northerners         	24	67	26.4%
Loyalists vs Loyalists           	73	20	78.5%
Loyalists vs Knalgan Alliance    	57	50	53.3%
Loyalists vs Drakes              	61	27	69.3%
Loyalists vs Rebels              	55	48	53.4%
Total Loyalists                  	295	285	50.9%

Northerners vs Undead            	91	5	94.8%
Northerners vs Northerners       	72	26	73.5%
Northerners vs Loyalists         	91	2	97.8%
Northerners vs Knalgan Alliance  	83	2	97.6%
Northerners vs Drakes            	67	12	84.8%
Northerners vs Rebels            	107	6	94.7%
Total Northerners                	511	53	90.6%

Rebels vs Undead                 	81	19	81.0%
Rebels vs Northerners            	34	64	34.7%
Rebels vs Loyalists              	97	6	94.2%
Rebels vs Knalgan Alliance       	92	23	80.0%
Rebels vs Drakes                 	68	32	68.0%
Rebels vs Rebels                 	84	14	85.7%
Total Rebels                     	456	158	74.3%

Undead vs Undead                 	65	29	69.1%
Undead vs Northerners            	39	42	48.1%
Undead vs Loyalists              	108	9	92.3%
Undead vs Knalgan Alliance       	94	9	91.3%
Undead vs Drakes                 	90	19	82.6%
Undead vs Rebels                 	82	24	77.4%
Total Undead                     	478	132	78.4%

Known issues

Bugs

1. Haven't added new Waffles files to Visual C++, so it won't compile under VC++. I need some help with this.
2. ML Recruiter runs fine vs. RCA AI and vs. human, but can't run against itself (ML Recruiter vs. ML Recruiter). This appears to be an issue with Lua in Wesnoth 1.11 and not with ML Recruiter

Current Limitations

1. Only tested on two-player multiplayer games. Doesn't work when there are more than two leaders on the map.
2. Works optimally on the following two-player maps (trained on these)
   1. Weldyn Channel
   2. The Freelands
   3. Den of Onis
   4. Fallenstar Lake
3. Tested on all other two-player maps and runs nearly as well except it crashes on the following:
   • Aethermaw, Hornshark Island, Dark Forecast, Sablestone Delta, Elensefar Courtyard, Silverhead Crossing
4. Currently writes log messages as "print" statements to stdout. I need some advice on this. I've added a new method called "ai_log_message" to core.cpp to allow Lua to write to the "ai/engine/lua" log domain, but would like some advice on whether this is a good idea