Difference between revisions of "Machine Learning Recruiter"
Alarantalara (talk | contribs) (→Win percentages for different AI Pairs: Add result when Arcanclave Citadel is not included) |
Alarantalara (talk | contribs) (→Win percentages for different AI Pairs: Remove result for Ron, since expected release did not occur in time and Ron has since changed) |
||
| Line 84: | Line 84: | ||
|54.1% | |54.1% | ||
|Ron Recruit is the recruiter build into [http://forums.wesnoth.org/viewtopic.php?f=10&t=34976&start=405 AI Demos] | |Ron Recruit is the recruiter build into [http://forums.wesnoth.org/viewtopic.php?f=10&t=34976&start=405 AI Demos] | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
|- | |- | ||
|ML Recruiter 0.3 (Recommended) | |ML Recruiter 0.3 (Recommended) | ||
Revision as of 03:52, 23 November 2012
This page documents the new machine learning recruiter submitted as a patch for Wesnoth 1.11.0. We describe how to run it, discuss experiments showing that the ML Recruiter achieves dramatically better performance than the RCA AI recruiter, describe known issues and suggest a development road map.
Note that the ML Recruiter is a work in progress. We welcome feedback on it. Please discuss it on the thread "Machine Learning Recruiter" at http://forums.wesnoth.org/viewtopic.php?f=10&t=36642.
Contents
Why include ML Recruiter in Wesnoth?
The ML Recruiter makes use of a small subset of the Waffles Machine Learning toolkit adding 13 pairs of .cpp/.h files to Wesnoth. In addition, the neural nets used by ML Recruiter are serialized as .json files, which is a format Wesnoth has not yet contained. So why is this patch worthwhile?
Why the ML Recruiter will be great for Wesnoth
- Performance is great. ML Recruiter defeats RCA AI 66-73% of the time. We suspect that this would also translate into better performance against human opponents because it also performs better against the "ron" recruiter included in AI-Demos, which is a more challenging opponent.
- This superior performance is achieved with comparable "fun factor"
- Variety of units recruited by recommended ML Recruiter is comparable to or better than RCA AI
- Don't need to eliminate RCA AI recruiter. Campaign designers can choose to use one or the other
- ML Recruiter should be easier to customize than RCA AI because
- All core logic is in Lua, which is easier to modify than existing C++
- Performance "out of the box" on known units likely to be strong
- When new recruitable units are introduced by campaign designers, it can be trained by running c. 600 games in two hours. The new model is included as a .json file with the campaign data
- Plug and play architecture of machine learning "features" easily allows minor modifications to mainline recruiter or to campaign-specific recruiters
- Easy way to adjust campaign difficulty: Adjusting ML AI for more/less randomness makes is easier/harder to defeat
- Inclusion of ML Recruiter could lead to greater publicity and more contributors to Wesnoth
- SeattleDad plans to submit this work as a scientific paper to a conference such as Computational Intelligence in Games
- Others might later build on this work by, for instance, trying ML algorithms other than neural nets, adding new features, further generalizing the algorithm, etc.
- The machine learning infrastructure is not specific to recruiting and could be repurposed for, for instance, attack planning, weapon selection, and making "retreat and heal" vs. "attack" decisions
- All of the above is potentially publishable research, so Wesnoth could attract contributions from computer science graduate students
- Note that, having established the basic framework with this patch, future work on machine learning will be much easier
Using ML Recruiter
Applying the patch
- Get the latest version of the patch from https://gna.org/patch/?3479
- Get the Wesnoth 1.11.0 source as per http://wesnoth.org
- Apply the patch as follows:
patch -p1 -i [path to patch file]
- Compile Wesnoth using CMake, SCons, or XCode
Playing against the ML Recruiter
- From the main menu, choose "Multiplayer"
- Choose "Local Game"
- Pick a map and adjust settings as desired. ML Recruiter has been trained with the default setting for village gold and support, but it should work fine on other settings
- Hit Okay
- For one side, Choose Player/Type-->Computer Player and then either ML AI (Recommended) or ML AI (Less Variety, probably stronger)
- For the opponent, either play against it yourself (pick your name), watch it play the default AI (Computer Player-->RCA AI), or watch it play itself (pick ML AI again)
Watching the ML Recruiter play a single game in nogui mode
The following command is convenient for watching the ML Recruiter play a single game in nogui mode, which allows you to quickly and easily see the ML Recruiter's decision-making process. In this example, we would be running the ML AI (Recommended mode) for the Knalgan Alliance, while the default AI would be playing the Rebels. Note that when run this way (with --log-info=ai/testing,ai/ml), a lot of logging messages will be printed to the console which will describe how the ML Recruiter is analyzing its options.
Wesnoth --log-info=ai/testing,ai/ml --nogui --multiplayer --controller 1:ai --controller 2:ai --parm 1:gold:100 --parm 2:gold:100 --parm 1:village_gold:2 --parm 2:village_gold:2 --scenario multiplayer_Weldyn_Channel --parm 1:gold:100 --parm 2:gold:100 --ai-config 1:ai/ais/ml_ai.cfg --ai-config 2:ai/dev/default_ai_with_recruit_log.cfg --side 1:"Knalgan Alliance" --side 2:Rebels
Testing the ML Recruiter in batch mode
Testing in batch mode is easy with the new version of ai_test2.py included in the patch. After applying the ML Recruiter patch, copy utils/ai_test/ai_test2.cfg to the directory in which you want to run the experiment. Then edit the first line of the .cfg file, "path_to_wesnoth_binary" to point to your Wesnoth executable. Then adjust faction1 and faction2 to point to the factions you want to experiment with and point ai_config1 at the ML configuration file you want to try out. Finally, to make everything easier, add the following to your path:
[Wesnoth_Install]/utils/ai_test/
Now you can test Wesnoth in batch as follows:
ai_test2.py ai_test2.cfg
Experimental Results
Win percentages for different AI Pairs
| AI1 | AI2 | Games | Win % for AI1 | Comment |
|---|---|---|---|---|
| ML Recruiter 0.3 (Less Variety) | RCA AI | 1179 | 69.3% | "Less variety/probably stronger version". Wins 73.4% of the time on the maps version 0.2 was trained on |
| ML Recruiter 0.3 (Recommended) | RCA AI | 2363 | 66.6% | "Recommended version". Same version as above, but has more randomness in its choice of units. See documentation on weighted random recruiter |
| ML Recruiter 0.3 | ML Recruiter 0.2 | 1937 | 58.0% | We've made a lot of progress since version 0.2 |
| ML Recruiter 0.3 | Ron Recruiter 0.11.4 | 1186 | 54.1% | Ron Recruit is the recruiter build into AI Demos |
| ML Recruiter 0.3 (Recommended) | ML Recruiter 0.3 (Less variety) | 1186 | 49.6% | Difference is not statistically significant, so we pick the variant with more variety as the "Recommended" version. Note, though, that the "Less variety" version does a bit better against the RCA AI as you can see above. |
| RCA AI | RCA AI | 50% | Any AI against itself will win 50% of the time | |
| RCA AI | Random | 3,000 | 52.7% | Interesting result. You would expect a completely random choice to get beat by a wider margin |
Faction vs. faction win % for ML Recruiter 0.3 vs. RCA AI
These results are for the "Recommended" version
all/data/138 $ analyze_log.py *.log
Overall Stats
AI Wins Win %
default_ai_with_recruit_log 789 33.4%
ml_ai 1574 66.6%
Totals: 2363
Wins Loss Win %
Drakes vs Undead 33 39 45.8%
Drakes vs Northerners 20 39 33.9%
Drakes vs Loyalists 66 11 85.7%
Drakes vs Knalgan Alliance 38 40 48.7%
Drakes vs Drakes 52 19 73.2%
Drakes vs Rebels 46 2 95.8%
Total Drakes 255 150 63.0%
Knalgan Alliance vs Undead 48 19 71.6%
Knalgan Alliance vs Northerners 23 48 32.4%
Knalgan Alliance vs Loyalists 40 14 74.1%
Knalgan Alliance vs Knalgan Alliance 36 23 61.0%
Knalgan Alliance vs Drakes 34 26 56.7%
Knalgan Alliance vs Rebels 46 21 68.7%
Total Knalgan Alliance 227 151 60.1%
Loyalists vs Undead 32 41 43.8%
Loyalists vs Northerners 17 48 26.2%
Loyalists vs Loyalists 58 5 92.1%
Loyalists vs Knalgan Alliance 53 19 73.6%
Loyalists vs Drakes 52 10 83.9%
Loyalists vs Rebels 30 29 50.8%
Total Loyalists 242 152 61.4%
Northerners vs Undead 59 6 90.8%
Northerners vs Northerners 49 21 70.0%
Northerners vs Loyalists 51 9 85.0%
Northerners vs Knalgan Alliance 60 7 89.6%
Northerners vs Drakes 56 14 80.0%
Northerners vs Rebels 55 5 91.7%
Total Northerners 330 62 84.2%
Rebels vs Undead 42 14 75.0%
Rebels vs Rebels 51 15 77.3%
Rebels vs Loyalists 71 8 89.9%
Rebels vs Knalgan Alliance 50 15 76.9%
Rebels vs Drakes 44 22 66.7%
Rebels vs Northerners 21 40 34.4%
Total Rebels 279 114 71.0%
Undead vs Undead 41 37 52.6%
Undead vs Northerners 25 55 31.2%
Undead vs Loyalists 51 18 73.9%
Undead vs Knalgan Alliance 57 6 90.5%
Undead vs Drakes 41 9 82.0%
Undead vs Rebels 26 35 42.6%
Total Undead 241 160 60.1%
Units recruited by Recommended ML Recruiter
Results are for Recommended AI vs. RCA AI for ML Recruiter 0.3
Grand Totals
Drakes Recruitment Count %
Drake Burner 2925 28.3%
Drake Clasher 909 8.8%
Drake Fighter 2159 20.9%
Drake Glider 1156 11.2%
Saurian Augur 2197 21.3%
Saurian Skirmisher 985 9.5%
Total: 10331
Knalgan Alliance Recruitment Count %
Dwarvish Fighter 1535 14.4%
Dwarvish Guardsman 543 5.1%
Dwarvish Thunderer 4171 39.0%
Dwarvish Ulfserker 767 7.2%
Footpad 1573 14.7%
Gryphon Rider 847 7.9%
Poacher 677 6.3%
Thief 576 5.4%
Total: 10689
Loyalists Recruitment Count %
Bowman 1799 15.6%
Cavalryman 551 4.8%
Fencer 345 3.0%
Heavy Infantryman 1109 9.6%
Horseman 930 8.0%
Mage 934 8.1%
Merman Fighter 852 7.4%
Spearman 5040 43.6%
Total: 11560
Northerners Recruitment Count %
Goblin Spearman 181 1.4%
Naga Fighter 504 3.9%
Orcish Archer 2480 19.1%
Orcish Assassin 1842 14.2%
Orcish Grunt 2524 19.5%
Troll Whelp 4833 37.3%
Wolf Rider 608 4.7%
Total: 12972
Rebels Recruitment Count %
Elvish Archer 1833 17.6%
Elvish Fighter 3213 30.8%
Elvish Scout 1086 10.4%
Elvish Shaman 222 2.1%
Mage 435 4.2%
Merman Hunter 768 7.4%
Wose 2865 27.5%
Total: 10422
Undead Recruitment Count %
Dark Adept 2460 20.7%
Ghost 827 7.0%
Ghoul 1474 12.4%
Skeleton 2386 20.1%
Skeleton Archer 3772 31.7%
Vampire Bat 668 5.6%
Walking Corpse 308 2.6%
Total: 11895
Units recruited by Recommended ML Recruiter vs Undead
As a breakdown of the above, it's interesting to look at the different unit blends that ML Recruiter 0.3 selects vs. the Undead as opposed to the overall totals shown above. MLR's RCA AI opponent recruits a unit blend which consists of just the following four units:
RCA AI Recruitment for Undead:
Undead Recruitment Count %
Dark Adept 3163 28.4%
Ghost 2451 22.0%
Skeleton 3574 32.1%
Skeleton Archer 1941 17.4%
Total: 11129
ML Recruiter 0.3 units recruited against the RCA AI Undead. Notice the large increase in the number of units with impact and fire attacks, which would be effective against Skeletons and the decrease in Orcish Assassins and Ghouls, which are ineffective against every Undead unit except Dark Adepts.
Results for enemy faction:Undead
Drakes Recruitment Count %
Drake Burner 666 37.8%
Drake Clasher 37 2.1%
Drake Fighter 478 27.1%
Drake Glider 339 19.2%
Saurian Augur 91 5.2%
Saurian Skirmisher 152 8.6%
Total: 1763
Knalgan Alliance Recruitment Count %
Dwarvish Fighter 352 17.2%
Dwarvish Guardsman 31 1.5%
Dwarvish Thunderer 259 12.7%
Dwarvish Ulfserker 170 8.3%
Footpad 945 46.2%
Gryphon Rider 153 7.5%
Poacher 66 3.2%
Thief 70 3.4%
Total: 2046
Loyalists Recruitment Count %
Bowman 125 6.7%
Cavalryman 45 2.4%
Fencer 65 3.5%
Heavy Infantryman 533 28.6%
Horseman 20 1.1%
Mage 538 28.8%
Merman Fighter 103 5.5%
Spearman 437 23.4%
Total: 1866
Northerners Recruitment Count %
Goblin Spearman 24 1.1%
Naga Fighter 60 2.8%
Orcish Archer 778 36.9%
Orcish Assassin 43 2.0%
Orcish Grunt 202 9.6%
Troll Whelp 952 45.1%
Wolf Rider 51 2.4%
Total: 2110
Rebels Recruitment Count %
Elvish Archer 92 6.9%
Elvish Fighter 265 19.9%
Elvish Scout 83 6.2%
Elvish Shaman 50 3.8%
Mage 176 13.2%
Merman Hunter 41 3.1%
Wose 625 46.9%
Total: 1332
Undead Recruitment Count %
Dark Adept 548 23.1%
Ghost 56 2.4%
Ghoul 153 6.4%
Skeleton 777 32.7%
Skeleton Archer 669 28.1%
Vampire Bat 80 3.4%
Walking Corpse 94 4.0%
Total: 2377
How the ML Recruiter works
When it's deciding what to recruit, the ML Recruiter works by predicting a "metric" which is a measure of how well a given unit will do in the game in the current situation. A good measure of a unit's usefulness is a tricky question and we will discuss three different metrics below, but let's start with the easiest one, which is the sum of the following quantities for each unit:
- Experience points at the end of the game or when the unit is killed
- Number of villages captured by the unit
Note that this metric is blind to other ways a unit can help you (in particular, it doesn't know about poison, healing, and slowing).
This sum, which we'll call the "metric" is then divided by the unit cost to get metric/cost (think of this as goodness per unit of gold). You can see this in the debugging output that the ML Recruiter prints to stderr when run with the flag --log-info=ai/testing,ai/ml:
unit type metric cost wt cost weighted metric Elvish Shaman 8.58 15 15.00 0.57 Elvish Fighter 10.42 14 14.00 0.74 Elvish Scout 8.47 18 18.00 0.47 Wose 17.07 20 20.00 0.85 Mage 10.37 20 20.00 0.52 Elvish Archer 8.46 17 17.00 0.50 Merman Hunter 8.55 15 15.00 0.57
This is from the first turn of a game between the Rebels and the Undead. The ML Recruiter is predicting that if it recruits a Wose now, it will end with 17.07 XP + Village Captures. 17.07/20 = 0.85, which is the highest weighted metric at this time, so it picks a Wose as it's top choice.
How does it know to pick a Wose? It looks at the "features" which describe the current situation. Here's another chart from the same game:
unit type metric cost wt cost weighted metric Elvish Shaman 7.18 15 15.00 0.48 Elvish Fighter 11.82 14 14.00 0.84 Elvish Scout 7.91 18 18.00 0.44 Wose 15.53 20 20.00 0.78 Mage 9.11 20 20.00 0.46 Elvish Archer 9.38 17 17.00 0.55 Merman Hunter 8.36 15 15.00 0.56 Side: 1 Gold: 21 Unit we want: Elvish Fighter PRERECRUIT:, enemy Dark Adept:1 , enemy Deathblade:1 , enemy Ghost:2 , enemy Skeleton:3 , enemy faction:Undead , enemy gold:10 , enemy level3+:0 , enemy total-gold:139 , enemy unit-gold:129 , friendly Elvish Captain:1 , friendly Elvish Fighter:1 , friendly Wose:4 , friendly faction:Rebels , friendly gold:21 , friendly level3+:0 , friendly total-gold:161 , friendly unit-gold:140 , side:1 , terrain-forest:0.082 , terrain-mountain-hill:0.113 , terrain-water-swamp:0.164 , total-gold-ratio:0.537 , turn:4 , village-control-margin:-2 , village-control-ratio:0.417 , village-enemy:7 , village-friendly:5 , village-neutral:4 ,
The "features" that it sees are the values following "PRERECRUIT". The ML AI sees that the enemy faction is the Undead and that they have one Deathblade, two ghosts, three Skeletons. The Rebels currently have 4 Wose, 1 Elvish Fighter, and 1 Elvish Captain. It also sees a number of other features like how much gold it and its opponent have, what percentage of each the map is covered by different terrain, and how many friendly, neutral, and enemy villages there are. In this situation, although it still sees that the Wose is likely to score higher on the XP + village capture metric (15.5 vs. 11.8), this isn't enough to overcome the price differential, so it chooses an Elvish Fighter as it's best choice with a weighted metric of 0.84.
Note that these predictions of 15.5 vs. 11.8 are computed by the neural net based on a model built from what the algorithm has seen has happened in similar situations during training.
Unit Goodness Metrics
We have experimented with three different unit goodness metrics. All of these metrics are designed to have the property that the higher the value of the metric, the better the unit performed in a a given game. Clearly there is a random element here. In some games when playing against a Skeleton-heavy Undead army, an Elvish Archer, which uses mainly a pierce attack, may get lucky and do better than a Wose, which has an impact attack, but on average the metric should show that the Wose performs better.
The three metrics we've looked at are as follows:
Experience Point plus Village Capture
This is the metric used in ML Recruiter 0.2. As noted above, it is the sum of the following quantities:
- Experience points at the end of the game or when the unit is killed
- Number of villages captured by the unit
This metric has the advantage that experience points lead to promotion, which is a very good thing. Also, getting kills should be correlated with how much damage the unit is doing. Adding village captures to experience points is a little flaky, but is intended to give credit to fast units, which are more likely to capture villages.
Victory
This metric gives a unit 1.0 if its side wins and 0 if its side loses. The effect is that the neural net's prediction for each unit can be seen as "what is the probability of victory if I recruit this unit in this situation". This is the most natural of all metrics, but experimentally it hasn't performed as well in terms of leading to actual victories as recruiting based on unit-based metrics. Performance has peaked at around a 59% win ratio for a victory metric vs. around 66 - 73% for the XP+VC metric. We think the problem is that the impact of recruiting a single unit of Type A vs. Type B on the victory probability is very small, so the neural net isn't differentiating among the choices enough.
Gold Yield
As of ML Recruiter 0.3, this is the new default metric. It is the sum of the following quantities, all of which are intended to quantify a unit's usefulness in terms of how much gold benefit it has yielded for the friendly side plus gold damage done to the enemy side. This metric builds off of a suggestion from Sapient.
- Basic Damage Metric: Target unit cost * (Damage inflicted/target max HP). The concept is that you cost your opponent this much gold by destroying this fraction of the unit. Obviously in any given attack, we would calculate this for both the attacker and the defender.
- Village Capture: capturing_unit.variables.ml_gold_yield += wesnoth.game_config.village_income. (Defaults to crediting 2 gold per village capture)
- The idea, again, is that fast units tend to get more captures than slow units and this gives units credit for being fast.
- Poison: Treated the same as Basic Damage Metric by crediting for the amount of damage done in that turn. On the turn in which the unit is cured, the poisoner is credited with Target Unit Cost * (8/target max HP) to reflect the damage that it would have healed if it hadn't been poisoned (obviously, lessened if it has less than 8 HP of damage)
- Slowing: When a unit is on defense and it slows the attacker, the defender gets no special credit because the attacker just unslows at the end of its turn. When you slow a unit as the attacker, the slowing unit gets credit for the Basic Damage Metric accumulated by the slowed unit until it unslows (the slowed unit would otherwise have done twice as much damage, so you get credit for the damage it didn't do)
- Healing: healing_unit.variables.ml_gold_yield += Healed_unit_cost * (healed amount/healed unit max HP)
- Directly analogous to the Basic Damage Metric
- Note that healers also get credit for curing/stopping poison
- Walking Corpse Creation: Credit a unit which gets a kill which creates a unit due to its plague ability with 8 gold (the value of a Walking Corpse). (not implemented)
- Leadership: Credit the leader for the bonus damage inflicted by the unit being led (not implemented)
- Maintenance: Charge units for their share of the maintenance costs, weighted by level. Hence, level 0 units never pay maintenance. Level 2 units pay for twice as much maintenance. (not implemented)
How MLR makes weighted random choices
The recommended recruiter is defined in ai/ais/ml_ai.cfg. It is called "Recommended" in the user interface. Although we are currently measuring it as performing roughly the same or slightly worse than the "Less variety/probably stronger" ML AI (ai/ais/ml_ai_less_random.cfg), we recommend it because it allows the player to see a greater variety of opposing units.
The weighted random printout looks like the following:
Turn 13:
unit type metric cost metric/ weighted %
cost m/c of total
Merman Hunter 1.10 15 0.07 0.0000 0.0%
Wose 1.72 20 0.09 0.0000 0.1%
Elvish Shaman 1.66 15 0.11 0.0000 0.4%
Elvish Archer 2.71 17 0.16 0.0000 4.0%
Elvish Fighter 2.54 14 0.18 0.0000 8.5%
Mage 4.18 20 0.21 0.0001 20.1%
Elvish Scout 4.60 18 0.26 0.0003 66.8%
Random Number chosen was 376
Side: 1 Gold: 27 Unit we want: Elvish Scout
PRERECRUIT:, enemy Dark Adept:2 , enemy Revenant:1 , enemy Skeleton:1 , enemy faction:Undead , enemy gold:8 ,
enemy level3+:0 , enemy total-gold:83 , enemy unit-gold:75 , friendly Elder Wose:1 , friendly Elvish Fighter:2 ,
friendly Elvish Ranger:1 , friendly Mage:1 , friendly Wose:3 , friendly faction:Rebels , friendly gold:27 ,
friendly level3+:0 , friendly total-gold:225 , friendly unit-gold:198 , side:1 , terrain-forest:0.082 ,
terrain-mountain-hill:0.113 , terrain-water-swamp:0.164 , total-gold-ratio:0.731 , turn:13 ,
village-control-margin:0 , village-control-ratio:0.5 , village-enemy:8 , village-friendly:8 , village-neutral:0 ,
This situation occurs towards the end of a game that the Rebels are winning. Note that total-gold-ratio (the ratio between the sum of gold + the value of all units on each side) is 0.731, so it's heavily in the Rebels' favor. The ML AI sees an Elvish Scout as being the best choice in this situation with a Mage in second place. The Elvish Scout is probably favored because the game is likely to be won rapidly and only a fast unit will be able to reach the enemy or reach a village fast enough to add to its "experience points + village capture" metric.
The Weighted Random does the following:
- It takes every metric/cost value and raises it to the sixth power. Why? We want to magnify the differences. In this example 0.26/0.21 = 1.23, but (0.26**6)/(0.21**6) = 3.60.
- We then randomly choose a unit with a probability proportional to this weighted value which, in this case, was an Elvish Scout.
The Less Variety/Probably Stronger AI does the same thing, but raises metric/cost to the 24th power instead of the 6th power. This still allows for some randomness, but weights the selection much more strongly towards the more favored units.
How to train your own ML Recruiter
utils/ai_test/run_model_and_make_new_model.py is an end-to-end script for running a whole bunch of training games of Wesnoth and then training a new model based on the data output by that run. Documentation on this script can be seen by running
run_model_and_make_new_model.py --help
Note that this script assumes that the Waffles machine learning toolkit is installed and that waffles/bin/ is in your path.
Known issues
Bugs
- Haven't added new Waffles files to Visual C++, so it won't compile under VC++. I need some help with this.
- The default for multiplayer games is that units require only 70% of normal experience to promote, however when a game is run from the command line, it always requires normal 100% of experience to promote. Consequently MLR doesn't see units promote as much as they should in training, which would slightly distort its training data. This is a limitation of Wesnoth, not MLR.
Current Limitations
- Only tested on two-player multiplayer games. Doesn't work when there are more than two leaders on the map.
- Works on all two-player maps except for Hornshark Island, Thousand Stings, Caves of the Basilisk, and Dark Forecast
- As noted above in Gold Yield Metric, we account for all special abilities available in the main-line multiplayer scenarios except for plague, leadership, and unit maintenance costs
ML Recruiter development roadmap
- ML Recruiter 0.1: Initial drop
- ML Recruiter 0.1.1: Minor retraining of the model
- ML Recruiter 0.2:
- Logging messages changed from print statements to using lg::log_domain.
- Now have an explicit debug mode by running with --log-debug=ai/ml.
- ML Recruiter can play against itself. Previously could only have ML Recruiter on one side.
- Some work on ML recruiting model (i.e. the core logic). Experimented with different training strategies, but features unchanged from 0.1.
- ML Recruiter 0.3 (10/25/2012) :
- New "gold yield" metric for judging a unit's goodness
- Several new ML features to aid in prediction: alignment, race, time of day, map size, friendly and enemy leader hit point percentage remaining, and nearest enemy unit to friendly leader
- Runs on all 2-player maps except for Hornshark Island, Thousand Stings, Caves of the Basilisk, and Dark Forecast
- Greatly improved ai_test2.py script for running thousands of games to test AI and gather data for the neural net
- New script (run_model_and_make_new_model.py) for running games and building a new neural net based on the data gathered from those games
- Improved performance: Defeats ML Recruiter 0.2 58% of the time
- ML Recruiter 0.4 (11/11/2012):
- Run on all 2-player maps (except Dark Forecast, which has a custom recruiter)
- Refactor code to separate features from predicted values
- Added timeout option to ai_test2.py. Also report time statistics in analyze_log.py
- Improved recruiter for the Ron recruiter. It still underperforms the Ron recruiter on most maps when used with the other Ron CA's, though.
- Move all code into AI-Demos project on GitHub. the ML Recruiter 0.4 patch now consists, essentially, of only the C++ code modifications.
- ML Recruiter 0.5 (planned)
- Run on all mainline multiplayer maps
- Experiment with using as the AI-Demos recruiter
- Add missing special abilities (plague, leadership, and unit upkeep)
- Add 95% confidence intervals to the win ratios in analyze_log.py and add measures of entropy (randomness) to analyze_recruitment.py. Entropy is a good measure of the variety of units that a recruiter is recruiting--for game play, more is better.