FormulaAIandDynamicScripting

These is a *very* rough outline of some ideas I have for the AI improvement. I'm pretty much open to any suggestions but this outlines what I had in mind, giving the AI some adaptive capability and seems to mesh well with the idea of making the AI tunable by scenario designers and for all out combat.

Formula AI

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of rules containing formula AI for some behavior. These rulebases can then be associated with units. Much of the work will be defining what these rulebases will be and implementing them in Formula AI. I would imagine there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be two thirds of the project.

Rulebases

I am thinking of the relationship between rulebases and units similar to object oriented programming in C++. That is a unit inherits predefined rulebases which will dictate that unit's behavior. Unit's can then be associated with very general behaviors, such as 'aggressive', 'defensive', etc. which are overall strategies for the units as well as more specific rulebases which supply unit specific behavior, such as a faction vs faction.

The rulebases will also be parameterized to allow for as much reuse as possible. So for instance, a scenario designer could associate wolf riders with and aggressive rulebase as well as a faction vs faction rulebase parameterized for elves. This would give the designer's wolf riders a behavior biased towards aggressively attacking elves. Recruitment rulebases should also be designated to govern the recruitment process. Futhermore, general strategy rulebases which govern all units can also be applied, so the designer could apply a rulebase tuned towards killing the opponents leader to a global scope.

This should allow scenario designers an easy way to customize the behavior of the AI by simply plugging together appropriate rulebases, perhaps simply in WML, without any more programming knowledge then is already required to design a scenario. Of course, if the scenario designer is comfortable with Formula AI and really wants to fine tune things, the designer can implement his or her own rulebases and use them in conjunction with those provided.

Here is a list of the rulebase categories, hopefully it can change/fill up with specific behaviors as ideas are added. I think coming up with specific rulebases and such will really start rolling after a fair amount of interaction with the community, specifically players and scenario designers.

Rulebase Categories

Recruitment Rulebases

Rulebases to govern the recruitment process, this may just be one parameterized rulebase or several rulebases for different recruiting strategies.

Unit Rulebases

These are rulebases which can be associated to units. Scenario designers can apply some unit rulebases to all units in scenario and then customize specific units on a case by case basis if needed, to say setup faction behaviors between certain units. This could not be just aggresive but also perhaps a healing unit prefers to heal units of it's own faction. Healing units could be associated with a general unit rulebase which tells them to heal wounded units and elves could be even further customized with a heal elves over dwarves behavior.

Global Rulebases

These will be more general rules for which all units would abide (We could create an easy way for certain units to ignore any global rulebase, or to follow another to setup a renegade faction within a computer controlled side, for instance a band of outlaws (thugs, poachers) which are helping the computer controlled side fight but simply for the sake of killing elves and have no care for the overall objective of the AI.) Mostly, the provided rulebases in this category will cover the most common scenarios and deathmatch games, i.e. kill opponents leader, defend for x turns, escort x to hex y, kill at all cost, etc.

Rulebase Flow

So once the global, recruitment and unit rulebases have been defined, the Formula AI in these rulebases forms a sort of script for the AI to follow. The formulaAI will have some form of priority or probability defined to them by the rulebase and this will be used to determine which action in the script to follow for purposes of recruitment and actions. For instance, an 'aggresive', 'faction vs undead' behavior is associated to a white mage being pursued by walking dead. The action to go nuke a few zombies would have a much lower probability of being executed if a global 'protect the leader' behavior is associated to the scenario and the leader is injured and within healing vicinity of the white mage. These probabilities / priorities are where the adaptive nature of the algorithm come into play in the next section.

Dynamic Scripting

I was hoping to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

One method that seems quite suitable is dynamic scripting. This method relies on a manually designed rulebase from which the AI can construct moves. Each rule in the rulebase has an associated weight which influences the probability of a given rule being used in an upcoming move. As a game progresses, the weights are updated automatically according to the success or failure of the rules in the past of the current game (or campaign). This would allow the AI to adapt to a player's tactics and encourage the player to develop diverse strategies for use in campaign mode.

Related Papers

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [1]