The Battle for Wesnoth Wiki - User contributions [en]

FormulaAI Functions

2008-07-03T07:58:12Z

Dhains: /* Gamemap functions */

= Overview =

Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list or a map.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [http://www.wesnoth.org/wiki/FormulaAI#Custom_Functions define your own functions].

=core functions=

=== 'abs' function ===

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

=== 'choose' function ===

<result> = choose( <input list> , [ <string> ,] <formula> )

This function evaluates <formula> for each item in the <input> (which can be a list ro a map). Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

'''Note:''' The implicit input when evaluating a mapping/filtering function's <formula> component will be that specific item under evaluation (in this example one of "my_units"), and it can be explicitly referenced as 'self' when necessary. Optional <string> paremater indicates what word used in <formula> is equivalent to 'self'.

When evaluating the map data type, we can reference to each key by 'key' and each value by 'value'. For example:

choose( { 'elf' -> 10, 'dwarf' -> 20 }, value )

Will return a key-value pair { key->'dwarf', value->20 }

=== 'dir' function ===

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

=== 'filter' function ===

<reult> = filter( <input>, [ <string> ,] <formula> )

This function will run <formula> on each item in the <input> (which can be a list or a map). Will evaluate to a <result> which only contains items the <formula> was true for. Optional <string> indicates what word used in <formula> is equivalent to 'self'. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

=== 'find' function ===

<result> = find( <input>, [ <string>,] <formula> )

This function will run <formula> on each item in the <input> (which can be a list or a map) and will return a first item for which <formula> was true. Optional <string> indicates what word used in <formula> is equivalent to 'self'. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

=== 'head' function ===

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

=== 'if' function ===

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

=== 'keys' function ===

<result list> = keys( <input map> )

Extract key values from a <input map> and return them as a <result list>

keys( { 'Elvish Fighter' -> 50, 'Elvish Archer' -> 60 }

Returns

[ 'Elvish Fighter', 'Elvish Archer' ]

=== 'map' function ===

<result> = map( <input> , [ <string> ,] <formula> )

This function will run <formula> on each item in the <input> (which can be a list or a map), and evaluate to a new <result> list, or a map, which contains the same number of items as in <input>, with the formulas run on each item. Optional <string> indicates what word used in <formula> is equivalent to 'self'. For example:

map( [10,20], self*self)

and

map( [10,20], 'value', value*value)

both will result in [100, 400]. Formula:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

map( { 'elf' -> 10, 'dwarf' -> 20 }, value*2 )

Above will produce { 'elf' -> 20, 'dwarf' -> 40 ). Note that in case of a map data type, 'map' function can modify only the value.

=== 'max' function ===

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

=== 'min' function ===

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

=== 'size' function ===

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

=== 'sort' function ===

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

=== 'sum' function ===

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

=== 'switch' function ===

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

=== 'values' function ===

<result list> = values( <input map> )

Extract values assigned to keys from a <input map> and return them as a <result list>

values( { 'Elvish Fighter' -> 50, 'Elvish Archer' -> 60 }

Returns

[ 50, 60 ]

= AI specific functions =

== Base functions ==

=== 'attack' function ===

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

=== 'fallback' function ===

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

=== 'move' function ===

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

=== 'set_var' function ===

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

== Evaluation ==

=== 'chance to hit' function ===

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

=== 'evaluate_village_possession' function ===

<number> = evaluate_village_possession()

This function returns number between 0 and 100, where 0 means that all of the villages belong to the enemy, and 100 means that all of the villages belong to the AI. Value of 50 represents draw-like situation.

=== 'max_possible_damage' function ===

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

== Gamemap functions ==

=== 'close enemies' function ===

<units list> = close_enemies( <location> , <distance> )

This function gets a list of enemies in the given or smaller distance from the location. For example:

close_enemies(loc(10,10), 5)

gives back a list of enemies in the distance of 5 tiles or less from the tile (10, 10).

=== 'distance_between' function ===

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

=== 'distance_to_nearest_unowned_village' function ===

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

=== 'defense_on' function ===

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

=== 'find_shroud' function ===

<locations list> = find_shroud()

This function will return a list of locations of shrouded hexes.

=== 'is_village' function ===

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

=== 'loc' function ===

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

=== 'unit_at' function ===

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

=== 'nearest_keep' function ===

<keep location> = nearest_keep( <input location> )

Function returns location of nearest keep to the specified <input location>, or null if there is no keep on the map.

=== 'unit_moves' function ===

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

=== 'units_can_reach' function ===

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

== Recruitment ==

=== 'recruit' function ===

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

=== 'unit_chooser' function ===

<action> = unit_chooser( <map with unit names and weights> )

This function takes a map of unit names and attached to these names numbers - representing how should 'ideal army' look like - and as a result it recruits one unit which is most needed, according to the input map. For example:

unit_chooser( { 'Elvish Fighter' -> 60, 'Elvish Archer' -> 30, 'Elvish Shaman' -> 10 } )

We 'inform' the function that our army should consist of 60% fighters, 30% archers and 10% shamans, function checks the current ratio of the units and choses one unit that is most needed at the moment.

[[Category:Development]]
[[Category:Create]]
[[Category:WML Reference]]

GsocAIProjectProgress

2008-05-26T17:59:30Z

Dhains: /* WML Tags */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI & WML ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

==== WML Tags ====

Currently formula can be included with formula=. The AI will need to know what the purpose of some formula is, i.e. we might have special formula for recruitment, candidate moves selection, etc. We also wish to return an integer which would be an evaluation for the proposed move.

The following syntax is proposed to accomplish this:

[register_candidate_move]
name= <name of move>
type= movement/recruitment/etc
action= <action returning formula>
evaluation= <int returning formula>
[/register_candidate_move]

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

[[Category:Summer of Code]]

GsocAIProjectProgress

2008-05-26T17:59:17Z

Dhains: /* WML Tags */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI & WML ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

==== WML Tags ====

Currently formula can be included with formula=. The AI will need to know what the purpose of some formula is, i.e. we might have special formula for recruitment, candidate moves selection, etc. We also wish to return an integer which would be an evaluation for the proposed move.

The following syntax is proposed to accomplish this:

[register_candidate_move]
name= <name of move>
type= movement/recruitment/etc
result= <action returning formula>
evaluation= <int returning formula>
[/register_candidate_move]

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

[[Category:Summer of Code]]

GsocAIProjectProgress

2008-05-26T17:49:18Z

Dhains: /* WML Tags */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI & WML ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

==== WML Tags ====

Currently formula can be included with formula=. The AI will need to know what the purpose of some formula is, i.e. we might have special formula for recruitment, candidate moves selection, etc. We also wish to return an integer which would be an evaluation for the proposed move.

The following syntax is proposed to accomplish this:

[register_candidate_move]
name= <name of move>
type= movement/recruitment/etc
evaluation= <int returning formula>
result= <action returning formula>
[/register_candidate_move]

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-05-19T16:38:21Z

Dhains:

==Project Progess==

[[GsocAIProjectProgress]]

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature. Terrain information is now available to Formula AI using 'map.terrain' from within Formula AI. This allowed implementation of the 'woodchopper' feature requested on the forums.

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365], [http://svn.gna.org/viewcvs/wesnoth?rev=25455&view=rev]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

Various bug fixes: [http://svn.gna.org/viewcvs/wesnoth?rev=25452&view=rev]

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

=== Recruitment Rulebases ===

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

=== Team Rulebases ===

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

=== Unit Rulebases ===

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

GsocAIProjectProgress

2008-04-30T19:01:48Z

Dhains: /* AI API: Formula AI -> WML -> AI */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI & WML ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

==== WML Tags ====

Currently formula can be included with formula=. The AI will need to know what the purpose of some formula is, i.e. we might have special formula for recruitment, candidate moves selection, etc. We also wish to return an integer which would be an evaluation for the proposed move.

The following syntax is proposed to accomplish this:

[register_candidate_move]
type= movement/recruitment/etc
evaluation= <int returning formula>
result= <action returning formula>
[/register_candidate_move]

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-30T19:01:08Z

Dhains: /* WML Tags */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI -> WML -> AI ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

==== WML Tags ====

Currently formula can be included with formula=. The AI will need to know what the purpose of some formula is, i.e. we might have special formula for recruitment, candidate moves selection, etc. We also wish to return an integer which would be an evaluation for the proposed move.

The following syntax is proposed to accomplish this:

[register_candidate_move]
type= movement/recruitment/etc
evaluation= <int returning formula>
result= <action returning formula>
[/register_candidate_move]

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-30T18:48:24Z

Dhains: /* WML Tags */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI -> WML -> AI ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

==== WML Tags ====

Currently formula can be included with formula=. The AI will need to know what the prupose of some formula is, i.e. we might have special formula for recruitment, evaluation functions, candidate move selection, etc. Additional formula inclusion tags will be used to classify these different formula function types

* formula_recruit : this tag will specify a formula to be used for recruitment
* formula_eval : This tag specifies evaluation functions
* move: formula_move : Candidate moves (?), we might not need this, it may be OK to just use the normal formula tag

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-30T18:48:06Z

Dhains: /* AI API: Formula AI -> WML -> AI */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI -> WML -> AI ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers and AI designers.

== WML Tags ==

Currently formula can be included with formula=. The AI will need to know what the prupose of some formula is, i.e. we might have special formula for recruitment, evaluation functions, candidate move selection, etc. Additional formula inclusion tags will be used to classify these different formula function types

* formula_recruit : this tag will specify a formula to be used for recruitment
* formula_eval : This tag specifies evaluation functions
* move: formula_move : Candidate moves (?), we might not need this, it may be OK to just use the normal formula tag

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-29T15:19:53Z

Dhains: /* Basic Idea */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules are sufficient for many situations, but they cannot predict everything. There will always be some area of incomplete knowledge, i.e. we cannot code into the deterministic rules complete knowledge of the game and every situation the AI will ever face. In fact, experienced players may even be able to infer some of the deterministic rules and use this knowledge to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is less complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI -> WML -> AI ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers.

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-29T15:17:57Z

Dhains: /* Basic Idea */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event, the AI had no knowledge of an enemy tactic or unit and a certain unit which the Formula AI ranked very high begins to be massacred due to this incomplete knowledge. The weight associated with that particular unit can now be adjusted to some value < 1 by the AI. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules maybe good for many situations, but they cannot predict everything. In fact, experienced players may infer some of the deterministic rules and use them to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is aless complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI -> WML -> AI ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers.

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-28T18:49:39Z

Dhains: /* Questions */

GSOC AI Project Progress

== Basic Idea ==

The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these moves to 'tweak' them in each scenario.

For example, consider a recruitment example:

Eval functions in FormulaAI will be provided which will associate a 'score' to each unit the AI can recruit. Based on these scores, the AI will recruit particular units. Along with each of these 'scores' is an associated weight, originally set to 1. Now suppose due fog of war or some other event the AI had no knowledge of certain players tactics or units and a certain unit which the Formula AI ranked very high begins to be massacred. Now the weight associated with that particular unit can be adjusted to some value < 1. This will decrease the chance of the AI choosing these units, even though the FormulaAI scores them fairly high.

The point of this learning system is that deterministic rules maybe good for many situations, but they cannot predict everything. In fact, experienced players may infer some of the deterministic rules and use them to their advantage. The learning system will give the AI the ability to tweak it's strategy on the fly to cope with player strategies that the deterministic rules cannot deal with. Recruitment is aless complex then a full fledged AI and the learning system may not be as necessary as it will be in a full fledged AI, but recruitment serves as a useful illustration.

=== Questions ===

Some of the main questions that come up from this idea:

# ) How does the FormulaAI provide the move and evals to the C++ learning system?
# ) How will the C++ learning system evaluate game state and know if it is in a similar situation to one it has been in before (i.e. how does it know which weights to associate to which moves)?
# ) How do we evaluate the success/failure of a move?
# ) How much should we adjust the weights based on this evaluation?

1.) For this I think the formulaAI can provide a list of <move,score> pairs. This provides a simple way for the AI to choose a move, simply pick the move with the highest associated score. This also makes it easy to apply weights, before choosing a particular move, it will apply the weight associated to each move to it's associated score. This leads to the second question, how will the AI know if a certain move is the same as one it has seen previously?

2.) The search space is too complex to keep any kind of state table. I think the best way to associate weights to moves is to 'categorize' the moves. For instance, in the recruitment example the categorization is easy, we simply categorize each recruitment move by the unit that move recruits. The C++ learning system can then keep a list of the categories and the associated weights.

For the more complex examples of non-recruitment moves, I think we can extend this idea of categorization. A very simple high level move categorization is: attack, move, defend. We might be able to further categorize attack moves by tuples (attacking unit, defending unit, terrain). Move moves might be categorized by the tuple (moving unit, terrain), etc.

I think this idea will allow us to associate weights with moves via these categories. There will always be a finite number of these tuples and we need not track the entire state of the space for the purpose of associating weights to moves.

The deterministic eval functions which associate the initial scores to moves are what evaluate the state space in terms of the overall team goals, etc. Even if a < 1 weight is associated with a move, if the eval functions determine the move to be the best move by a large margin, it still has a high chance of being selected. If this move is chosen and executed and is found to be successful, the associated weight will be increased. If the move is deemed to be a failure, the weight will be further decreased. In this way, we won't throw out very strong moves based on one failure. To offset the chance of a very strong move being chosen, it most have been found to be a failure more then once or perhaps an epic failure one time (i.e. loss of a key team member, something fairly catastrophic). This makes the learning system somewhat robust in terms of flukes in the RNG, etc.

3., 4.) This brings up another question(s), how to adjust the weights. We don't want to adjust the weights too fast, or simple RNG fluke could make it so that a strong, viable move is never chosen. We don't want to adjust to small, or the learning system will not be effective over a single scenario. We can call this weight step size alpha, and finding a good alpha value as well as functions to evaluate the success and failure of the moves are additional questions to be answered. In my AI experience, finding a good alpha value is more or less a trial and error process. We can make a close initial guess, but play testing and fine tuning will most likely be necessary to find the optimal alpha.

The success/failure of a move will also be an important component. This might also change depending on the scenario. One possible idea is to also write this in Formula AI, thus allowing it easily to be changed on a scenario by scenario basis. The basic idea is that after a move is taken, the new situation is evaluated and a score is given. We might even be able to use the already defined eval functions to produce this score, but we want to come up with a delta value, i.e. the difference of the score before the move and after the move. Then this delta value will be applied to the weight associated with the move as such:

weight_new = weight_old + delta * alpha

Where alpha is our step size as discussed earlier. In this way, a negative delta will cause the weight to decrease since the move resulted in a poorer game state and vice versa for a good move. Again, it's important to stress the importance of the alpha size and delta values to be calculated carefully. A poor move might be better in the long run (or a very good move might look bad due to a bad RNG round) and we don't want to punish moves too severely for causing a negative delta. On the other hand, we don't want to adjust weights too slowly.

The other nice thing is the learning system can be turned off quite easily, perhaps for an initial experimentation stage, difficulty setting, or if the scenario designer wishes the AI to strictly followed a supplied FormulaAI script, by simply using a flag to determine if the weights should be calculated or not. Since they are initialized to 1, it's a trivial thing to turn this system on and off.

== AI API ==
=== AI API: Formula AI ===

Here we define how Formula AI should be written (Formula AI scripts, Formula AI libraries, etc.)

=== AI API: Formula AI -> WML -> AI ===

Here we define how the formula AI will be included in WML and how the WML will be processed. This layer should be transparent to the AI itself, but will be the one mainly used by scenario designers.

=== AI API: Formula AI -> C++ ===

Here we define how the Formula AI interfaces with the C++ part (i.e. how should moves be supplied, how the moves will be selected and weights applied)

GsocAIProjectProgress

2008-04-28T18:39:17Z

Dhains:

GsocAIProjectProgress

2008-04-28T18:32:20Z

Dhains: /* AI: C++ <-> Formula AI Interface */

GsocAIProjectProgress

2008-04-28T18:31:59Z

Dhains: /* Questions */

GsocAIProjectProgress

2008-04-28T18:28:20Z

Dhains: /* Questions */

GsocAIProjectProgress

2008-04-28T18:20:01Z

Dhains: /* Basic Idea */

GsocAIProjectProgress

2008-04-28T18:19:10Z

Dhains: /* Basic Idea */

GsocAIProjectProgress

2008-04-28T18:17:10Z

Dhains: /* Questions */

GsocAIProjectProgress

2008-04-28T18:16:30Z

Dhains: /* Basic Idea */

GsocAIProjectProgress

2008-04-28T17:53:42Z

Dhains: New page: GSOC AI Project Progress == Basic Idea == The basic idea is to develop libraries of FormulaAI moves based on eval functions. The C++ part will then be able to adjust weights on these m...

FormulaAIandDynamicScripting

2008-04-28T17:43:34Z

Dhains: /* About Me */

[[GsocAIProject]]

==Project Progess==

[[GsocAIProjectProgress]]

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature. Terrain information is now available to Formula AI using 'map.terrain' from within Formula AI. This allowed implementation of the 'woodchopper' feature requested on the forums.

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365], [http://svn.gna.org/viewcvs/wesnoth?rev=25455&view=rev]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

Various bug fixes: [http://svn.gna.org/viewcvs/wesnoth?rev=25452&view=rev]

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

=== Recruitment Rulebases ===

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

=== Team Rulebases ===

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

=== Unit Rulebases ===

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-04-28T17:42:58Z

Dhains: added link to project page

[[GsocAIProject]]

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature. Terrain information is now available to Formula AI using 'map.terrain' from within Formula AI. This allowed implementation of the 'woodchopper' feature requested on the forums.

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365], [http://svn.gna.org/viewcvs/wesnoth?rev=25455&view=rev]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

Various bug fixes: [http://svn.gna.org/viewcvs/wesnoth?rev=25452&view=rev]

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

=== Recruitment Rulebases ===

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

=== Team Rulebases ===

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

=== Unit Rulebases ===

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-04-03T07:02:40Z

Dhains: /* Terrain Knowledge */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature. Terrain information is now available to Formula AI using 'map.terrain' from within Formula AI. This allowed implementation of the 'woodchopper' feature requested on the forums.

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365], [http://svn.gna.org/viewcvs/wesnoth?rev=25455&view=rev]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

Various bug fixes: [http://svn.gna.org/viewcvs/wesnoth?rev=25452&view=rev]

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

=== Recruitment Rulebases ===

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

=== Team Rulebases ===

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

=== Unit Rulebases ===

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-04-03T04:39:03Z

Dhains: /* My Contribs */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365], [http://svn.gna.org/viewcvs/wesnoth?rev=25455&view=rev]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

Various bug fixes: [http://svn.gna.org/viewcvs/wesnoth?rev=25452&view=rev]

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

=== Recruitment Rulebases ===

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

=== Team Rulebases ===

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

=== Unit Rulebases ===

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-31T19:26:45Z

Dhains: /* Rulebases - Formula AI */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

=== Recruitment Rulebases ===

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

=== Team Rulebases ===

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

=== Unit Rulebases ===

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-31T19:22:12Z

Dhains: /* My Contribs */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal by implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-31T19:21:06Z

Dhains: /* My Contribs */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal and implementing necessary features:

==== Terrain Knowledge ====
These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

==== Formula AI Scripts ====

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

==== Candidate moves and Eval functions ====

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-31T19:19:07Z

Dhains: /* My Contribs */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal and implementing necessary features:

These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Exposed terrain, nearest_loc function, woodchopper example - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365]

Documentation can be found on the [[FormulaAI|FormulaAI]] wiki page and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-31T19:18:17Z

Dhains: /* My Contribs */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal and implementing necessary features:

These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Expose terrain, nearest_loc, woodchopper - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365]

Some documentation can be found on the [[FormulaAI|FormulaAI]] and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

Formula AI scripts are now supported. Documentation can be found on the [[FormulaAI|FormulaAI]] page.

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-31T19:14:49Z

Dhains: Added contibs section to show patches/commits

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

=== My Contribs ===

Since I have been active with Wesnoth development for the past few weeks, I have submitted several patches and gained SVN access. Since then, I have been focusing on getting Formula AI to a point where I can begin development of the ideas set forth in this proposal and implementing necessary features:

These checkins exposed map and terrain features to Formula AI, a much needed feature, and implemented the 'woodchopper' feature requested on the forums:

* Expose terrain, nearest_loc, woodchopper - [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365]

Some documentation can be found on the [[FormulaAI|FormulaAI]] and also on the [http://www.wesnoth.org/forum/viewtopic.php?f=10&t=20471&st=0&sk=t&sd=a&start=15 Woodchopper forum thread]

The overall goal of the following checkins was to create and support a new way of writing Formula AI, i.e. formula AI script files to allow reuse and support building of pluggable Formula libraries:

* Created def keyword for custom functions: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=24993]
* Formula Script support: [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25052]
* Cleaned up Formula AI script read method : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25094]
* Fixed Parsing Bug in Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25099]
* Added Comment Support to Formula AI : [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25136]
* Checked in Vim Syntax Highlighting: [http://svn.gna.org/viewcvs/wesnoth?rev=25178&view=rev]

These files are now working...documentation can be found on the [[FormulaAI|FormulaAI]] page.

The following checkins are to support the building of candidate move lists and evaluation functions (work in progress):

* Initial support for candidate move lists [http://svn.gna.org/viewcvs/wesnoth?view=rev&rev=25365],[http://svn.gna.org/viewcvs/wesnoth?rev=25217&view=rev]

Initial Patches: [https://gna.org/patch/?1014], [https://gna.org/patch/?1016].

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAI

2008-03-31T18:23:40Z

Dhains: grammar edit

== Overview ==

The Wesnoth Formula AI is an attempt to develop an AI framework for Wesnoth that allows easy and fun development and modification of AIs for Wesnoth.

Wesnoth already has support for AIs written in Python, but writing AIs in Python has a couple of problems:

* it's still rather difficult, especially for a non-programmer, to develop an AI, even in Python
* Python is insecure; a malicious trojan horse Python script masquerading as an AI could do untold damage

The Wesnoth Formula AI aims to create a fairly simple, pure functional language which allows one to implement an AI. It also aims to allow AIs to be tweaked and modified by people with relatively little technical skill; anyone who can use WML should also be able to use the Formula AI to tweak an AI to make the AI in a scenario behave how they want.

The Wesnoth Formula AI is currently in an experimental stage of development. One can play with it and develop a rudimentary AI. Feedback is appreciated.

To develop an AI using the Formula AI, set ai_algorithm=formula_ai in [side].

== Approach ==

To use the Formula AI, one should put an [ai] tag inside the [side] tag. Inside this [ai] tag, one should specify the 'move' attribute to be a formula which specifies what movement the AI will make. Each time it's the AI's move, this formula will be run, and the move it results in will be executed. Then the formula will be run again; it'll continue to be run until it stops producing a valid move, at which point the AI will end its turn. Alternatively there is a command that the formula may return which will make it end its turn immediately.

A sample AI which does nothing but recruit Wolf Riders is as follows:

[side]
...
ai_algorithm=formula_ai
[ai]
move="recruit('Wolf Rider')"
[/ai]
[/side]

== Formula Command Line ==

To attempt to make it convenient to debug formulas, one can run formulas from within Wesnoth, and see the results. To run a formula, just start game and type 'f'. A command textbox will appear, where you can type a formula, and the results will be printed. For instance, typing

8 + 4

will result in "12" appearing on the screen. You can now use Wesnoth like a calculator. :-)

== Formula Basics ==

* The Formula language supports basic arithmetic operations, such as: +, -, *, /, % and ^. It supports integers but does NOT support decimal or floating point numbers. For example:

4 + 8*7 #evaluates to 60
(4 + 8)*7 #evaluates to 84
8 % 6 #evaluates to 2
5 / 2 #evaluates to 2
3 ^ 2 #evaluates to 9

* It also supports equality, = and !=, and comparison operators, <, >, <=, and >=. 'false' values are 0 (integer) and null. Other values are true. It also supports common operators such as and, or, and not:

2 = 4 #evaluates to 0
2 <= 3 #evaluates to 1
0 != 1 #evaluates to 1
not 4 #evaluates to 0
not 0 #evaluates to 1
(2 < 4) and (3 > 6) #evaluates to 1 and 0 which evaluates to 0
(2 < 4) or (3 > 6) #evaluates to 1 or 0 which evaluates to 1

* Formula language supports also 'dice' operator 'd'. Example usage is:

3d5

Which will give you one of results of rolling three five-sided dice (so random number between 3 and 15).

== Data Types ==

Formula System supports different types of data, which can be stored as a variables and are used in evaluations:

* Numbers: like 0, 1, 2 etc. Floating-point numbers are not supported. 0 is equal to logical 'false', any other number is 'true'.

* Text strings:

'this is a text string'

* Lists: A list is a sequence of values. For example, ai.my_units is a list of unit objects. A list is represented as square brackets, [], surrounding a comma-seperated list. For instance:

[4, 8, 7]

is a list of three numbers, and

[]

is a empty list. Various functions can operate on lists.

== AI Formula Language ==

=== Overview ===

The formula language must be able to access information about the scenario being played to make intelligent decisions. Thus there are various 'inputs' that one may access. A simple example of an input is the turn number one is on, given by the input, 'turn'. Try bringing up the formula command line using 'f' and then type in

turn

The AI will print out the current turn number the game is on.

The 'turn' input is a simple integer. However, some inputs are complex types which contain other inputs, or which may be lists of inputs. For instance, the input 'my_units' contains a list of all the AI's units.

A complex input such as a unit will contain a variety of inputs inside it. If one has a unit input, called 'u' for instance, one can access the 'x' co-ordinate of that unit by using u.loc.x -- u.loc accesses the 'location' object inside the 'unit' object, and the 'location' object contains 'x' and 'y' inputs inside it, which are the x and y co-ordinate of the unit.

=== Built-in functions ===
The formula language contains a large number of built-in functions which allow you to carry out all kinds of complex tasks. Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [[#Custom Functions|define your own functions]].

==== 'attack' function ====

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

==== 'abs' function ====

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

==== 'chance to hit' function ====

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

==== 'choose' function ====

<result> = choose( <input list> , <formula> )

This function evaluates <formula> for each item in the <input list>. Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

==== 'dir' function ====

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

==== 'distance_between' function ====

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

==== 'distance_to_nearest_unowned_village' function ====

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

==== 'defense_on' function ====

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

==== 'evaluate_for_position' function ====

==== 'fallback' function ====

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

==== 'filter' function ====

<reult list> = filter( <input list>, <formula> )

This function will run <formula> on each item in the <input list>. Will evaluate to a <result list> which only contains items the <formula> was true for. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

==== 'find' function ====

<reult> = find( <input list>, <formula> )

This function will run <formula> on each item in the <input list> and will return a first item for which <formula> was true. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

==== 'head' function ====

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

==== 'is_village' function ====

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

==== 'if' function ====

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

==== 'loc' function ====

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

==== 'map' function ====

<result list> = map( <input list> , <formula> )

This function will run <formula> on each item in the <input list>, and evaluate to a new <result list> which contains the same number of items as in <input list>, with the formulas run on each item. For example:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

==== 'max' function ====

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

==== 'max_possible_damage' function ====

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

==== 'min' function ====

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

==== 'move' function ====

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

==== 'nearest_keep' function ====

<keep location> = nearest_keep( <input location> )

Function returns location of nearest keep to the specified <input location>, or null if there is no keep on the map.

==== 'nearest_loc' function ====

<nearest location> = nearest_loc(<location>, <list of locations> )

Returns the closest location in <list of locations> which is closest in distance to <location>.

Example:

nearest_loc(me.loc, map(filter(terrain.map,id = "hills"),loc))

will return the location of the hex with a hills terrain which is closest to current unit.

==== 'recruit' function ====

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

==== 'set_var' function ====

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

==== 'size' function ====

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

==== 'sort' function ====

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

==== 'sum' function ====

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

==== 'switch' function ====

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

==== 'unit_at' function ====

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

==== 'unit_moves' function ====

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

==== 'units_can_reach' function ====

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

=== Custom Functions ===

* You can define your own functions. A function is a formula which takes some inputs as parameters. Suppose we wanted to define a function that puts some value on a unit, we might add the following to the [ai] tag:

[function]
name=value_unit
inputs="unit"
formula="unit.hitpoints + unit.level*4"
[/function]

This has defined a new function which takes a 'unit' as an input, and runs the given calculation over it.

* We can have multiple inputs in our functions, to define them, just create comma-separated inputs list:

inputs="attacker,defender"

This has defined a new function which takes both 'attacker' and 'defender' as an inputs.

* Sometimes, we use one of our inputs really often in our function - to make our life easier we can make its members (inputs) directly accessible from within the formula. This is improved version of function from above:

[function]
name=value_unit
inputs="unit*"
formula="hitpoints + level*4"
[/function]

As you can see, if we define input with a * char at the end, we make it a 'default input' for a formula. Note, that you can define only one default input per function.

* It is important to know difference between formulas defined in custom functions, and formula defined by a 'move=' in a [ai] tag. For example, if we want to get info about leader, we write in formula 'my_leader' - which acces member of the AI. To be able to use 'my_leader' in custom functions we have to add 'ai' as an input for our function:

inputs="ai"

allows us to access leader info by writing 'ai.my_leader', or:

inputs="ai*"

allows us to access leader info by simply writing 'my_leader'

* You can also use 'def' [[#Keywords|keyword]] to define custom functions

=== Comments ===

Comments in Formula AI scripts are enclosed by {}:

{Define opening move}
def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)),
[])

Comments may also be included at the end of a line:

def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)), {capture village}
[]) {do nothing}

and they may also be used inline:

def opening(*ai)
if(turn = 1,
move(loc(11,23) {my_leader}, loc(14,24) {closest village}),
[]) {do nothing}

== Keywords ==

The formula language has some reserved keywords to provide primitive functionality. Currently the following keywords are defined:

* where: This keyword is used to defining statements in formulas. You can define multiple comma-separated statements. Syntax:

<formula> where <comma-separated list of statements>

For example formula:

a + b where a = 2, b = 4

will give as result 6.

* functions: Returns a list of all built-in and custom functions available to the AI

* def: This keyword creates functions using the syntax:

def function_name(arg1, arg2, ....) function_body

For example,

def sum(x,y) x + y

creates a function sum taking two arguments and returns their sum.

== Unit Formulas ==

You san specify a formula for any kind of unit. This is a simple way of doing it:

[unit]
...
formula="move(me.loc, loc(me.loc.x, me.loc.y - 1))"
[/unit]

Custom unit formulas are executed first at the begining of side's turn. Above formula will simply move unit one hex to the north every turn. Note how "me" keyword allows access to unit itself.

You can also define AI unit-specific variables and use them in you formulas:

[unit]
...
formula="if(attack, attack, move(me.loc, choose(unit_moves(me.loc), -distance_between(self, me.vars.guard_loc))))
where attack = choose(filter(attacks, units = [me.loc] and distance_between(me.vars.guard_loc, target) <= me.vars.guard_radius), avg_damage_inflicted)"
[ai_vars]
guard_radius=3
guard_loc="loc(8,5)"
[/ai_vars]
[/unit]

This formula will get location position from variable guard_loc and make sure that unit attacks opponent which is 3 hexes (value specified by guard_radius variable) or less from guard_loc.

Types of variables that are supported:
*number:
variable=3
*text (important: note the ' ' within " "):
name="'I am variable'"
*list:
number_list=[ 1, 2, 3]
*location:
place="loc(X,Y)"

= Formula AI Scripts =

Formula AI scripts are a way to separate Formula AI from WML files. As the complexity of the AI rules grow, so will the formula. Embedding formula in WML can create bloated cfg files and make it cumbersome to reuse AI scripts. By using Formula AI scripts, the formula AI is independent of the WML files and can be included in WML very easily.

== Format ==

Formula AI files end with a .fai extension. They may contain any number of custom functions created by the def keyword and end with a single executable Formula. Each function definition is separated by a semi-colon. As an example, a formula will be converted to a formula script. Here is the formula as it would appear in WML:

formula="move(me.loc, nearest_loc(me.loc, map(filter(map.terrain,id=me.vars.terrain_type),loc)))"

The same formula will now be converted to a formula AI script file. Comments are added and a function is defined to return a list of locations for each hex of a terrain type:

{Return a list of locations with terrain of type terrain_type}
def get_terrain_locs(terrain_type)
map(filter(map.terrain,id=terrain_type),loc);
{Move unit to closest terrain of type terrain_type}
move(me.loc, nearest_loc(me.loc, get_terrain_locs(me.vars.terrain_type)

If this example is saved to the file 'move_to_terrain.fai', to include in WML:

[unit]
x,y=16,7
type="Walking Corpse"
generate_description=yes
formula="{move_to_terrain.fai}"
[ai_vars]
terrain_type='forest'
[/ai_vars]
[/unit]

While this example is fairly trivial, it does illustrate how formula scripts can be used to remove formula from WML. This not only makes the WML easier to read for those unfamiliar with Formula AI, but also allows reusability of predefined behaviors and supports the construction of a repository of Formula AI behaviors which can easily be plugged into WML.

It also creates a more convenient method of building complex behaviors for the Formula AI developer, especially in conjunction with the Formula AI support for developer [[#ctags|tools]].

= Tool Support =

== ctags ==

For some rudimentary Formula AI script support for exuberant ctags, add the following to .ctags (or create the file if it doesn't exist):

--langdef=formulaai
--langmap=formulaai:.fai
--regex-formulaai=/^def[ \t]*([a-zA-Z0-9_]+)/\1/d,definition/

This is especially nice when used with an editor or plugin with ctags support, such as Taglist for Vim.

== Vim ==

===Syntax Highlighting===

Follow these steps to enjoy vim syntax highlighting support for Formula AI scripts.

# Grab the Formula AI vim syntax file, [http://svn.gna.org/viewcvs/*checkout*/wesnoth/trunk/data/tools/vim/formulaai.vim formulaai.vim].
# Copy formulaai.vim to .vim/syntax
# Add the following to .vimrc :
autocmd! BufRead,BufNewFile *.fai setfiletype formulaai

===Taglist Support===

First you will need the very nice [http://www.vim.org/scripts/script.php?script_id=273 taglist plugin]. Follow the link for downloads and install directions if you don't already have it installed.

Next, you'll need Formula AI support for exuberant ctags, follow the instructions in the [[#ctags|ctags]] section.

Once you have all that, simply add the following line to your .vimrc:

let tlist_formulaai_settings = 'formulaai;d:definition'

To test it all out, open a Formula AI script file and enter the command
:Tlist

You should now have some nice highlighting and be able to easily navigate through formula, enjoy!

[[Category:Development]]

[[Category:Development]]

FormulaAI

2008-03-31T18:21:39Z

Dhains: Added section describing Formula AI scripts

== Overview ==

The Wesnoth Formula AI is an attempt to develop an AI framework for Wesnoth that allows easy and fun development and modification of AIs for Wesnoth.

Wesnoth already has support for AIs written in Python, but writing AIs in Python has a couple of problems:

* it's still rather difficult, especially for a non-programmer, to develop an AI, even in Python
* Python is insecure; a malicious trojan horse Python script masquerading as an AI could do untold damage

The Wesnoth Formula AI aims to create a fairly simple, pure functional language which allows one to implement an AI. It also aims to allow AIs to be tweaked and modified by people with relatively little technical skill; anyone who can use WML should also be able to use the Formula AI to tweak an AI to make the AI in a scenario behave how they want.

The Wesnoth Formula AI is currently in an experimental stage of development. One can play with it and develop a rudimentary AI. Feedback is appreciated.

To develop an AI using the Formula AI, set ai_algorithm=formula_ai in [side].

== Approach ==

To use the Formula AI, one should put an [ai] tag inside the [side] tag. Inside this [ai] tag, one should specify the 'move' attribute to be a formula which specifies what movement the AI will make. Each time it's the AI's move, this formula will be run, and the move it results in will be executed. Then the formula will be run again; it'll continue to be run until it stops producing a valid move, at which point the AI will end its turn. Alternatively there is a command that the formula may return which will make it end its turn immediately.

A sample AI which does nothing but recruit Wolf Riders is as follows:

[side]
...
ai_algorithm=formula_ai
[ai]
move="recruit('Wolf Rider')"
[/ai]
[/side]

== Formula Command Line ==

To attempt to make it convenient to debug formulas, one can run formulas from within Wesnoth, and see the results. To run a formula, just start game and type 'f'. A command textbox will appear, where you can type a formula, and the results will be printed. For instance, typing

8 + 4

will result in "12" appearing on the screen. You can now use Wesnoth like a calculator. :-)

== Formula Basics ==

* The Formula language supports basic arithmetic operations, such as: +, -, *, /, % and ^. It supports integers but does NOT support decimal or floating point numbers. For example:

4 + 8*7 #evaluates to 60
(4 + 8)*7 #evaluates to 84
8 % 6 #evaluates to 2
5 / 2 #evaluates to 2
3 ^ 2 #evaluates to 9

* It also supports equality, = and !=, and comparison operators, <, >, <=, and >=. 'false' values are 0 (integer) and null. Other values are true. It also supports common operators such as and, or, and not:

2 = 4 #evaluates to 0
2 <= 3 #evaluates to 1
0 != 1 #evaluates to 1
not 4 #evaluates to 0
not 0 #evaluates to 1
(2 < 4) and (3 > 6) #evaluates to 1 and 0 which evaluates to 0
(2 < 4) or (3 > 6) #evaluates to 1 or 0 which evaluates to 1

* Formula language supports also 'dice' operator 'd'. Example usage is:

3d5

Which will give you one of results of rolling three five-sided dice (so random number between 3 and 15).

== Data Types ==

Formula System supports different types of data, which can be stored as a variables and are used in evaluations:

* Numbers: like 0, 1, 2 etc. Floating-point numbers are not supported. 0 is equal to logical 'false', any other number is 'true'.

* Text strings:

'this is a text string'

* Lists: A list is a sequence of values. For example, ai.my_units is a list of unit objects. A list is represented as square brackets, [], surrounding a comma-seperated list. For instance:

[4, 8, 7]

is a list of three numbers, and

[]

is a empty list. Various functions can operate on lists.

== AI Formula Language ==

=== Overview ===

The formula language must be able to access information about the scenario being played to make intelligent decisions. Thus there are various 'inputs' that one may access. A simple example of an input is the turn number one is on, given by the input, 'turn'. Try bringing up the formula command line using 'f' and then type in

turn

The AI will print out the current turn number the game is on.

The 'turn' input is a simple integer. However, some inputs are complex types which contain other inputs, or which may be lists of inputs. For instance, the input 'my_units' contains a list of all the AI's units.

A complex input such as a unit will contain a variety of inputs inside it. If one has a unit input, called 'u' for instance, one can access the 'x' co-ordinate of that unit by using u.loc.x -- u.loc accesses the 'location' object inside the 'unit' object, and the 'location' object contains 'x' and 'y' inputs inside it, which are the x and y co-ordinate of the unit.

=== Built-in functions ===
The formula language contains a large number of built-in functions which allow you to carry out all kinds of complex tasks. Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [[#Custom Functions|define your own functions]].

==== 'attack' function ====

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

==== 'abs' function ====

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

==== 'chance to hit' function ====

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

==== 'choose' function ====

<result> = choose( <input list> , <formula> )

This function evaluates <formula> for each item in the <input list>. Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

==== 'dir' function ====

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

==== 'distance_between' function ====

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

==== 'distance_to_nearest_unowned_village' function ====

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

==== 'defense_on' function ====

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

==== 'evaluate_for_position' function ====

==== 'fallback' function ====

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

==== 'filter' function ====

<reult list> = filter( <input list>, <formula> )

This function will run <formula> on each item in the <input list>. Will evaluate to a <result list> which only contains items the <formula> was true for. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

==== 'find' function ====

<reult> = find( <input list>, <formula> )

This function will run <formula> on each item in the <input list> and will return a first item for which <formula> was true. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

==== 'head' function ====

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

==== 'is_village' function ====

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

==== 'if' function ====

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

==== 'loc' function ====

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

==== 'map' function ====

<result list> = map( <input list> , <formula> )

This function will run <formula> on each item in the <input list>, and evaluate to a new <result list> which contains the same number of items as in <input list>, with the formulas run on each item. For example:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

==== 'max' function ====

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

==== 'max_possible_damage' function ====

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

==== 'min' function ====

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

==== 'move' function ====

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

==== 'nearest_keep' function ====

<keep location> = nearest_keep( <input location> )

Function returns location of nearest keep to the specified <input location>, or null if there is no keep on the map.

==== 'nearest_loc' function ====

<nearest location> = nearest_loc(<location>, <list of locations> )

Returns the closest location in <list of locations> which is closest in distance to <location>.

Example:

nearest_loc(me.loc, map(filter(terrain.map,id = "hills"),loc))

will return the location of the hex with a hills terrain which is closest to current unit.

==== 'recruit' function ====

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

==== 'set_var' function ====

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

==== 'size' function ====

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

==== 'sort' function ====

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

==== 'sum' function ====

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

==== 'switch' function ====

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

==== 'unit_at' function ====

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

==== 'unit_moves' function ====

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

==== 'units_can_reach' function ====

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

=== Custom Functions ===

* You can define your own functions. A function is a formula which takes some inputs as parameters. Suppose we wanted to define a function that puts some value on a unit, we might add the following to the [ai] tag:

[function]
name=value_unit
inputs="unit"
formula="unit.hitpoints + unit.level*4"
[/function]

This has defined a new function which takes a 'unit' as an input, and runs the given calculation over it.

* We can have multiple inputs in our functions, to define them, just create comma-separated inputs list:

inputs="attacker,defender"

This has defined a new function which takes both 'attacker' and 'defender' as an inputs.

* Sometimes, we use one of our inputs really often in our function - to make our life easier we can make its members (inputs) directly accessible from within the formula. This is improved version of function from above:

[function]
name=value_unit
inputs="unit*"
formula="hitpoints + level*4"
[/function]

As you can see, if we define input with a * char at the end, we make it a 'default input' for a formula. Note, that you can define only one default input per function.

* It is important to know difference between formulas defined in custom functions, and formula defined by a 'move=' in a [ai] tag. For example, if we want to get info about leader, we write in formula 'my_leader' - which acces member of the AI. To be able to use 'my_leader' in custom functions we have to add 'ai' as an input for our function:

inputs="ai"

allows us to access leader info by writing 'ai.my_leader', or:

inputs="ai*"

allows us to access leader info by simply writing 'my_leader'

* You can also use 'def' [[#Keywords|keyword]] to define custom functions

=== Comments ===

Comments in Formula AI scripts are enclosed by {}:

{Define opening move}
def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)),
[])

Comments may also be included at the end of a line:

def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)), {capture village}
[]) {do nothing}

and they may also be used inline:

def opening(*ai)
if(turn = 1,
move(loc(11,23) {my_leader}, loc(14,24) {closest village}),
[]) {do nothing}

== Keywords ==

The formula language has some reserved keywords to provide primitive functionality. Currently the following keywords are defined:

* where: This keyword is used to defining statements in formulas. You can define multiple comma-separated statements. Syntax:

<formula> where <comma-separated list of statements>

For example formula:

a + b where a = 2, b = 4

will give as result 6.

* functions: Returns a list of all built-in and custom functions available to the AI

* def: This keyword creates functions using the syntax:

def function_name(arg1, arg2, ....) function_body

For example,

def sum(x,y) x + y

creates a function sum taking two arguments and returns their sum.

== Unit Formulas ==

You san specify a formula for any kind of unit. This is a simple way of doing it:

[unit]
...
formula="move(me.loc, loc(me.loc.x, me.loc.y - 1))"
[/unit]

Custom unit formulas are executed first at the begining of side's turn. Above formula will simply move unit one hex to the north every turn. Note how "me" keyword allows access to unit itself.

You can also define AI unit-specific variables and use them in you formulas:

[unit]
...
formula="if(attack, attack, move(me.loc, choose(unit_moves(me.loc), -distance_between(self, me.vars.guard_loc))))
where attack = choose(filter(attacks, units = [me.loc] and distance_between(me.vars.guard_loc, target) <= me.vars.guard_radius), avg_damage_inflicted)"
[ai_vars]
guard_radius=3
guard_loc="loc(8,5)"
[/ai_vars]
[/unit]

This formula will get location position from variable guard_loc and make sure that unit attacks opponent which is 3 hexes (value specified by guard_radius variable) or less from guard_loc.

Types of variables that are supported:
*number:
variable=3
*text (important: note the ' ' within " "):
name="'I am variable'"
*list:
number_list=[ 1, 2, 3]
*location:
place="loc(X,Y)"

= Formula AI Scripts =

Formula AI scripts are a way to separate Formula AI from WML files. As the complexity of the AI rules grow, so will the formula. Embedding formula in WML can create bloated cfg files and make it cumbersome to reuse AI scripts. By using Formula AI scripts, the formula AI is independent of the WML files and can be included in WML very easily.

== Format ==

Formula AI files end with .fai extension. They may contain any number of custom functions created by the def keyword and end with a single executable Formula. Each function definition is separated by a semi-colon. As an example, a formula will be converted to a formula script. Here is the formula as it would appear in WML:

formula="move(me.loc, nearest_loc(me.loc, map(filter(map.terrain,id=me.vars.terrain_type),loc)))"

The same formula will now be converted to a formula AI script file. Comments are added and a function is defined to return the location list of a terrain type:

{Return a list of locations with terrain of type terrain_type}
def get_terrain_locs(terrain_type)
map(filter(map.terrain,id=terrain_type),loc);
{Move unit to closest terrain of type terrain_type}
move(me.loc, nearest_loc(me.loc, get_terrain_locs(me.vars.terrain_type)

If this example is saved to the file 'move_to_terrain.fai', to include in WML:

[unit]
x,y=16,7
type="Walking Corpse"
generate_description=yes
formula="{move_to_terrain.fai}"
[ai_vars]
terrain_type='forest'
[/ai_vars]
[/unit]

While this example is fairly trivial, it does illustrate how formula scripts can be used to remove formula from WML. This not only makes the WML easier to read for those unfamiliar with Formula AI, but also allows reusability of predefined behaviors and supports the construction of a repository of Formula AI behaviors which can easily be plugged into WML.

It also creates a more convenient method of building complex behaviors for the Formula AI developer, especially in conjunction with the Formula AI support for developer [[#ctags|tools]].

= Tool Support =

== ctags ==

For some rudimentary Formula AI script support for exuberant ctags, add the following to .ctags (or create the file if it doesn't exist):

--langdef=formulaai
--langmap=formulaai:.fai
--regex-formulaai=/^def[ \t]*([a-zA-Z0-9_]+)/\1/d,definition/

This is especially nice when used with an editor or plugin with ctags support, such as Taglist for Vim.

== Vim ==

===Syntax Highlighting===

Follow these steps to enjoy vim syntax highlighting support for Formula AI scripts.

# Grab the Formula AI vim syntax file, [http://svn.gna.org/viewcvs/*checkout*/wesnoth/trunk/data/tools/vim/formulaai.vim formulaai.vim].
# Copy formulaai.vim to .vim/syntax
# Add the following to .vimrc :
autocmd! BufRead,BufNewFile *.fai setfiletype formulaai

===Taglist Support===

First you will need the very nice [http://www.vim.org/scripts/script.php?script_id=273 taglist plugin]. Follow the link for downloads and install directions if you don't already have it installed.

Next, you'll need Formula AI support for exuberant ctags, follow the instructions in the [[#ctags|ctags]] section.

Once you have all that, simply add the following line to your .vimrc:

let tlist_formulaai_settings = 'formulaai;d:definition'

To test it all out, open a Formula AI script file and enter the command
:Tlist

You should now have some nice highlighting and be able to easily navigate through formula, enjoy!

[[Category:Development]]

[[Category:Development]]

FormulaAI

2008-03-31T17:40:35Z

Dhains: clarified example for nearest_loc function

== Overview ==

The Wesnoth Formula AI is an attempt to develop an AI framework for Wesnoth that allows easy and fun development and modification of AIs for Wesnoth.

Wesnoth already has support for AIs written in Python, but writing AIs in Python has a couple of problems:

* it's still rather difficult, especially for a non-programmer, to develop an AI, even in Python
* Python is insecure; a malicious trojan horse Python script masquerading as an AI could do untold damage

The Wesnoth Formula AI aims to create a fairly simple, pure functional language which allows one to implement an AI. It also aims to allow AIs to be tweaked and modified by people with relatively little technical skill; anyone who can use WML should also be able to use the Formula AI to tweak an AI to make the AI in a scenario behave how they want.

The Wesnoth Formula AI is currently in an experimental stage of development. One can play with it and develop a rudimentary AI. Feedback is appreciated.

To develop an AI using the Formula AI, set ai_algorithm=formula_ai in [side].

== Approach ==

To use the Formula AI, one should put an [ai] tag inside the [side] tag. Inside this [ai] tag, one should specify the 'move' attribute to be a formula which specifies what movement the AI will make. Each time it's the AI's move, this formula will be run, and the move it results in will be executed. Then the formula will be run again; it'll continue to be run until it stops producing a valid move, at which point the AI will end its turn. Alternatively there is a command that the formula may return which will make it end its turn immediately.

A sample AI which does nothing but recruit Wolf Riders is as follows:

[side]
...
ai_algorithm=formula_ai
[ai]
move="recruit('Wolf Rider')"
[/ai]
[/side]

== Formula Command Line ==

To attempt to make it convenient to debug formulas, one can run formulas from within Wesnoth, and see the results. To run a formula, just start game and type 'f'. A command textbox will appear, where you can type a formula, and the results will be printed. For instance, typing

8 + 4

will result in "12" appearing on the screen. You can now use Wesnoth like a calculator. :-)

== Formula Basics ==

* The Formula language supports basic arithmetic operations, such as: +, -, *, /, % and ^. It supports integers but does NOT support decimal or floating point numbers. For example:

4 + 8*7 #evaluates to 60
(4 + 8)*7 #evaluates to 84
8 % 6 #evaluates to 2
5 / 2 #evaluates to 2
3 ^ 2 #evaluates to 9

* It also supports equality, = and !=, and comparison operators, <, >, <=, and >=. 'false' values are 0 (integer) and null. Other values are true. It also supports common operators such as and, or, and not:

2 = 4 #evaluates to 0
2 <= 3 #evaluates to 1
0 != 1 #evaluates to 1
not 4 #evaluates to 0
not 0 #evaluates to 1
(2 < 4) and (3 > 6) #evaluates to 1 and 0 which evaluates to 0
(2 < 4) or (3 > 6) #evaluates to 1 or 0 which evaluates to 1

* Formula language supports also 'dice' operator 'd'. Example usage is:

3d5

Which will give you one of results of rolling three five-sided dice (so random number between 3 and 15).

== Data Types ==

Formula System supports different types of data, which can be stored as a variables and are used in evaluations:

* Numbers: like 0, 1, 2 etc. Floating-point numbers are not supported. 0 is equal to logical 'false', any other number is 'true'.

* Text strings:

'this is a text string'

* Lists: A list is a sequence of values. For example, ai.my_units is a list of unit objects. A list is represented as square brackets, [], surrounding a comma-seperated list. For instance:

[4, 8, 7]

is a list of three numbers, and

[]

is a empty list. Various functions can operate on lists.

== AI Formula Language ==

=== Overview ===

The formula language must be able to access information about the scenario being played to make intelligent decisions. Thus there are various 'inputs' that one may access. A simple example of an input is the turn number one is on, given by the input, 'turn'. Try bringing up the formula command line using 'f' and then type in

turn

The AI will print out the current turn number the game is on.

The 'turn' input is a simple integer. However, some inputs are complex types which contain other inputs, or which may be lists of inputs. For instance, the input 'my_units' contains a list of all the AI's units.

A complex input such as a unit will contain a variety of inputs inside it. If one has a unit input, called 'u' for instance, one can access the 'x' co-ordinate of that unit by using u.loc.x -- u.loc accesses the 'location' object inside the 'unit' object, and the 'location' object contains 'x' and 'y' inputs inside it, which are the x and y co-ordinate of the unit.

=== Built-in functions ===
The formula language contains a large number of built-in functions which allow you to carry out all kinds of complex tasks. Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [[#Custom Functions|define your own functions]].

==== 'attack' function ====

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

==== 'abs' function ====

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

==== 'chance to hit' function ====

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

==== 'choose' function ====

<result> = choose( <input list> , <formula> )

This function evaluates <formula> for each item in the <input list>. Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

==== 'dir' function ====

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

==== 'distance_between' function ====

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

==== 'distance_to_nearest_unowned_village' function ====

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

==== 'defense_on' function ====

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

==== 'evaluate_for_position' function ====

==== 'fallback' function ====

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

==== 'filter' function ====

<reult list> = filter( <input list>, <formula> )

This function will run <formula> on each item in the <input list>. Will evaluate to a <result list> which only contains items the <formula> was true for. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

==== 'find' function ====

<reult> = find( <input list>, <formula> )

This function will run <formula> on each item in the <input list> and will return a first item for which <formula> was true. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

==== 'head' function ====

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

==== 'is_village' function ====

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

==== 'if' function ====

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

==== 'loc' function ====

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

==== 'map' function ====

<result list> = map( <input list> , <formula> )

This function will run <formula> on each item in the <input list>, and evaluate to a new <result list> which contains the same number of items as in <input list>, with the formulas run on each item. For example:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

==== 'max' function ====

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

==== 'max_possible_damage' function ====

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

==== 'min' function ====

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

==== 'move' function ====

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

==== 'nearest_keep' function ====

<keep location> = nearest_keep( <input location> )

Function returns location of nearest keep to the specified <input location>, or null if there is no keep on the map.

==== 'nearest_loc' function ====

<nearest location> = nearest_loc(<location>, <list of locations> )

Returns the closest location in <list of locations> which is closest in distance to <location>.

Example:

nearest_loc(me.loc, map(filter(terrain.map,id = "hills"),loc))

will return the location of the hex with a hills terrain which is closest to current unit.

==== 'recruit' function ====

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

==== 'set_var' function ====

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

==== 'size' function ====

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

==== 'sort' function ====

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

==== 'sum' function ====

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

==== 'switch' function ====

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

==== 'unit_at' function ====

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

==== 'unit_moves' function ====

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

==== 'units_can_reach' function ====

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

=== Custom Functions ===

* You can define your own functions. A function is a formula which takes some inputs as parameters. Suppose we wanted to define a function that puts some value on a unit, we might add the following to the [ai] tag:

[function]
name=value_unit
inputs="unit"
formula="unit.hitpoints + unit.level*4"
[/function]

This has defined a new function which takes a 'unit' as an input, and runs the given calculation over it.

* We can have multiple inputs in our functions, to define them, just create comma-separated inputs list:

inputs="attacker,defender"

This has defined a new function which takes both 'attacker' and 'defender' as an inputs.

* Sometimes, we use one of our inputs really often in our function - to make our life easier we can make its members (inputs) directly accessible from within the formula. This is improved version of function from above:

[function]
name=value_unit
inputs="unit*"
formula="hitpoints + level*4"
[/function]

As you can see, if we define input with a * char at the end, we make it a 'default input' for a formula. Note, that you can define only one default input per function.

* It is important to know difference between formulas defined in custom functions, and formula defined by a 'move=' in a [ai] tag. For example, if we want to get info about leader, we write in formula 'my_leader' - which acces member of the AI. To be able to use 'my_leader' in custom functions we have to add 'ai' as an input for our function:

inputs="ai"

allows us to access leader info by writing 'ai.my_leader', or:

inputs="ai*"

allows us to access leader info by simply writing 'my_leader'

* You can also use 'def' [[#Keywords|keyword]] to define custom functions

=== Comments ===

Comments in Formula AI scripts are enclosed by {}:

{Define opening move}
def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)),
[])

Comments may also be included at the end of a line:

def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)), {capture village}
[]) {do nothing}

and they may also be used inline:

def opening(*ai)
if(turn = 1,
move(loc(11,23) {my_leader}, loc(14,24) {closest village}),
[]) {do nothing}

== Keywords ==

The formula language has some reserved keywords to provide primitive functionality. Currently the following keywords are defined:

* where: This keyword is used to defining statements in formulas. You can define multiple comma-separated statements. Syntax:

<formula> where <comma-separated list of statements>

For example formula:

a + b where a = 2, b = 4

will give as result 6.

* functions: Returns a list of all built-in and custom functions available to the AI

* def: This keyword creates functions using the syntax:

def function_name(arg1, arg2, ....) function_body

For example,

def sum(x,y) x + y

creates a function sum taking two arguments and returns their sum.

== Unit Formulas ==

You san specify a formula for any kind of unit. This is a simple way of doing it:

[unit]
...
formula="move(me.loc, loc(me.loc.x, me.loc.y - 1))"
[/unit]

Custom unit formulas are executed first at the begining of side's turn. Above formula will simply move unit one hex to the north every turn. Note how "me" keyword allows access to unit itself.

You can also define AI unit-specific variables and use them in you formulas:

[unit]
...
formula="if(attack, attack, move(me.loc, choose(unit_moves(me.loc), -distance_between(self, me.vars.guard_loc))))
where attack = choose(filter(attacks, units = [me.loc] and distance_between(me.vars.guard_loc, target) <= me.vars.guard_radius), avg_damage_inflicted)"
[ai_vars]
guard_radius=3
guard_loc="loc(8,5)"
[/ai_vars]
[/unit]

This formula will get location position from variable guard_loc and make sure that unit attacks opponent which is 3 hexes (value specified by guard_radius variable) or less from guard_loc.

Types of variables that are supported:
*number:
variable=3
*text (important: note the ' ' within " "):
name="'I am variable'"
*list:
number_list=[ 1, 2, 3]
*location:
place="loc(X,Y)"

= Tool Support =

== ctags ==

For some rudimentary support for exuberant ctags, add the following to .ctags (or create the file if it doesn't exist):

--langdef=formulaai
--langmap=formulaai:.fai
--regex-formulaai=/^def[ \t]*([a-zA-Z0-9_]+)/\1/d,definition/

This is especially nice when used with an editor or plugin with ctags support, such as Taglist for Vim.

== Vim ==

===Syntax Highlighting===

Follow these steps to enjoy vim syntax highlighting support for Formula AI.

# Grab the Formula AI vim syntax file, [http://svn.gna.org/viewcvs/*checkout*/wesnoth/trunk/data/tools/vim/formulaai.vim formulaai.vim].
# Copy formulaai.vim to .vim/syntax
# Add the following to .vimrc :
autocmd! BufRead,BufNewFile *.fai setfiletype formulaai

===Taglist Support===

First you will need the very nice [http://www.vim.org/scripts/script.php?script_id=273 taglist plugin]. Follow the link for downloads and install directions if you don't already have it installed.

Next, you'll need Formula AI support for exuberant ctags, follow the instructions in the [[#ctags|ctags]] section.

Once you have all that, simply add the following line to your .vimrc:

let tlist_formulaai_settings = 'formulaai;d:definition'

To test it all out, open a Formula AI script file and enter the command
:Tlist

You should now have some nice highlighting and be able to easily navigate through formula, enjoy!

[[Category:Development]]

[[Category:Development]]

FormulaAI

2008-03-31T17:38:40Z

Dhains: Added help for nearest_loc function

== Overview ==

The Wesnoth Formula AI is an attempt to develop an AI framework for Wesnoth that allows easy and fun development and modification of AIs for Wesnoth.

Wesnoth already has support for AIs written in Python, but writing AIs in Python has a couple of problems:

* it's still rather difficult, especially for a non-programmer, to develop an AI, even in Python
* Python is insecure; a malicious trojan horse Python script masquerading as an AI could do untold damage

The Wesnoth Formula AI aims to create a fairly simple, pure functional language which allows one to implement an AI. It also aims to allow AIs to be tweaked and modified by people with relatively little technical skill; anyone who can use WML should also be able to use the Formula AI to tweak an AI to make the AI in a scenario behave how they want.

The Wesnoth Formula AI is currently in an experimental stage of development. One can play with it and develop a rudimentary AI. Feedback is appreciated.

To develop an AI using the Formula AI, set ai_algorithm=formula_ai in [side].

== Approach ==

To use the Formula AI, one should put an [ai] tag inside the [side] tag. Inside this [ai] tag, one should specify the 'move' attribute to be a formula which specifies what movement the AI will make. Each time it's the AI's move, this formula will be run, and the move it results in will be executed. Then the formula will be run again; it'll continue to be run until it stops producing a valid move, at which point the AI will end its turn. Alternatively there is a command that the formula may return which will make it end its turn immediately.

A sample AI which does nothing but recruit Wolf Riders is as follows:

[side]
...
ai_algorithm=formula_ai
[ai]
move="recruit('Wolf Rider')"
[/ai]
[/side]

== Formula Command Line ==

To attempt to make it convenient to debug formulas, one can run formulas from within Wesnoth, and see the results. To run a formula, just start game and type 'f'. A command textbox will appear, where you can type a formula, and the results will be printed. For instance, typing

8 + 4

will result in "12" appearing on the screen. You can now use Wesnoth like a calculator. :-)

== Formula Basics ==

* The Formula language supports basic arithmetic operations, such as: +, -, *, /, % and ^. It supports integers but does NOT support decimal or floating point numbers. For example:

4 + 8*7 #evaluates to 60
(4 + 8)*7 #evaluates to 84
8 % 6 #evaluates to 2
5 / 2 #evaluates to 2
3 ^ 2 #evaluates to 9

* It also supports equality, = and !=, and comparison operators, <, >, <=, and >=. 'false' values are 0 (integer) and null. Other values are true. It also supports common operators such as and, or, and not:

2 = 4 #evaluates to 0
2 <= 3 #evaluates to 1
0 != 1 #evaluates to 1
not 4 #evaluates to 0
not 0 #evaluates to 1
(2 < 4) and (3 > 6) #evaluates to 1 and 0 which evaluates to 0
(2 < 4) or (3 > 6) #evaluates to 1 or 0 which evaluates to 1

* Formula language supports also 'dice' operator 'd'. Example usage is:

3d5

Which will give you one of results of rolling three five-sided dice (so random number between 3 and 15).

== Data Types ==

Formula System supports different types of data, which can be stored as a variables and are used in evaluations:

* Numbers: like 0, 1, 2 etc. Floating-point numbers are not supported. 0 is equal to logical 'false', any other number is 'true'.

* Text strings:

'this is a text string'

* Lists: A list is a sequence of values. For example, ai.my_units is a list of unit objects. A list is represented as square brackets, [], surrounding a comma-seperated list. For instance:

[4, 8, 7]

is a list of three numbers, and

[]

is a empty list. Various functions can operate on lists.

== AI Formula Language ==

=== Overview ===

The formula language must be able to access information about the scenario being played to make intelligent decisions. Thus there are various 'inputs' that one may access. A simple example of an input is the turn number one is on, given by the input, 'turn'. Try bringing up the formula command line using 'f' and then type in

turn

The AI will print out the current turn number the game is on.

The 'turn' input is a simple integer. However, some inputs are complex types which contain other inputs, or which may be lists of inputs. For instance, the input 'my_units' contains a list of all the AI's units.

A complex input such as a unit will contain a variety of inputs inside it. If one has a unit input, called 'u' for instance, one can access the 'x' co-ordinate of that unit by using u.loc.x -- u.loc accesses the 'location' object inside the 'unit' object, and the 'location' object contains 'x' and 'y' inputs inside it, which are the x and y co-ordinate of the unit.

=== Built-in functions ===
The formula language contains a large number of built-in functions which allow you to carry out all kinds of complex tasks. Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [[#Custom Functions|define your own functions]].

==== 'attack' function ====

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

==== 'abs' function ====

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

==== 'chance to hit' function ====

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

==== 'choose' function ====

<result> = choose( <input list> , <formula> )

This function evaluates <formula> for each item in the <input list>. Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

==== 'dir' function ====

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

==== 'distance_between' function ====

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

==== 'distance_to_nearest_unowned_village' function ====

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

==== 'defense_on' function ====

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

==== 'evaluate_for_position' function ====

==== 'fallback' function ====

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

==== 'filter' function ====

<reult list> = filter( <input list>, <formula> )

This function will run <formula> on each item in the <input list>. Will evaluate to a <result list> which only contains items the <formula> was true for. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

==== 'find' function ====

<reult> = find( <input list>, <formula> )

This function will run <formula> on each item in the <input list> and will return a first item for which <formula> was true. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

==== 'head' function ====

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

==== 'is_village' function ====

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

==== 'if' function ====

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

==== 'loc' function ====

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

==== 'map' function ====

<result list> = map( <input list> , <formula> )

This function will run <formula> on each item in the <input list>, and evaluate to a new <result list> which contains the same number of items as in <input list>, with the formulas run on each item. For example:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

==== 'max' function ====

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

==== 'max_possible_damage' function ====

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

==== 'min' function ====

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

==== 'move' function ====

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

==== 'nearest_keep' function ====

<keep location> = nearest_keep( <input location> )

Function returns location of nearest keep to the specified <input location>, or null if there is no keep on the map.

==== 'nearest_loc' function ====

<nearest location> = nearest_loc(<location>, <list of locations> )

Returns the closest location in <list of locations> which is closest in distance to <location>.

Example:

nearest_loc(me.loc, map(filter(terrain.map,id = "hills"),loc))

will return the location of the nearest hex with a hills terrain.

==== 'recruit' function ====

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

==== 'set_var' function ====

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

==== 'size' function ====

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

==== 'sort' function ====

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

==== 'sum' function ====

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

==== 'switch' function ====

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

==== 'unit_at' function ====

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

==== 'unit_moves' function ====

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

==== 'units_can_reach' function ====

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

=== Custom Functions ===

* You can define your own functions. A function is a formula which takes some inputs as parameters. Suppose we wanted to define a function that puts some value on a unit, we might add the following to the [ai] tag:

[function]
name=value_unit
inputs="unit"
formula="unit.hitpoints + unit.level*4"
[/function]

This has defined a new function which takes a 'unit' as an input, and runs the given calculation over it.

* We can have multiple inputs in our functions, to define them, just create comma-separated inputs list:

inputs="attacker,defender"

This has defined a new function which takes both 'attacker' and 'defender' as an inputs.

* Sometimes, we use one of our inputs really often in our function - to make our life easier we can make its members (inputs) directly accessible from within the formula. This is improved version of function from above:

[function]
name=value_unit
inputs="unit*"
formula="hitpoints + level*4"
[/function]

As you can see, if we define input with a * char at the end, we make it a 'default input' for a formula. Note, that you can define only one default input per function.

* It is important to know difference between formulas defined in custom functions, and formula defined by a 'move=' in a [ai] tag. For example, if we want to get info about leader, we write in formula 'my_leader' - which acces member of the AI. To be able to use 'my_leader' in custom functions we have to add 'ai' as an input for our function:

inputs="ai"

allows us to access leader info by writing 'ai.my_leader', or:

inputs="ai*"

allows us to access leader info by simply writing 'my_leader'

* You can also use 'def' [[#Keywords|keyword]] to define custom functions

=== Comments ===

Comments in Formula AI scripts are enclosed by {}:

{Define opening move}
def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)),
[])

Comments may also be included at the end of a line:

def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)), {capture village}
[]) {do nothing}

and they may also be used inline:

def opening(*ai)
if(turn = 1,
move(loc(11,23) {my_leader}, loc(14,24) {closest village}),
[]) {do nothing}

== Keywords ==

The formula language has some reserved keywords to provide primitive functionality. Currently the following keywords are defined:

* where: This keyword is used to defining statements in formulas. You can define multiple comma-separated statements. Syntax:

<formula> where <comma-separated list of statements>

For example formula:

a + b where a = 2, b = 4

will give as result 6.

* functions: Returns a list of all built-in and custom functions available to the AI

* def: This keyword creates functions using the syntax:

def function_name(arg1, arg2, ....) function_body

For example,

def sum(x,y) x + y

creates a function sum taking two arguments and returns their sum.

== Unit Formulas ==

You san specify a formula for any kind of unit. This is a simple way of doing it:

[unit]
...
formula="move(me.loc, loc(me.loc.x, me.loc.y - 1))"
[/unit]

Custom unit formulas are executed first at the begining of side's turn. Above formula will simply move unit one hex to the north every turn. Note how "me" keyword allows access to unit itself.

You can also define AI unit-specific variables and use them in you formulas:

[unit]
...
formula="if(attack, attack, move(me.loc, choose(unit_moves(me.loc), -distance_between(self, me.vars.guard_loc))))
where attack = choose(filter(attacks, units = [me.loc] and distance_between(me.vars.guard_loc, target) <= me.vars.guard_radius), avg_damage_inflicted)"
[ai_vars]
guard_radius=3
guard_loc="loc(8,5)"
[/ai_vars]
[/unit]

This formula will get location position from variable guard_loc and make sure that unit attacks opponent which is 3 hexes (value specified by guard_radius variable) or less from guard_loc.

Types of variables that are supported:
*number:
variable=3
*text (important: note the ' ' within " "):
name="'I am variable'"
*list:
number_list=[ 1, 2, 3]
*location:
place="loc(X,Y)"

= Tool Support =

== ctags ==

For some rudimentary support for exuberant ctags, add the following to .ctags (or create the file if it doesn't exist):

--langdef=formulaai
--langmap=formulaai:.fai
--regex-formulaai=/^def[ \t]*([a-zA-Z0-9_]+)/\1/d,definition/

This is especially nice when used with an editor or plugin with ctags support, such as Taglist for Vim.

== Vim ==

===Syntax Highlighting===

Follow these steps to enjoy vim syntax highlighting support for Formula AI.

# Grab the Formula AI vim syntax file, [http://svn.gna.org/viewcvs/*checkout*/wesnoth/trunk/data/tools/vim/formulaai.vim formulaai.vim].
# Copy formulaai.vim to .vim/syntax
# Add the following to .vimrc :
autocmd! BufRead,BufNewFile *.fai setfiletype formulaai

===Taglist Support===

First you will need the very nice [http://www.vim.org/scripts/script.php?script_id=273 taglist plugin]. Follow the link for downloads and install directions if you don't already have it installed.

Next, you'll need Formula AI support for exuberant ctags, follow the instructions in the [[#ctags|ctags]] section.

Once you have all that, simply add the following line to your .vimrc:

let tlist_formulaai_settings = 'formulaai;d:definition'

To test it all out, open a Formula AI script file and enter the command
:Tlist

You should now have some nice highlighting and be able to easily navigate through formula, enjoy!

[[Category:Development]]

[[Category:Development]]

FormulaAI

2008-03-27T02:20:39Z

Dhains: Added Tools support section and entries for ctags and vim

== Overview ==

The Wesnoth Formula AI is an attempt to develop an AI framework for Wesnoth that allows easy and fun development and modification of AIs for Wesnoth.

Wesnoth already has support for AIs written in Python, but writing AIs in Python has a couple of problems:

* it's still rather difficult, especially for a non-programmer, to develop an AI, even in Python
* Python is insecure; a malicious trojan horse Python script masquerading as an AI could do untold damage

The Wesnoth Formula AI aims to create a fairly simple, pure functional language which allows one to implement an AI. It also aims to allow AIs to be tweaked and modified by people with relatively little technical skill; anyone who can use WML should also be able to use the Formula AI to tweak an AI to make the AI in a scenario behave how they want.

The Wesnoth Formula AI is currently in an experimental stage of development. One can play with it and develop a rudimentary AI. Feedback is appreciated.

To develop an AI using the Formula AI, set ai_algorithm=formula_ai in [side].

== Approach ==

To use the Formula AI, one should put an [ai] tag inside the [side] tag. Inside this [ai] tag, one should specify the 'move' attribute to be a formula which specifies what movement the AI will make. Each time it's the AI's move, this formula will be run, and the move it results in will be executed. Then the formula will be run again; it'll continue to be run until it stops producing a valid move, at which point the AI will end its turn. Alternatively there is a command that the formula may return which will make it end its turn immediately.

A sample AI which does nothing but recruit Wolf Riders is as follows:

[side]
...
ai_algorithm=formula_ai
[ai]
move="recruit('Wolf Rider')"
[/ai]
[/side]

== Formula Command Line ==

To attempt to make it convenient to debug formulas, one can run formulas from within Wesnoth, and see the results. To run a formula, just start game and type 'f'. A command textbox will appear, where you can type a formula, and the results will be printed. For instance, typing

8 + 4

will result in "12" appearing on the screen. You can now use Wesnoth like a calculator. :-)

== Formula Basics ==

* The Formula language supports basic arithmetic operations, such as: +, -, *, /, % and ^. It supports integers but does NOT support decimal or floating point numbers. For example:

4 + 8*7 #evaluates to 60
(4 + 8)*7 #evaluates to 84
8 % 6 #evaluates to 2
5 / 2 #evaluates to 2
3 ^ 2 #evaluates to 9

* It also supports equality, = and !=, and comparison operators, <, >, <=, and >=. 'false' values are 0 (integer) and null. Other values are true. It also supports common operators such as and, or, and not:

2 = 4 #evaluates to 0
2 <= 3 #evaluates to 1
0 != 1 #evaluates to 1
not 4 #evaluates to 0
not 0 #evaluates to 1
(2 < 4) and (3 > 6) #evaluates to 1 and 0 which evaluates to 0
(2 < 4) or (3 > 6) #evaluates to 1 or 0 which evaluates to 1

* Formula language supports also 'dice' operator 'd'. Example usage is:

3d5

Which will give you one of results of rolling three five-sided dice (so random number between 3 and 15).

== Data Types ==

Formula System supports different types of data, which can be stored as a variables and are used in evaluations:

* Numbers: like 0, 1, 2 etc. Floating-point numbers are not supported. 0 is equal to logical 'false', any other number is 'true'.

* Text strings:

'this is a text string'

* Lists: A list is a sequence of values. For example, ai.my_units is a list of unit objects. A list is represented as square brackets, [], surrounding a comma-seperated list. For instance:

[4, 8, 7]

is a list of three numbers, and

[]

is a empty list. Various functions can operate on lists.

== AI Formula Language ==

=== Overview ===

The formula language must be able to access information about the scenario being played to make intelligent decisions. Thus there are various 'inputs' that one may access. A simple example of an input is the turn number one is on, given by the input, 'turn'. Try bringing up the formula command line using 'f' and then type in

turn

The AI will print out the current turn number the game is on.

The 'turn' input is a simple integer. However, some inputs are complex types which contain other inputs, or which may be lists of inputs. For instance, the input 'my_units' contains a list of all the AI's units.

A complex input such as a unit will contain a variety of inputs inside it. If one has a unit input, called 'u' for instance, one can access the 'x' co-ordinate of that unit by using u.loc.x -- u.loc accesses the 'location' object inside the 'unit' object, and the 'location' object contains 'x' and 'y' inputs inside it, which are the x and y co-ordinate of the unit.

=== Built-in functions ===
The formula language contains a large number of built-in functions which allow you to carry out all kinds of complex tasks. Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [[#Custom Functions|define your own functions]].

==== 'attack' function ====

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

==== 'abs' function ====

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

==== 'chance to hit' function ====

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

==== 'choose' function ====

<result> = choose( <input list> , <formula> )

This function evaluates <formula> for each item in the <input list>. Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

==== 'dir' function ====

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

==== 'distance_between' function ====

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

==== 'distance_to_nearest_unowned_village' function ====

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

==== 'defense_on' function ====

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

==== 'evaluate_for_position' function ====

==== 'fallback' function ====

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

==== 'filter' function ====

<reult list> = filter( <input list>, <formula> )

This function will run <formula> on each item in the <input list>. Will evaluate to a <result list> which only contains items the <formula> was true for. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

==== 'find' function ====

<reult> = find( <input list>, <formula> )

This function will run <formula> on each item in the <input list> and will return a first item for which <formula> was true. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

==== 'head' function ====

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

==== 'is_village' function ====

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

==== 'if' function ====

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

==== 'loc' function ====

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

==== 'map' function ====

<result list> = map( <input list> , <formula> )

This function will run <formula> on each item in the <input list>, and evaluate to a new <result list> which contains the same number of items as in <input list>, with the formulas run on each item. For example:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

==== 'max' function ====

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

==== 'max_possible_damage' function ====

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

==== 'min' function ====

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

==== 'move' function ====

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

==== 'recruit' function ====

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

==== 'set_var' function ====

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

==== 'size' function ====

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

==== 'sort' function ====

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

==== 'sum' function ====

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

==== 'switch' function ====

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

==== 'unit_at' function ====

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

==== 'unit_moves' function ====

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

==== 'units_can_reach' function ====

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

=== Custom Functions ===

* You can define your own functions. A function is a formula which takes some inputs as parameters. Suppose we wanted to define a function that puts some value on a unit, we might add the following to the [ai] tag:

[function]
name=value_unit
inputs="unit"
formula="unit.hitpoints + unit.level*4"
[/function]

This has defined a new function which takes a 'unit' as an input, and runs the given calculation over it.

* We can have multiple inputs in our functions, to define them, just create comma-separated inputs list:

inputs="attacker,defender"

This has defined a new function which takes both 'attacker' and 'defender' as an inputs.

* Sometimes, we use one of our inputs really often in our function - to make our life easier we can make its members (inputs) directly accessible from within the formula. This is improved version of function from above:

[function]
name=value_unit
inputs="unit*"
formula="hitpoints + level*4"
[/function]

As you can see, if we define input with a * char at the end, we make it a 'default input' for a formula. Note, that you can define only one default input per function.

* It is important to know difference between formulas defined in custom functions, and formula defined by a 'move=' in a [ai] tag. For example, if we want to get info about leader, we write in formula 'my_leader' - which acces member of the AI. To be able to use 'my_leader' in custom functions we have to add 'ai' as an input for our function:

inputs="ai"

allows us to access leader info by writing 'ai.my_leader', or:

inputs="ai*"

allows us to access leader info by simply writing 'my_leader'

* You can also use 'def' [[#Keywords|keyword]] to define custom functions

=== Comments ===

Comments in Formula AI scripts are enclosed by {}:

{Define opening move}
def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)),
[])

Comments may also be included at the end of a line:

def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)), {capture village}
[]) {do nothing}

and they may also be used inline:

def opening(*ai)
if(turn = 1,
move(loc(11,23) {my_leader}, loc(14,24) {closest village}),
[]) {do nothing}

== Keywords ==

The formula language has some reserved keywords to provide primitive functionality. Currently the following keywords are defined:

* where: This keyword is used to defining statements in formulas. You can define multiple comma-separated statements. Syntax:

<formula> where <comma-separated list of statements>

For example formula:

a + b where a = 2, b = 4

will give as result 6.

* functions: Returns a list of all built-in and custom functions available to the AI

* def: This keyword creates functions using the syntax:

def function_name(arg1, arg2, ....) function_body

For example,

def sum(x,y) x + y

creates a function sum taking two arguments and returns their sum.

== Unit Formulas ==

You san specify a formula for any kind of unit. This is a simple way of doing it:

[unit]
...
formula="move(me.loc, loc(me.loc.x, me.loc.y - 1))"
[/unit]

Custom unit formulas are executed first at the begining of side's turn. Above formula will simply move unit one hex to the north every turn. Note how "me" keyword allows access to unit itself.

You can also define AI unit-specific variables and use them in you formulas:

[unit]
...
formula="if(attack, attack, move(me.loc, choose(unit_moves(me.loc), -distance_between(self, me.vars.guard_loc))))
where attack = choose(filter(attacks, units = [me.loc] and distance_between(me.vars.guard_loc, target) <= me.vars.guard_radius), avg_damage_inflicted)"
[ai_vars]
guard_radius=3
guard_loc="loc(8,5)"
[/ai_vars]
[/unit]

This formula will get location position from variable guard_loc and make sure that unit attacks opponent which is 3 hexes (value specified by guard_radius variable) or less from guard_loc.

Types of variables that are supported:
*number:
variable=3
*text (important: note the ' ' within " "):
name="'I am variable'"
*list:
number_list=[ 1, 2, 3]
*location:
place="loc(X,Y)"

= Tool Support =

== ctags ==

For some rudimentary support for exuberant ctags, add the following to .ctags (or create the file if it doesn't exist):

--langdef=formulaai
--langmap=formulaai:.fai
--regex-formulaai=/^def[ \t]*([a-zA-Z0-9_]+)/\1/d,definition/

This is especially nice when used with an editor or plugin with ctags support, such as Taglist for Vim.

== Vim ==

===Syntax Highlighting===

Follow these steps to enjoy vim syntax highlighting support for Formula AI.

# Grab the Formula AI vim syntax file, [http://svn.gna.org/viewcvs/*checkout*/wesnoth/trunk/data/tools/vim/formulaai.vim formulaai.vim].
# Copy formulaai.vim to .vim/syntax
# Add the following to .vimrc :
autocmd! BufRead,BufNewFile *.fai setfiletype formulaai

===Taglist Support===

First you will need the very nice [http://www.vim.org/scripts/script.php?script_id=273 taglist plugin]. Follow the link for downloads and install directions if you don't already have it installed.

Next, you'll need Formula AI support for exuberant ctags, follow the instructions in the [[#ctags|ctags]] section.

Once you have all that, simply add the following line to your .vimrc:

let tlist_formulaai_settings = 'formulaai;d:definition'

To test it all out, open a Formula AI script file and enter the command
:Tlist

You should now have some nice highlighting and be able to easily navigate through formula, enjoy!

[[Category:Development]]

[[Category:Development]]

FormulaAI

2008-03-26T05:42:13Z

Dhains: Added documentation on comments

== Overview ==

The Wesnoth Formula AI is an attempt to develop an AI framework for Wesnoth that allows easy and fun development and modification of AIs for Wesnoth.

Wesnoth already has support for AIs written in Python, but writing AIs in Python has a couple of problems:

* it's still rather difficult, especially for a non-programmer, to develop an AI, even in Python
* Python is insecure; a malicious trojan horse Python script masquerading as an AI could do untold damage

The Wesnoth Formula AI aims to create a fairly simple, pure functional language which allows one to implement an AI. It also aims to allow AIs to be tweaked and modified by people with relatively little technical skill; anyone who can use WML should also be able to use the Formula AI to tweak an AI to make the AI in a scenario behave how they want.

The Wesnoth Formula AI is currently in an experimental stage of development. One can play with it and develop a rudimentary AI. Feedback is appreciated.

To develop an AI using the Formula AI, set ai_algorithm=formula_ai in [side].

== Approach ==

To use the Formula AI, one should put an [ai] tag inside the [side] tag. Inside this [ai] tag, one should specify the 'move' attribute to be a formula which specifies what movement the AI will make. Each time it's the AI's move, this formula will be run, and the move it results in will be executed. Then the formula will be run again; it'll continue to be run until it stops producing a valid move, at which point the AI will end its turn. Alternatively there is a command that the formula may return which will make it end its turn immediately.

A sample AI which does nothing but recruit Wolf Riders is as follows:

[side]
...
ai_algorithm=formula_ai
[ai]
move="recruit('Wolf Rider')"
[/ai]
[/side]

== Formula Command Line ==

To attempt to make it convenient to debug formulas, one can run formulas from within Wesnoth, and see the results. To run a formula, just start game and type 'f'. A command textbox will appear, where you can type a formula, and the results will be printed. For instance, typing

8 + 4

will result in "12" appearing on the screen. You can now use Wesnoth like a calculator. :-)

== Formula Basics ==

* The Formula language supports basic arithmetic operations, such as: +, -, *, /, % and ^. It supports integers but does NOT support decimal or floating point numbers. For example:

4 + 8*7 #evaluates to 60
(4 + 8)*7 #evaluates to 84
8 % 6 #evaluates to 2
5 / 2 #evaluates to 2
3 ^ 2 #evaluates to 9

* It also supports equality, = and !=, and comparison operators, <, >, <=, and >=. 'false' values are 0 (integer) and null. Other values are true. It also supports common operators such as and, or, and not:

2 = 4 #evaluates to 0
2 <= 3 #evaluates to 1
0 != 1 #evaluates to 1
not 4 #evaluates to 0
not 0 #evaluates to 1
(2 < 4) and (3 > 6) #evaluates to 1 and 0 which evaluates to 0
(2 < 4) or (3 > 6) #evaluates to 1 or 0 which evaluates to 1

* Formula language supports also 'dice' operator 'd'. Example usage is:

3d5

Which will give you one of results of rolling three five-sided dice (so random number between 3 and 15).

== Data Types ==

Formula System supports different types of data, which can be stored as a variables and are used in evaluations:

* Numbers: like 0, 1, 2 etc. Floating-point numbers are not supported. 0 is equal to logical 'false', any other number is 'true'.

* Text strings:

'this is a text string'

* Lists: A list is a sequence of values. For example, ai.my_units is a list of unit objects. A list is represented as square brackets, [], surrounding a comma-seperated list. For instance:

[4, 8, 7]

is a list of three numbers, and

[]

is a empty list. Various functions can operate on lists.

== AI Formula Language ==

=== Overview ===

The formula language must be able to access information about the scenario being played to make intelligent decisions. Thus there are various 'inputs' that one may access. A simple example of an input is the turn number one is on, given by the input, 'turn'. Try bringing up the formula command line using 'f' and then type in

turn

The AI will print out the current turn number the game is on.

The 'turn' input is a simple integer. However, some inputs are complex types which contain other inputs, or which may be lists of inputs. For instance, the input 'my_units' contains a list of all the AI's units.

A complex input such as a unit will contain a variety of inputs inside it. If one has a unit input, called 'u' for instance, one can access the 'x' co-ordinate of that unit by using u.loc.x -- u.loc accesses the 'location' object inside the 'unit' object, and the 'location' object contains 'x' and 'y' inputs inside it, which are the x and y co-ordinate of the unit.

=== Built-in functions ===
The formula language contains a large number of built-in functions which allow you to carry out all kinds of complex tasks. Syntax used to explain functions usage in this document is:

<result> = <function name>( <comma-separated list of parameters> [, <comma-separated list of optional parameters] )

Function may return <result> as:
* <variable> - any of the supported variable types
* <boolean> - false ( 0 or null ) or true ( 1 )
* <unit> - unit
* <location> - place on a gamemap
* <action> - object, which, if later passed to 'move= ' as the result of formula evaluation, make the AI perform a desired action.
* <result> - any of the above

Also function may return only single argument, or be able to return a whole list.

There are a wide variety of functions which can be used to accomplish many different tasks. You can also [[#Custom Functions|define your own functions]].

==== 'attack' function ====

<action> = attack( <attacker's position>, <destination>, <attack location> [, <weapon> ] )

The first three parameters are locations. At the begining, unit which is standing at <attacker's position> is moved to <destination> place. Then, from that place unit is attacking unit which stands in place marked by <attack location>. Fourth optional parameter is number, and indicates which weapon attacker should use - if not specified, best possible weapon is chosed automatically.

==== 'abs' function ====

<number> = abs( <input number> )

Function returns absolute value of an <input number>, for example

abs( -5 )

will return 5.

==== 'chance to hit' function ====

<number> = chance_to_hit( <unit> , <location> )

This function returns how possible ( in % ) it is to hit given <unit> in a specific <location>. For example:

chance_to_hit( my_leader , my_leader.loc )

shows how easy it is to hit your leader has in a place he is currently standing on.

==== 'choose' function ====

<result> = choose( <input list> , <formula> )

This function evaluates <formula> for each item in the <input list>. Will evaluate to the one item which <formula> gave the highest value. For example:

choose(my_units, level)

gives back the unit with the highest level.

==== 'dir' function ====

<list of names> = dir ( <input object> )

This function return list with all names of <input object's> members. For example:

dir( my_leader )

will result in output:

[ 'x', 'y', 'loc', 'id', 'leader', 'hitpoints', 'max_hitpoints', 'experience', 'max_experience', 'level',
'total_movement', 'movement_left', 'side', 'is_enemy', 'is_mine']

This command is useful in formula command line, to get information about members of different type of data. To get list of members of the ai, type:

dir( self )

==== 'distance_between' function ====

<number> = distance_between( <location A> , <location B> )

This function returns distance (in hexes) between <location A> and <location B>. For example:

distance_between( loc( 1, 1) , loc( 3, 3) )

will return 3.

==== 'distance_to_nearest_unowned_village' function ====

<number> = distance_to_nearest_unowned_village( <location A> )

This function returns distance (in hexes) between <location A> and nearest unowned village.

==== 'defense_on' function ====

<number> = defense_on( <unit> , <location> )

This function returns defense rate of given <unit> in a specific <location>. For example:

defense_on( my_leader , my_leader.loc )

shows how good defense your leader has in a place he is currently standing on.

==== 'evaluate_for_position' function ====

==== 'fallback' function ====

<action> = fallback( <name> )

This function allows to chose different AI which will take control over side untill the end of current turn. For example:

fallback( 'default' )

will transfer control to the default C++ AI.

==== 'filter' function ====

<reult list> = filter( <input list>, <formula> )

This function will run <formula> on each item in the <input list>. Will evaluate to a <result list> which only contains items the <formula> was true for. For example:

filter(my_units, hitpoints < max_hitpoints)

will return all of your units which have less than maximum hitpoints. For instance this could be used if looking for candidates for healing.

==== 'find' function ====

<reult> = find( <input list>, <formula> )

This function will run <formula> on each item in the <input list> and will return a first item for which <formula> was true. For example:

filter(units, id = 'Elvish Archer' )

will return first unit with id equal to 'Elvish Archer'.

==== 'head' function ====

<variable> = head( <list of variables> )

Head returns first item from the <list of variables>, for example

head( [ 5, 7, 9] ) #returns 5
head( [ 'Orc', 'Human' ] ) #returns 'Orc'

==== 'is_village' function ====

<boolean> = is_village( <map or ai.map> , <location> ) #1

<boolean> = is_village( <map or ai.map> , <coordinate x> , <coordinate y> ) #2

The first argument is always a 'map' - member of the ai which provides information about the gamemap.

In #1 usage, we put in as a second argument location. In #2, second and third arguments are numbers: coordniates of the certain hex on a map. Function checks if that place is a village, and returns either 1 (yes, it is village) or 0 (no, it isn't village). Example of usage:

is_village( map , loc( 2, 3) )

is_village( map , 2, 3)

Both check, if hex with coordinates 2,3 is a village.

Remember, when using is_village in custom function, you either have to access map by writing 'ai.map', or specify ai as a 'default input'.

==== 'if' function ====

<result> = if( <condition> , <if true> , <otherwise> )

If the <condition> parameter is true, the function will evaluate to being equal to its second input ( <if true> ), otherwise it will evaluate to being equal to its third input ( <otherwise> ).
For instance, an AI that recruits Wolf Riders on the first turn, and Orcish Grunts thereafter might look like this:

move="if(turn = 1, recruit('Wolf Rider'), recruit('Orcish Grunt'))"

==== 'loc' function ====

<location> = loc( <X number>, <Y number> )

Function will return a location (pair of numbers) from two given input arguments.

==== 'map' function ====

<result list> = map( <input list> , <formula> )

This function will run <formula> on each item in the <input list>, and evaluate to a new <result list> which contains the same number of items as in <input list>, with the formulas run on each item. For example:

map(my_units, hitpoints)

will give a list back with the number of hitpoints each unit has. This is more useful in conjunction with other functions.

==== 'max' function ====

<number> = max( <list of numbers> )

Function will return maximal number from a list,

max( [ 2, 8, -10, 3] )

will return 8.

==== 'max_possible_damage' function ====

<number> = max_possible_damage( <attacking unit> , <defending unit> )

Function returns highest possible damage that <attacking unit> can inflict to <defending unit>.

==== 'min' function ====

<number> = min( <list of numbers> )

Function will return minimal number from a list,

min( [ 3, 7, -2, 6] )

will return -2.

==== 'move' function ====

<action> = move( <source> , <destination> )

For example unit formula like:

move(me.loc, loc(me.loc.x, me.loc.y - 1) )

will make unit move one hex north.

==== 'recruit' function ====

<action> = recruit( <unit name> [, <location> ] )

This function results in recruting a unit specifed by <unit name> at first free castle hex, or at given <location>. Function:

recruit('Footpad', loc(3,3) )

will result in recruting Footpad at castle hex with coordinates 3,3.

==== 'set_var' function ====

<action> = set_var( <key> , <value> )

This action sets new variable, for example:

set_var( 'Number one' , 1 )

Will create variable with name 'Number one' and assign 1 to it.

==== 'size' function ====

<number> = size( <list of variables> )

This function returns how many variables are stored in a list:

size( [ 5, 7, 9] ) #return 3
size( [ 'Archer', 'Fighter' ] ) #return 2

==== 'sort' function ====

<result list> = sort( <input list> , <formula> )

This function evaluates to a <result list> sorted according to the comparison <formula> for each item 'a' and its successor 'b'. For instance, sorting units according to hitpoints would be done by:

sort( my_units, a.hitpoints > b.hitpoints )

==== 'sum' function ====

<number> = sum( <list of numbers> )

This function evaluates to the sum of the items in the <list of numbers>. For example

sum( [ 2, 5, 8] )

returns 15, and:

sum( map( my_units, max_hitpoints - hitpoints ) )

finds the total damage your units have taken.

==== 'switch' function ====

<result> = switch( <variable>, <value 1>, <outcome 1>, ... , <value N>, <outcome N> [, <default outcome> ] >

Switch funtion takes variable, and checks if it is equal to any of the specified <values>. If matching value is found, <outcome> assigned to it is returned, if not, then function returns either <default outcome> (if specified) or null.

==== 'unit_at' function ====

<unit> = unit_at( <location> )

This function takes only one argument - location, and returns unit if there is one standing in that location, or null otherwise. Example of usage:

unit_at( loc( 4, 4) )

==== 'unit_moves' function ====

<locations list> = unit_moves( <unit location> )

Function returns list of all possible locations which unit standing at <unit location> can reach. If unit can't move, or there is no unit standing at given location, empty list is returned.

==== 'units_can_reach' function ====

<units list> = units_can_reach( <possible moves list>, <location> )

Function takes as an input list of possible moves ( ai.my_moves for units that belong to AI, or ai.enemy_moves for units that belong to the opponent ) and checks which units from that list can reach <location>.

=== Custom Functions ===

* You can define your own functions. A function is a formula which takes some inputs as parameters. Suppose we wanted to define a function that puts some value on a unit, we might add the following to the [ai] tag:

[function]
name=value_unit
inputs="unit"
formula="unit.hitpoints + unit.level*4"
[/function]

This has defined a new function which takes a 'unit' as an input, and runs the given calculation over it.

* We can have multiple inputs in our functions, to define them, just create comma-separated inputs list:

inputs="attacker,defender"

This has defined a new function which takes both 'attacker' and 'defender' as an inputs.

* Sometimes, we use one of our inputs really often in our function - to make our life easier we can make its members (inputs) directly accessible from within the formula. This is improved version of function from above:

[function]
name=value_unit
inputs="unit*"
formula="hitpoints + level*4"
[/function]

As you can see, if we define input with a * char at the end, we make it a 'default input' for a formula. Note, that you can define only one default input per function.

* It is important to know difference between formulas defined in custom functions, and formula defined by a 'move=' in a [ai] tag. For example, if we want to get info about leader, we write in formula 'my_leader' - which acces member of the AI. To be able to use 'my_leader' in custom functions we have to add 'ai' as an input for our function:

inputs="ai"

allows us to access leader info by writing 'ai.my_leader', or:

inputs="ai*"

allows us to access leader info by simply writing 'my_leader'

* You can also use 'def' [[#Keywords|keyword]] to define custom functions

=== Comments ===

Comments in Formula AI scripts are enclosed by {}:

{Define opening move}
def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)),
[])

Comments may also be included at the end of a line:

def opening(*ai)
if(turn = 1,
move(loc(11,23), loc(14,22)), {capture village}
[]) {do nothing}

and they may also be used inline:

def opening(*ai)
if(turn = 1,
move(loc(11,23) {my_leader}, loc(14,24) {closest village}),
[]) {do nothing}

== Keywords ==

The formula language has some reserved keywords to provide primitive functionality. Currently the following keywords are defined:

* where: This keyword is used to defining statements in formulas. You can define multiple comma-separated statements. Syntax:

<formula> where <comma-separated list of statements>

For example formula:

a + b where a = 2, b = 4

will give as result 6.

* functions: Returns a list of all built-in and custom functions available to the AI

* def: This keyword creates functions using the syntax:

def function_name(arg1, arg2, ....) function_body

For example,

def sum(x,y) x + y

creates a function sum taking two arguments and returns their sum.

== Unit Formulas ==

You san specify a formula for any kind of unit. This is a simple way of doing it:

[unit]
...
formula="move(me.loc, loc(me.loc.x, me.loc.y - 1))"
[/unit]

Custom unit formulas are executed first at the begining of side's turn. Above formula will simply move unit one hex to the north every turn. Note how "me" keyword allows access to unit itself.

You can also define AI unit-specific variables and use them in you formulas:

[unit]
...
formula="if(attack, attack, move(me.loc, choose(unit_moves(me.loc), -distance_between(self, me.vars.guard_loc))))
where attack = choose(filter(attacks, units = [me.loc] and distance_between(me.vars.guard_loc, target) <= me.vars.guard_radius), avg_damage_inflicted)"
[ai_vars]
guard_radius=3
guard_loc="loc(8,5)"
[/ai_vars]
[/unit]

This formula will get location position from variable guard_loc and make sure that unit attacks opponent which is 3 hexes (value specified by guard_radius variable) or less from guard_loc.

Types of variables that are supported:
*number:
variable=3
*text (important: note the ' ' within " "):
name="'I am variable'"
*list:
number_list=[ 1, 2, 3]
*location:
place="loc(X,Y)"

[[Category:Development]]

FormulaAIandDynamicScripting

2008-03-26T02:21:07Z

Dhains: /* Formula AI - Rulebases */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Rulebases - Formula AI==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-26T02:19:25Z

Dhains: Some major changes to coincide with official GSoC Application

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science and entered a PhD graduate program focusing on AI research. I use C++ primarily for research and have been a teaching assistant for our undergraduate C++ course. I have approximately 4/5 years of C++ experience and some Python experience.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Formula AI - Rulebases==

Rulebases contain the FormulaAI rules the AI can use to form scripts. The rulebases developed for this project will form a repository of strategies and behaviors that designers can simply plug into a WML file to create a highly effective AI without ever touching FormulaAI or the underlying learning processes. Of course, if a designer desires a new strategy they can write custom FormulaAI rules.

I plan to involve the community in this portion of the project to identify the common strategies used by players and desired by scenario and MP bot designers. The how to play series will also serve as a useful guide. I foresee three categories of rulebases the designer can choose from to customize the AI for a particular scenario or deathmatch.

* RECRUITMENT RULEBASES

These rulebases will cover strategies to recruit units. A default recruitment strategy will be provided that selects the best units for the desired scenario goal.

* TEAM RULEBASES

Team rulebases will govern team strategies and will supply the majority of rules the AI can use when creating a script. Developed rulebases cover the majority of scenario and MP objectives, such as 'Escort unit x to hex y', 'Assassinate enemy leader', etc.

* UNIT RULEBASES

Unit rulebases provide rules for unit specific behavior. Some of these rulebases will be associated to units by default, for instance a 'healing' rule for healing units, 'backstab' to thieves, etc.

A variety of rulebases to customize unit behavior to a scenario storyline will also be available, such as 'Hatred towards faction x' and can be applied to single units or groups.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

== Adaptation and Learning - C++ ==

The C++ portion of the project allows the AI to learn and adapt. Once the candidate moves are determined, the AI will rank the moves based on evaluation functions. The actual evaluation functions will be written in Formula AI, the C++ code should never have to be touch for customization or extension purposes.

There will be two evaluation functions: A team evaluation function, in which the impact of each move on the overall team strategy is evaluated and a unit evaluation function, in which the impact of a move on unit involved is evaluated. The results these functions determine the final evaluation of a move. Once all moves are evaluated, the script is formed based on these evaluations (i.e. best moves first).

At the end of turn, the success of each move is used to adjust a weight associated with that move. The weights are incorporated into the evaluation of moves during the next turn (e.g. weight * (unit_eval(formula) + team_eval(formula))). In this way, the AI can learn from it's mistakes and exploit holes in an opponent's strategy found by successful moves.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-26T02:12:33Z

Dhains: /* Overview */

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science (where I achieved a perfect 4.0 gpa) and entered a PhD graduate program for computer science focusing on AI research. My research to this point has been focused on machine learning and evolutionary computing techniques for parameter optimization, but many of the techniques have a wide variety of applications. I use C++ primarily for my research and have been a teaching assistant for our undergraduate C++ course.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

==Overview==

Designing an AI for playing games is a challenge. Techniques such as minimax and alpha-beta pruning are useful in games such as chess or backgammon, but the complexity of Wesnoth's gamestate space precludes many of these approaches.

A common method to overcome this problem is to use scripting. Manually designed rules are created to determine the course of action, for instance if the AI should attack or defend. While these methods are effective it is generally not enough to provide challenging play, especially to experienced players. Players can 'outsmart' a scripted AI simply by exploiting the predictability inherent to scripting.

An ideal AI is one which exhibits human-like behavoir. One which can adapt its strategy to cover holes exploited by players. One which has the ability of surprise, to make the player feel as if he is playing a thinking, cunning opponent instead of just 'trying to beat the computer'. Such an AI is not a pipe dream and even more to the point, is quite feasible from a technical standpoint.

This is a proposal to implement such an AI. I propose to use dynamic scripting to combine manually designed rulebases and online learning to create a customizable, extensible and adaptive AI. his type of dynamic planning has already proven successful in creating adaptive, formidable A.I. in games such as F.E.A.R. and Neverwinter Nights [1,2].

The rulebases will be implemented in FormulaAI while the reinforcement learning portion will be handled by C++. This accomplishes two goals: 1, the AI can easily be customized and extended without touching the C++ code and with no knowledge of the reinforcement learning process and 2, the learning process can easily be 'switched off', perhaps as a difficulty setting or to force strict adherence to a script.

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself (using dynamic scripting) as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be two types of rulebases: team rulebases and unit rulebases. A team rulebase will contain formulas which more or less correspond to scenario objectives (or deathmatch, MP, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the AI to win the round. Unit rulebases, cover unit specific behavior such as 'Hatred towards faction X', 'Hide and Ambush' or 'Healing unit'. These rulebases can be applied to specific units or classes of units to allow them to use unit specific strategies and techniques. This allows for an easily customizable AI for scenario designers, MP bots for co-op campaign play, death matches, etc.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

==Evaluation Functions==

Along with the rulebases defined above also comes evaluation functions appropriate to each rulebase. These will be included with the formula scripts for each rulebase and will be transparent to the designer unless he or she wishes to modify them to fine tune the behavior. These evaluation functions will be used to determine the probability of a particular move.

I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the team goal (this one will come from the team formula) and another on the unit level (from the unit formulas). Both evaluation functions will be evaluated on the candidate moves and a combination of the results will be used to determine the final probability of a move being chosen (This is the method used in Dynamic Scripting).

==Dynamic Scripting==

Finally, I would like to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

This can be accomplished by Dynamic Scripting, a proven (and relatively cutting edge) technique. Actually, everything described thus far is the general dynamic scripting as I would apply it to Wesnoth. The adaptability comes into play with the following addition.

As described above, each unit will have some probability associated with it assigned by the evaluation functions. The evaluation functions themselves are defined in the formula scripts and as such cannot be modified directly by the AI (at least for now). However we can assign a weight (initially set to 1) to each probability and allow the AI to modify this weight as it determines whether moves are successful or not. In this way, we give the AI ability to adapt it's techniques and an easy way to turn the learning portion off (set the weights to 1, don't allow change) for experimentation and perhaps adjust the difficulty setting.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T07:51:53Z

Dhains: More personal info

==About Me==

I am currently a full-time graduate student living in Colorado, US. I've been coding since I was about 12, initially writing text based adventures for myself and my friends to play, moving on to RPGs using some primitive sprite based graphics and completed a mini RPG in high school, complete with graphics and music I wrote on my keyboard.

Since then, I have graduated from Pennsylvania State University with a bachelors degree in computer science (where I achieved a perfect 4.0 gpa) and entered a PhD graduate program for computer science focusing on AI research. My research to this point has been focused on machine learning and evolutionary computing techniques for parameter optimization, but many of the techniques have a wide variety of applications. I use C++ primarily for my research and have been a teaching assistant for our undergraduate C++ course.

===Contact Information===

* IRC: barbarianhero

* Forum id: rende

* Gna! username: dhains

* preferred email: dhains__A!T__ gmail.com

I'll be adding more to this page as I have the time. Please feel free to contact me if you have any questions.

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of a single large database of moves, scenario AIs, MP bots, death match AI's, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself (using dynamic scripting) as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be two types of rulebases: team rulebases and unit rulebases. A team rulebase will contain formulas which more or less correspond to scenario objectives (or deathmatch, MP, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the AI to win the round. Unit rulebases, cover unit specific behavior such as 'Hatred towards faction X', 'Hide and Ambush' or 'Healing unit'. These rulebases can be applied to specific units or classes of units to allow them to use unit specific strategies and techniques. This allows for an easily customizable AI for scenario designers, MP bots for co-op campaign play, death matches, etc.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

==Evaluation Functions==

Along with the rulebases defined above also comes evaluation functions appropriate to each rulebase. These will be included with the formula scripts for each rulebase and will be transparent to the designer unless he or she wishes to modify them to fine tune the behavior. These evaluation functions will be used to determine the probability of a particular move.

I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the team goal (this one will come from the team formula) and another on the unit level (from the unit formulas). Both evaluation functions will be evaluated on the candidate moves and a combination of the results will be used to determine the final probability of a move being chosen (This is the method used in Dynamic Scripting).

==Dynamic Scripting==

Finally, I would like to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

This can be accomplished by Dynamic Scripting, a proven (and relatively cutting edge) technique. Actually, everything described thus far is the general dynamic scripting as I would apply it to Wesnoth. The adaptability comes into play with the following addition.

As described above, each unit will have some probability associated with it assigned by the evaluation functions. The evaluation functions themselves are defined in the formula scripts and as such cannot be modified directly by the AI (at least for now). However we can assign a weight (initially set to 1) to each probability and allow the AI to modify this weight as it determines whether moves are successful or not. In this way, we give the AI ability to adapt it's techniques and an easy way to turn the learning portion off (set the weights to 1, don't allow change) for experimentation and perhaps adjust the difficulty setting.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T07:35:10Z

Dhains: Added a bit more detail

==Me==

IRC: barbarianhero

Forum id: Rende

gna! username: dhains

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of a single large database of moves, scenario AIs, MP bots, death match AI's, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself (using dynamic scripting) as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be two types of rulebases: team rulebases and unit rulebases. A team rulebase will contain formulas which more or less correspond to scenario objectives (or deathmatch, MP, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the AI to win the round. Unit rulebases, cover unit specific behavior such as 'Hatred towards faction X', 'Hide and Ambush' or 'Healing unit'. These rulebases can be applied to specific units or classes of units to allow them to use unit specific strategies and techniques. This allows for an easily customizable AI for scenario designers, MP bots for co-op campaign play, death matches, etc.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

==Evaluation Functions==

Along with the rulebases defined above also comes evaluation functions appropriate to each rulebase. These will be included with the formula scripts for each rulebase and will be transparent to the designer unless he or she wishes to modify them to fine tune the behavior. These evaluation functions will be used to determine the probability of a particular move.

I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the team goal (this one will come from the team formula) and another on the unit level (from the unit formulas). Both evaluation functions will be evaluated on the candidate moves and a combination of the results will be used to determine the final probability of a move being chosen (This is the method used in Dynamic Scripting).

==Dynamic Scripting==

Finally, I would like to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

This can be accomplished by Dynamic Scripting, a proven (and relatively cutting edge) technique. Actually, everything described thus far is the general dynamic scripting as I would apply it to Wesnoth. The adaptability comes into play with the following addition.

As described above, each unit will have some probability associated with it assigned by the evaluation functions. The evaluation functions themselves are defined in the formula scripts and as such cannot be modified directly by the AI (at least for now). However we can assign a weight (initially set to 1) to each probability and allow the AI to modify this weight as it determines whether moves are successful or not. In this way, we give the AI ability to adapt it's techniques and an easy way to turn the learning portion off (set the weights to 1, don't allow change) for experimentation and perhaps adjust the difficulty setting.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T07:23:05Z

Dhains: /* Evaluation Functions */

==Me==

IRC: barbarianhero

Forum id: Rende

gna! username: dhains

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of a single large database of moves, scenario AIs, MP bots, death match AI's, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself (using dynamic scripting) as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be two types of rulebases: team rulebases and unit rulebases. A team rulebase will contain formulas which more or less correspond to scenario objectives (or deathmatch, MP, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the AI to win the round. Unit rulebases, cover unit specific behavior such as 'Hatred towards faction X', 'Hide and Ambush' or 'Healing unit'. These rulebases can be applied to specific units or classes of units to allow them to use unit specific strategies and techniques. This allows for an easily customizable AI for scenario designers, MP bots for co-op campaign play, death matches, etc.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

==Evaluation Functions==

Along with the rulebases defined above also comes evaluation functions appropriate to each rulebase. These will be included with the formula scripts for each rulebase and will be transparent to the designer unless he or she wishes to modify them to fine tune the behavior. These evaluation functions will be used to determine the probability of a particular move.

I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the team goal (this one will come from the team formula) and another on the unit level (from the unit formulas). Both evaluation functions will be evaluated on the candidate moves and a combination of the results will be used to determine the final probability of a move being chosen (This is the method used in Dynamic Scripting).

==Dynamic Scripting==

I was hoping to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

One method that seems quite suitable is dynamic scripting. Each move in the rulebase has an associated weight which influences the probability of a given rule being used. As a game progresses, the weights are updated automatically according to the success or failure of past moves of the current game (or campaign) according to both and individuals state (unit eval function) and the overall game state (team eval function). This would allow the AI to adapt to a player's tactics and encourage the player to develop diverse strategies for use in campaign mode.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T07:12:03Z

Dhains: Updated rulebases with a clearer definition

==Me==

IRC: barbarianhero

Forum id: Rende

gna! username: dhains

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of a single large database of moves, scenario AIs, MP bots, death match AI's, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself (using dynamic scripting) as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be two types of rulebases: team rulebases and unit rulebases. A team rulebase will contain formulas which more or less correspond to scenario objectives (or deathmatch, MP, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the AI to win the round. Unit rulebases, cover unit specific behavior such as 'Hatred towards faction X', 'Hide and Ambush' or 'Healing unit'. These rulebases can be applied to specific units or classes of units to allow them to use unit specific strategies and techniques. This allows for an easily customizable AI for scenario designers, MP bots for co-op campaign play, death matches, etc.

For example, a scenario designer might want to create a scenario in which a group of orcs, goblins and ogres must escort an orcish leader across a map to hex 5,10. The storyline might dictate that the goblins and ogres are only helping the orcs for a chance to kill elves, which the player has the ability to recruit. The scenario designer could implement this quite easily in their cfg file for that scenario with something like

[ai]
[team_formula]
rulebase = "escort"
parameters = "Orcish Leader", 5, 10
[\team_formula]
[unit_formula]
apply_to_units = "goblins", "ogres"
rulebase = "faction_hatred"
parameters = "Elves"
[\unit_faction]
[\ai]

Of course the designer could make things a bit more complicated, by creating multiple team strategies associated to different units, e.g. suppose in the above example the AI also had a renegade faction of elvish rangers along for the ride, hellbent on destroying the human leader and don't really care about escorting the orcish leader. The designer might create an entirely new side, but if he or she wanted all the units on a single side, he might add the following to the above ai section.

[unit_formula]
apply_to_units = "Elvish Ranger", "Elvish Avenger"
rulebase = "hide_and_ambush" # Make elves stay hidden if possible until they attack
[team_formula] # This will override the "escort" team formula
rulebase = "assassinate"
parameters = "Human Leader"
[\team_formula]
[\unit_formula]

The AI designer can go deeper or shallower if necessary. An adequate default AI with appropriate unit rulebases and a 'kill all' team rulebase will be the default if none are specified. If the provided rulebases do not cover some specific behavior, the designer of course may implement his own rulebase by creating a custom formula script or by altering the evaluation functions of the existing rulebases.

==Evaluation Functions==
So once the global, recruitment and unit rulebases have been defined, the Formula AI in these rulebases forms a pool of candidate moves for the AI to follow. These moves will have a probability defined to them by the evaluation functions and these probabilities will be used in determining what action to take.

Following the concept of team and unit behaviors, I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the tam goal and another on the unit level. A combination of the result of both of these eval functions will used to determine the final probability of a move being chosen (This is also the same method used in Dynamic Scripting).

==Dynamic Scripting==

I was hoping to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

One method that seems quite suitable is dynamic scripting. Each move in the rulebase has an associated weight which influences the probability of a given rule being used. As a game progresses, the weights are updated automatically according to the success or failure of past moves of the current game (or campaign) according to both and individuals state (unit eval function) and the overall game state (team eval function). This would allow the AI to adapt to a player's tactics and encourage the player to develop diverse strategies for use in campaign mode.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T06:30:54Z

Dhains: /* Overview */

==Me==

IRC: barbarianhero

Forum id: Rende

gna! username: dhains

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of a single large database of moves, scenario AIs, MP bots, death match AI's, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be team rulebases, which more or less correspond to scenario objectives (or deathmatch, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the entire force to win the round. There will also be unit rulebases, which can be associated with particular units, such as a 'Hatred towards faction X' behavior. The rulebases will be as general and parameterized, to allow for as much reuse as possible.

This allow for an easily customizable AI for scenario designers, MP bots (perhaps for co-op campaign play), death matches, etc. For example, a scenario designer could simply select a team and recruitment rulebase appropriate to his scenario (probably specified in WML) and associate his Orcs with a hatred vs elves unit rulebase. This would give the designer's wolf riders a behavior biased towards aggressively attacking elves, but the overall team strategy will still be accomplished.

Of course the designer could make things a bit more complicated, perhaps by creating multiple team strategies associated to different units, i.e. Most units could be assigned candidate moves from a 'defend leader' team rulebase while camouflaged units could follow an 'assassinate leader' team rulebase, and since they are camouflaged units they might also be associated with a 'hide & attack' unit rulebase. The scenario designer could accomplish this in a few lines of WML and have some really cool results, i.e. the AI puts up a solid strong, ZoC defense but is also circling around a camo squad to ambush to opposing the leader. And of course, the whole time the AI is learning and adapting on it's own to the opposing strategies.

I think coming up with the specific rulebases and such will really start rolling after a fair amount of interaction with the community, specifically players and scenario designers.

===Rulebase Categories===
====Recruitment Rulebases====
Rulebases to govern the recruitment process, this may just be one parameterized rulebase to begin with, but most likely several rulebases for different recruiting strategies.

====Team Rulebases====
These will be more general rules for which all units would abide (We could create an easy way for certain units to ignore any team rulebase, or to follow another to setup a renegade faction within a computer controlled side, for instance a band of outlaws (thugs, poachers) which are helping the computer controlled side fight but simply for the sake of killing elves and have no care for the overall objective of the AI.) Mostly, the provided rulebases in this category will cover the most common scenarios and deathmatch games, i.e. kill opponents leader, defend for x turns, escort x to hex y, kill at all cost, etc.

====Unit Rulebases====
These are rulebases which can be associated to units. Scenario designers can apply some unit rulebases to all units in scenario and then customize specific units on a case by case basis if needed, to say setup faction behaviors between certain units. This could not be just aggressive but also perhaps a healing unit prefers to heal units of it's own faction. Healing units could be associated with a general unit rulebase which tells them to heal wounded units and could be even further customized with a heal elves over dwarves behavior.

==Evaluation Functions==
So once the global, recruitment and unit rulebases have been defined, the Formula AI in these rulebases forms a pool of candidate moves for the AI to follow. These moves will have a probability defined to them by the evaluation functions and these probabilities will be used in determining what action to take.

Following the concept of team and unit behaviors, I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the tam goal and another on the unit level. A combination of the result of both of these eval functions will used to determine the final probability of a move being chosen (This is also the same method used in Dynamic Scripting).

==Dynamic Scripting==

I was hoping to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

One method that seems quite suitable is dynamic scripting. Each move in the rulebase has an associated weight which influences the probability of a given rule being used. As a game progresses, the weights are updated automatically according to the success or failure of past moves of the current game (or campaign) according to both and individuals state (unit eval function) and the overall game state (team eval function). This would allow the AI to adapt to a player's tactics and encourage the player to develop diverse strategies for use in campaign mode.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T06:28:49Z

Dhains: /* Me */

==Me==

IRC: barbarianhero

Forum id: Rende

gna! username: dhains

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of large database of moves, MP bots, death match AI's, scenario AIs, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be team rulebases, which more or less correspond to scenario objectives (or deathmatch, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the entire force to win the round. There will also be unit rulebases, which can be associated with particular units, such as a 'Hatred towards faction X' behavior. The rulebases will be as general and parameterized, to allow for as much reuse as possible.

This allow for an easily customizable AI for scenario designers, MP bots (perhaps for co-op campaign play), death matches, etc. For example, a scenario designer could simply select a team and recruitment rulebase appropriate to his scenario (probably specified in WML) and associate his Orcs with a hatred vs elves unit rulebase. This would give the designer's wolf riders a behavior biased towards aggressively attacking elves, but the overall team strategy will still be accomplished.

Of course the designer could make things a bit more complicated, perhaps by creating multiple team strategies associated to different units, i.e. Most units could be assigned candidate moves from a 'defend leader' team rulebase while camouflaged units could follow an 'assassinate leader' team rulebase, and since they are camouflaged units they might also be associated with a 'hide & attack' unit rulebase. The scenario designer could accomplish this in a few lines of WML and have some really cool results, i.e. the AI puts up a solid strong, ZoC defense but is also circling around a camo squad to ambush to opposing the leader. And of course, the whole time the AI is learning and adapting on it's own to the opposing strategies.

I think coming up with the specific rulebases and such will really start rolling after a fair amount of interaction with the community, specifically players and scenario designers.

===Rulebase Categories===
====Recruitment Rulebases====
Rulebases to govern the recruitment process, this may just be one parameterized rulebase to begin with, but most likely several rulebases for different recruiting strategies.

====Team Rulebases====
These will be more general rules for which all units would abide (We could create an easy way for certain units to ignore any team rulebase, or to follow another to setup a renegade faction within a computer controlled side, for instance a band of outlaws (thugs, poachers) which are helping the computer controlled side fight but simply for the sake of killing elves and have no care for the overall objective of the AI.) Mostly, the provided rulebases in this category will cover the most common scenarios and deathmatch games, i.e. kill opponents leader, defend for x turns, escort x to hex y, kill at all cost, etc.

====Unit Rulebases====
These are rulebases which can be associated to units. Scenario designers can apply some unit rulebases to all units in scenario and then customize specific units on a case by case basis if needed, to say setup faction behaviors between certain units. This could not be just aggressive but also perhaps a healing unit prefers to heal units of it's own faction. Healing units could be associated with a general unit rulebase which tells them to heal wounded units and could be even further customized with a heal elves over dwarves behavior.

==Evaluation Functions==
So once the global, recruitment and unit rulebases have been defined, the Formula AI in these rulebases forms a pool of candidate moves for the AI to follow. These moves will have a probability defined to them by the evaluation functions and these probabilities will be used in determining what action to take.

Following the concept of team and unit behaviors, I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the tam goal and another on the unit level. A combination of the result of both of these eval functions will used to determine the final probability of a move being chosen (This is also the same method used in Dynamic Scripting).

==Dynamic Scripting==

I was hoping to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

One method that seems quite suitable is dynamic scripting. Each move in the rulebase has an associated weight which influences the probability of a given rule being used. As a game progresses, the weights are updated automatically according to the success or failure of past moves of the current game (or campaign) according to both and individuals state (unit eval function) and the overall game state (team eval function). This would allow the AI to adapt to a player's tactics and encourage the player to develop diverse strategies for use in campaign mode.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAIandDynamicScripting

2008-03-23T06:28:19Z

Dhains: /* Me */

==Me==

IRC: barbarianhero
Forum id: Rende

==Overview==

Many types of AI techniques to provide intelligent strategies in games revolve around the concept of search and evaluation. The problem with search is it can simply take too long. Anyone who has played a chess AI knows of the delays between moves due to this search and eval procedure. These delays would interrupt the flow of game play in Wesnoth, so the approach used in Dynamic Scripting can be applied to heavily cut down on the search portion by coding domain knowledge of the game into sets of moves known as rulebases. Evaluation and selection of exactly what moves to make is still up to the AI, but we can cut the computational costs significantly by providing a pool of candidate moves.

By using modular rulebases instead of large database of moves, MP bots, death match AI's, scenario AIs, etc. can be designed easily by plugging together appropriate rulebases as described below. The designer can achieve a high level of customization and still have an intelligent, adaptive AI as a result. The dynamic scripting technique has been applied successfully in creating an aftermarket AI to control characters in Neverwinter Nights. A party consisting of a variety of character classes was able to adapt and cooperate to defeat an opposing force. It is not much of a stretch to see how this technique could be applied to Wesnoth. There are quite a few more characters to be controlled, but the choice of actions is less (e.g. they don't have a plethora of items and spells at their disposal on top of moving and attacking).

==Formula AI==

Formula AI will be used for designing the AI rulebases described below. A rulebase can be thought of as a set of candidate moves written in Formula AI which accomplish some particular behavior or strategy. Much of the work will be defining what behaviors/strategies are required and implementing the appropriate moves in Formula AI. I believe there will be some back and forth between implementing rulebases in FormulaAI and extending Formula AI when additional functionality needs to be added. These two things will probably be a majority of the project.

==Rulebases==

Think of a rulebase as a collection of moves which coordinate some type of behavior (i.e. assassinate opposing leader, defend castle with ZoC, etc.). The moves in each rulebase are not necessarily the moves which are taken at any given point in a game but rather they create a pool of candidate moves the AI can choose from. Exactly what move is chosen from this pool of candidate moves depends on an evaluation function which assigns probabilities to each move. These probabilities can be altered by the AI itself as it plays through a game, thus providing the AI the ability to adapt to an opponents strategy.

There will be team rulebases, which more or less correspond to scenario objectives (or deathmatch, etc.) such as 'Escort unit X to Hex Y'. These moves will be used to govern the overall behavior of the entire force to win the round. There will also be unit rulebases, which can be associated with particular units, such as a 'Hatred towards faction X' behavior. The rulebases will be as general and parameterized, to allow for as much reuse as possible.

This allow for an easily customizable AI for scenario designers, MP bots (perhaps for co-op campaign play), death matches, etc. For example, a scenario designer could simply select a team and recruitment rulebase appropriate to his scenario (probably specified in WML) and associate his Orcs with a hatred vs elves unit rulebase. This would give the designer's wolf riders a behavior biased towards aggressively attacking elves, but the overall team strategy will still be accomplished.

Of course the designer could make things a bit more complicated, perhaps by creating multiple team strategies associated to different units, i.e. Most units could be assigned candidate moves from a 'defend leader' team rulebase while camouflaged units could follow an 'assassinate leader' team rulebase, and since they are camouflaged units they might also be associated with a 'hide & attack' unit rulebase. The scenario designer could accomplish this in a few lines of WML and have some really cool results, i.e. the AI puts up a solid strong, ZoC defense but is also circling around a camo squad to ambush to opposing the leader. And of course, the whole time the AI is learning and adapting on it's own to the opposing strategies.

I think coming up with the specific rulebases and such will really start rolling after a fair amount of interaction with the community, specifically players and scenario designers.

===Rulebase Categories===
====Recruitment Rulebases====
Rulebases to govern the recruitment process, this may just be one parameterized rulebase to begin with, but most likely several rulebases for different recruiting strategies.

====Team Rulebases====
These will be more general rules for which all units would abide (We could create an easy way for certain units to ignore any team rulebase, or to follow another to setup a renegade faction within a computer controlled side, for instance a band of outlaws (thugs, poachers) which are helping the computer controlled side fight but simply for the sake of killing elves and have no care for the overall objective of the AI.) Mostly, the provided rulebases in this category will cover the most common scenarios and deathmatch games, i.e. kill opponents leader, defend for x turns, escort x to hex y, kill at all cost, etc.

====Unit Rulebases====
These are rulebases which can be associated to units. Scenario designers can apply some unit rulebases to all units in scenario and then customize specific units on a case by case basis if needed, to say setup faction behaviors between certain units. This could not be just aggressive but also perhaps a healing unit prefers to heal units of it's own faction. Healing units could be associated with a general unit rulebase which tells them to heal wounded units and could be even further customized with a heal elves over dwarves behavior.

==Evaluation Functions==
So once the global, recruitment and unit rulebases have been defined, the Formula AI in these rulebases forms a pool of candidate moves for the AI to follow. These moves will have a probability defined to them by the evaluation functions and these probabilities will be used in determining what action to take.

Following the concept of team and unit behaviors, I expect to have two evaluation functions, one for evaluating a move based on the overall impact on the tam goal and another on the unit level. A combination of the result of both of these eval functions will used to determine the final probability of a move being chosen (This is also the same method used in Dynamic Scripting).

==Dynamic Scripting==

I was hoping to apply some machine learning concepts to the AI, specifically online learning. Online learning allows the AI to adapt it's strategy to fix holes being exploited by the human player and over all to perform more intelligently, giving the player a feeling he or she is playing against a thinking, clever opponent.

One method that seems quite suitable is dynamic scripting. Each move in the rulebase has an associated weight which influences the probability of a given rule being used. As a game progresses, the weights are updated automatically according to the success or failure of past moves of the current game (or campaign) according to both and individuals state (unit eval function) and the overall game state (team eval function). This would allow the AI to adapt to a player's tactics and encourage the player to develop diverse strategies for use in campaign mode.

==Related Papers==

Online Adaptation of Game Opponent AI in Simulation and in Practice, Spronk et al. [http://www.fdaw.unimaas.nl/education/3.1cs/postma/GAMEON2003-Paper8-Spronck.pdf]

[[Category:Summer of Code]]

FormulaAI

2008-03-22T01:34:49Z

Dhains: Added a list for keywords

FormulaAI

2008-03-22T01:33:41Z

Dhains: Added def keyword

FormulaAI

2008-03-20T23:37:48Z

Dhains: added Keywords section