A database of patterns is supplied in patterns.db These are ascii representations, of the form:
Pattern EB115 ??o| sente hane ?Oo| XX*| ...| ?.?| :8,50,0,O,5,5,5,0,0,sente_hane_helper
where `O' marks a friendly stone, `X' marks enemy stones, `.' marks an empty vertex, `*' marks `O''s next move, `o' marks a square either containing `O' or empty but not `X'. (The symbol `x', which does not appear in this pattern, means `X' or `.') Finally `?' Indicates a location where we don't care what is there, except that it cannot be off the edge of the board.
The line of `|''s along the right in this example is the edge of the board itself--this is an edge pattern. Corners can also be indicated. So this pattern describes a hane on the first line. The `o' makes sure that it would not be in atari immediately, though the matcher can check for this automatically (see class). Elements are not generated for `?' markers, but they are not completely ignored - see below. The line beginning `:' describes various attributes of the pattern, such as its symmetry and its importance. Optionally, a function called a "helper" can be provided to assist the matcher in deciding the worth of the move, and a simple measure of the influence of nearby stones can be factored in. In this case, there is a helper, the sente_hane_helper, which may be found in helpers.c. Most patterns do not require a helper, and this field is filled with NULL.
The matcher searches the board for places where this layout appears on the board, and chooses the highest scoring pattern.
After the pattern, some supplementary information in the format:
:trfno, patwt, assistance, classification, obonus, xbonus, splitbonus, minrand, maxrand, helper_function
Here trfno represents the number of transformations of the pattern to
consider, usually 8 (no symmetry, for historical reasons), or one of
`|', `\', `/', `-', `+', `X', where the
line represents the axis of symmetry. (E.g. `|' means
symmetrical about a vertical axis.) patwt
is the numerical pattern
value - if there is a helper function (see below), it is the maximum
weight it can return.
The assistance attribute reflects the fact that a pattern may have different values depending on external circumstances. For example, a pattern to connect is less important where I am powerful, but more important where my opponent is powerful. There are two different methods of assistance, wind assistance (see section Wind Assistance) and moyo assistance (see section Moyo Assistance).
The classification scheme is as follows : a sequence of zero or more of the following characters, each with a different meaning.
no checking is done. This is appropriate for sacrifice patterns. Otherwise, the matcher requires that the stone played cannot be trivially captured.
In addition to usual check that the stone played cannot be trivially captured, it is also confirmed that an opponent move here could not be captured.
It is checked that every friendly (`O') stone of the pattern belongs to a dragon which is classified as ALIVE or UNKNOWN.
It is checked that every friendly (`O') stone of the pattern belongs to a dragon which is classified as DEAD or UNKNOWN.
It is checked that every opponent (`X') stone of the pattern belongs to a dragon which is classified as ALIVE or UNKNOWN.
It is checked that every opponent (`X') stone of the pattern belongs to a dragon which is classified as DEAD or UNKNOWN
If it is found that an `X' stone at `(m,n)' of the pattern can be captured (
worm[m][n].attacki != 0
) then the move at `*' is tried. If it is found to attack the stone,worm[m][n].attack
is set to equal `*'. This means that the attacker will later use this move to attack the stone.
If it is found that an `O' stone at `(m,n)' of the pattern can be captured (
worm[m][n].attacki != 0
) then the move at * is tried. If it is found to defend the stone,worm[m][n].defend
is set to equal `*'. This means that the defender will later use this move to defend the stone.
If two distinct `O' dragons occur in the pattern, the pattern is given the minimum of the
connection_value
and the return value of the pattern.
If two distinct `X' dragons occur in the pattern, the pattern is given the minimum of the
connection_value
and the return value of the pattern.
Match the pattern and run the helper, even if the weight is less than that of the largest pattern found already. This is appropriate for patterns whose significance is in the side effects of the helpers.
One common classification is `OX' (which rejects pattern if either sides stones are dead). The string `-' may be used as a placeholder. (In fact any characters other than the above and `,' are ignored.)
`o' and `O' could conceivably appear in a class, meaning it
applies only to UNKNOWN
. Similarly `X' and `x' could
be used together.
Some care must be taken with the `A' and `D' classes. If more than one worm that can be attacked or defended is present in the pattern, only one of them, arbitrarily chosen, will be found.
The classes `B' and `C' can be used together. If both connection values are greater than 0, the pattern is given a combined value which is the larger of them plus a fraction of the smaller one.
The values obtained for the `B' and `C' classes are further
limited by the sum of the primary pattern weight (patwt
) and the
assistance value. The bonuses described below are applied after this
limitation.
The field obonus
is a bonus which is added when the pattern contains
any dragon of color `O' with dragon[m][n].safety != 0
.
This means that the dragon has size at least 2 and between 0 and 20
points of area, computed by the Bouzy 4/0 algorithm. Similarly the
xbonus is added when the pattern contains at least one weak X dragon.
The field splitbonus is a bonus which is added when the move splits opponent dragons on a large scale or joins own dragons in the same manner (see section Moyo).
To avoid playing the same moves each game, minrand and maxrand specifies a random adjustment of the move value, uniformly distributed between minrand and maxrand, inclusively. This feature is primarily used for fuseki moves, where the choice of exact moves is a matter of inspiration anyway.
helper_fn is the name of a C function which will be invoked to assist in the evaluation of the pattern. It will be passed the co-ordinates on the board of the pattern element marked `*', the rotation of the pattern which has been matched, and the color of the piece for whom the move is being considered. (`O' in the key above). Facilities are provided for navigating around the pattern taking the rotation into account.
Usually a pattern will only contribute a move if its value is large enough to outweigh all other moves which have been found. There is an exception to this, however. If the pattern classification string contains a `D', the pattern is a defensive one. If an `O' string is found in the pattern which can be captured, and if the move at `*' defends it, then the point of defense (worm[m][n].defendi, worm[m][n].defendj) is moved to `*'. This means that even if the pattern has small value, the defensive move will be remembered later when defender() is run.
Another exception is patterns with a classification string containing an `A'. These patterns are offensive ones. If an `X' string is found in the pattern which can be captured but also defended, and if the move at `*' also attacks it, then the point of attack (worm[m][n].attacki, worm[m][n].attackj) is moved to `*'. This means that even if the pattern has small value, the offensive move will be remembered later when attacker() is run. Notice that this means that the suggested move will never find an attack that wasn't found otherwise, but it can be used to capture enemy stones more efficiently or with better shape than the move attacker() would have found unassisted.
Helper functions can be provided to assist the matcher in weighing up the importance of a move. The helper is supplied with the compiled pattern entry in the table, and the (absolute) position on the board of the `*' point.
One difficulty is that the helper must be able to cope with all the
possible transformations of the pattern. To help with this, a
transformation number is supplied. This number can be passed to a
utility function offset()
with the relative co-ordinates in the
original, untransformed pattern. This function will return the actual
board co-ordinates to use for the indicated stone.
The actual helper functions are in `helpers.c'. They are declared in `patterns.h'.
As an example to show how to write a helper function, we consider defend_bamboo_helper. This begins with a comment:
/* ?X? ?X? O.O ObO O.* Oat */
The image on the left is the actual pattern. On the right we've taken this image and added letters to label (ti, tj), (ai, aj) and (bi, bj). Of course t is always at *, the point where GNU Go will move if the pattern is adopted.
int defend_bamboo_helper (ARGS) { int ai, aj, bi, bj; int tval=0; int other=OTHER_COLOR(color); OFFSET(0, -1, ai, aj); OFFSET(-1, -1, bi, bj); if (strategic_distance_to(other, ti, tj)>10) return (0); /* solid connection is better */ if (TRYMOVE(bi, bj, other)) { if (TRYMOVE(ai, aj, color)) { if (safe_move(ti, tj, other)) tval=COMPUTE_SCORE; popgo(); } popgo(); } return (tval); }
The OFFSET
s tell GNU Go the positions of the two stones at a=(ai,aj)
and b=(bi,bj)
. The correctness of the coordinates (relative to
t=*=(ti,tj)
) can be confirmed by consulting the diagram in the
prefatory comment. The macro TRYMOVE
invokes the function trymove()
,
ARGS
supplies standard arguments to the helper, and COMPUTE_SCORE
assigns the value of the pattern (see section Pattern Attributes).
The pattern is subjected to two tests. First, the strategic_distance to X (see section Worms and Dragons) must not be too great. The rationale behind this test is that if the strategic distance is great, then simply making a solid connection probably secures one point more territory. On the other hand if the strategic distance is small, the region in question may not be secure territory, and the bamboo joint is often better.
Hand-coding helpers such as this one is a powerful tool but not always needed. The same functionality can often be obtained more easily using an autohelper (see section Autohelpers).
Wind assistance, wind(ucutoff, uvalue, mycutoff, myvalue)
, is
based on the power of the stones in a neighborhood of the considered
move. My power (mypower
) and your power (upower
) are
measured by the function testwind()
. The actual value of the
wind assistance is given by the formula:
To calculate a color's power near the point *
, we sum 6-d
)
where d
is the distance of a stone to *
, where the sum
is over all stones of the given color with d<6
.
upower
and ucutoff
must have the same sign, as must
mypower
and mycutoff
. In practice we have mostly used
positive values for these parameters. We have always given uvalue and
myvalue the value 1 or in rare instances 2.
For example
"connect if invaded" OX.. .*.O .?.? :8,55,wind(20,1,0,0),-,0,0,0,NULL
These represent additional biases to the score for the influence of
nearby stones. The first pair are a multiplier and a cutoff for enemy
stones, and the second for friendly stones. The actual weight
(computed in the function compute_score()
) is given by the
formula:
uvalue*min(upower,abs(ucutoff)) + myvalue*min(mypower,abs(mycutoff)).
Typically uvalue (if nonzero) would have the value 1, meaning that the score increases by 1 for each increase in upower, up to a maximum of ucutoff, after which it does not increase. Thus in this example, the value of the pattern can increase up to 75, becoming more valuable when the opponent becomes strong in the area. This is a good feature for patterns which help the safety of our group.
Moyo assistance, moyo(moyocutoff, moyovalue)
, is based on an
estimation of "moyo" (see section Moyo).
In practice this is a combination of territory and influence for both
players. The function delta_moyo()
computes the difference in
moyo between the current position and after the move has been
made. The actual value of the moyo assistance is given by the formula:
moyovalue*min(deltamoyo,moyocutoff)
Since the pattern database decides GNU Go's personality to a very great extent, much time can be devoted to "tuning" it. Here are some suggestions.
If you want to experiment with modifying the pattern database, invoke with the -a option. This will cause every pattern to be evaluated, even if its maximum possible contribution is smaller than a pattern already found. This makes the program less efficient, but then you can see how much you must increase a pattern value in order to `promote' a better move over the move actually chosen.
You can obtain a Smart Go Format (SGF) record of your game in at least two different ways. One is to use CGoban to record the game. You can also have GNU Go record the game in Smart Go Format, using the -o option. It is best to combine this with -a. Do not try to read the sgf file until the game is finished and you have closed the sgf window. This does not mean that you have to play the game out to its conclusion. You may close the CGoban window on the game and GNU Go will close the sgf file so that you can read it.
If you record a game in SGF form using the -o
option, GNU Go
will add labels to the board to show all the moves it considered, with
their values. This is an extremely useful feature, since one can see
at a glance whether the right moves with appropriate weights are being
proposed by the pattern matcher. If bad moves are being proposed, one
may modify a pattern to exclude it, or reduce the value of the
pattern. If important moves are not proposed at all, you may have
found a gap in the pattern database, and you can add a pattern. If the
right move is proposed but with too low a score, this may be a sign
that you should adjust its weight upwards. It is almost always best to
make the *minimum* adjustment needed to correct the bad behavior.
If you decide to add a pattern, give some thought to adding the pattern in exactly the right generality by putting `?' at irrelevant locations, and by using the `o' and `x' options.
First, due to a bug of unknown nature, it occasionally happens that GNU Go will not receive the SIGTERM signal from CGoban that it needs to know that the game is over. When this happens, the sgf file ends without a closing parenthesis, and CGoban will not open the file. You can fix the file by typing:
echo ")" >>[filename]
at the command line to add this closing parenthesis. Or you could add the ")" using an editor.
Pattern weights exceeding 99 can be displayed by CGoban but you may have to resize the window in order to see all three digits. Grab the lower right margin of the CGoban window and pull it until the window is large. All three digits should be visible.
If you are playing a game without the -o option and you wish to analyze a move, you may still use CGoban's "Save Game" button to get an SGF file. It will not have the values of the moves labelled, of course.
Once you have a game game saved in SGF format, you can analyze any particular move by running:
gnugo -l [filename] -L [move number] -t -a -w
to see why GNU Go made that move, and if you make changes to the pattern database and recompile the program, you may ask GNU Go to repeat the move to see how the behavior changes.
Alternatively, you can use the CGoban tools to delete all moves after and including the one you want to study, and then load without the -L option.
If a pattern is contributing a bad move, you can adjust its weight downward, or you you can adjust the weight of a pattern which is contributing a good move up. If no pattern is contributing the move that you think should be made, then you may add a pattern.
You can also get a visual display of the dragons using the -T option. The default GNU Go configuration tries to build a version with color support using either curses or the ansi escape sequences. You are more likely to find color support in rxvt than xterm, at least on many systems, so we recommend running:
gnugo -l [filename] -L [move number] -T
in an rxvt window. If you do not see a color display, and if your host is a GNU/Linux machine, try this again in the Linux console.
Worms belonging to the same dragon are labelled with the same letters.
The colors indicate the value of the field dragon.safety
, which
is set in `moyo.c'.
Green: GNU Go thinks the dragon is alive Yellow: Status unknown Blue: GNU Go thinks the dragon is dead Red: Status critical (1.5 eyes) or weak by the algorithm in `moyo.c'
If you want to get the same game over and over again, you can
eliminate the randomness in GNU Go's play by changing the value of
seed in main.c to a fixed nonzero integer. If seed==0
, then GNU
Go will play a different game each time.
In addition to the hand-written helper functions in helpers.c there can be automatic helper functions. These are briefly described in the pattern database and compiled into C source which goes into `patterns.c'.
The "pattern compiler" is based on the idea of constraint diagrams and expressions. To give an example, one pattern consists of a pair of diagrams:
Pattern ED54 |oOOX maybe capture corner |..XO |.*XO |..?? +---- :8,85,0,s,10,0,0,0,0,NULL |obbX |..Aa |.*Aa |..?? +---- ;(lib(A)<=4) && (lib(a)>=lib(A)-1) && (lib(b)>=lib(A)) ;&& (!dead(A)||attack(a))
The first diagram and the colon line has exactly the same interpretation as usual. The diagram below and the semicolon line(s) are optional and can only be used to reject the pattern given above. Note that some strings now have labels which are referred to in the constraint.
Only strings for which we have constraints need be labeled. Labels may be any letter except OoXxt. To make the database consistent and easy to read it is our convention that `X' strings should be upper-case and `O' strings lower-case, but the implementation does not enforce this. Neither does it require that all stones in a string be labeled (it goes with the first appearance) but it is good practice to do so anyway.
The constraint expression is transformed by mkpat into an
automatically generated helper function (there is a new field in the
pattern struct, so it does not conflict with the old helper). lib(x)
can be regarded as a macro which is expanded by mkpat into
worm[xi][xj].liberties
. The resulting expression must be valid C code,
otherwise the generated `patterns.c' won't compile. In principle any
code can be written on the line but to keep the database maintainable
we should restrict ourselves to boolean and arithmetic expressions,
which anyone should be able to understand and write with no more than
a little trouble. If the expression evaluates to true the pattern is
accepted by the autohelper, if false it is rejected. If there are
multiple semicolon lines for the same pattern, these are concatenated
before generating the code.
lib(x) lib2(x) lib3(x) lib4(x)
Number of first, second, third, and fourth order liberties of a worm respectively (see section Worms and Dragons).
xlib(x) olib(x)
The number of liberties that an enemy or own stone, respectively, would obtain if played at the empty intersection x.
ko(x)
True if x is either a stone or an empty point involved in a ko position.
status(x) alive(x) unknown(x) critical(x) dead(x)
Status of a dragon. status(x)
returns an integer that can have
the values ALIVE
, UNKNOWN
, CRITICAL
, or
DEAD
. The four other functions returns true if the dragon has
the corresponding status and false otherwise (see section Worms and Dragons).
genus(x)
The number of eyes of a dragon. It is only meaningful to compare this value against 0, 1, or 2.
xarea(x) oarea(x) xmoyo(x) omoyo(x) xterri(x) oterri(x)
Functions related to various kinds of influence and territory
estimations (see section Moyo). xarea(x)
evaluates to true if
`x' is either a living enemy stone or an empty point within his "area".
oarea(x)
is analogous but with respect to our stones and area.
cutstone(x)
returns worm[x].cutstone
, which can be 0, 1,
or 2 (see section Worms and Dragons).
weak(x)
True for a dragon with safety==CRITICAL
.
attack(x) defend(x)
These give the results of tactical reading. attack(x)
is true if the
worm can be captured, defend(x)
is true if there also is a defending
move. Please notice that defend(x)
will return false if there is no
attack on the worm.
safe_xmove(x) safe_omove(x)
True if an enemy or own stone, respectively, can safely be played at `x'. By safe it is understood that the move is legal and that it cannot be captured right away.
odefend_against(x,y) xdefend_against(x,y)
True if an own stone at `x' would stop the enemy from safely playing at `y', and conversely for the second function.
does_defend(x,y) does_attack(x,y)
True if a move at `x' defends/attacks the worm at `y'. For defense a move of the same color as `y' is tried and for attack a move of the opposite color.
xplay_defend(a,b,c,...,z) oplay_defend(a,b,c,...,z) xplay_attack(a,b,c,...,z) oplay_attack(a,b,c,...,z)
These functions make it possible to do more complex reading experiments in the constraints. All of them work so that first the sequence of moves `a', `b', `c',... is played through with alternating colors, starting with `X' or `O' as indicated by the name. Then it is tested whether the worm at `z' can be attacked or defended, respectively. It doesn't matter who would be in turn to move, a worm of either color may be attacked or defended. For attacks the opposite color of the string being attacked starts moving and for defense the same color starts. The defend functions return true if the worm cannot be attacked in the position or if it can be attacked but also defended. The attack functions return true if there is a way to capture the worm, whether or not it can also be defended.
eye(x) proper_eye(x) marginal_eye(x)
True if `x' is an eye space for either color, a non-marginal eye space for either color, or a marginal eye space for either color, respectively.
The pattern code in GNU Go 2.6 is fairly straightforward conceptually, but because the matcher consumes a significant part of the time in choosing a move, the code is optimized for speed. Because of this there are implementation details which obscure things slightly.
In GNU Go 2.6, the ascii patterns.db file is precompiled into tables (see patterns.h) by a standalone program mkpat.c, and the resulting file patterns.c is compiled and linked into the main gnugo executable.
Each pattern is compiled to a header, and a sequence of elements, which are (notionally) checked sequentially at every position and orientation of the board. These elements are relative to the pattern 'anchor' (or origin). One `X' or `O' stone is (arbitrarily) chosen to represent the origin of the pattern. (We cannot dictate one or the other since some patterns contain only one colour or the other.) All the elements are in co-ordinates relative to this position. So a pattern matches "at" board position (m,n,o) if the the pattern anchor stone is on (m,n), and the other elements match the board when the pattern is transformed by transformation number 'o'. (See below for the details of the transformations, though these should not be necessary)
In general, each pattern must be tried in each of 8 different permutations, to reflect the symmetry of the board. But some patterns have symmetries which mean that it is unnecessary (and therefore inefficient) to try all eight. The first character after the ':' can be one of '8','|','\','/', 'X', '-', '+', representing the axes of symmetry.
transformation I - | . \ l r / ABC GHI CBA IHG ADG CFI GDA IFC DEF DEF FED FED BEH BEH HEB HEB GHI ABC IHG CBA CFI ADG IFC GDA a b c d e f g h
Then if the pattern has the following symmetries, the following are true...
| c=a, d=b, f=e, h=g - b=a, c=d, e=f, g=i \ e=a, g=c, f=b, h=d / h=a, f=c, g=b, e=d X a=d=e=h, b=c=f=g + a=b=c=d, e=f=g=h
We can choose to use transformations a,d,f,g as the unique transformations for patterns with either `|' or `\' symmetry.
Thus we choose to order the transformations a,f,d,g,.... and choose first 2 for `X' and `-', the first 4 for `|', `-', `/', and `\', and all 8 for non-symmetrical patterns.
i) An entry in the pattern header states whether the anchor is an X or an O. This helps performance, since all transformations can be rejected at once if the anchor stone does not match. (Ideally, we could just define that the anchor is always O or always X, but some patterns contain no O's and some contain no X's.)
ii) The pattern header contains the size of the pattern (ie the co-ordinates of the top left and bottom right elements) relative to the anchor. This allows the pattern can be rejected quickly if there is not room for the pattern to fit around the anchor stone in a given orientation (ie it is too near the edge of the board). The bounding box information must first be transformed like the elements before it can be tested, and after transforming, we need to work out where the top-left and bottom-right corners are.
iii) The edge constraints are implemented by notionally padding the pattern with rows or columns of '?' until it is exactly 19 elements wide or high. Then the pattern is quickly rejected by (ii) above if it is not at the edge. So the example pattern above is compiled as if it was written
"example" .OO???????????????? *XX???????????????? o?????????????????? :8,80
iv) The elements in a pattern are sorted so that non-space elements are checked before space elements. It is hoped that, for most of the game, more squares are empty, and so the pattern can be more quickly rejected doing it this way.
v) The patterns themselves are sorted by decreasing maximum-weight, which is the maximum value the pattern can take, taking weight and wind assistance into account. For this to work, the weight stored for patterns with helpers must be the maximum which the helper can return. As a hint, to simplify maintenance, the helper can access the stored weight from the pattern structure passed in.
vi) The actual tests are performed using an 'and-compare' sequence. Each board position is a 2-bit quantity. %00 for empty, %01 for O, %10 for X. We can test for an exact match by and-ing with %11 (no-op), then comparing with 0, 1 or 2. The test for 'o' is the same as a test for 'not-X', ie not %10. So and with %01 should give 0 if it matches. Similarly 'x' is a test that bit 0 is not set.
This is a compile time option. By editing the makefile, you can use this faster code to match patterns. The only disadvantage to using this code is that it might be harder to understand and debug.
As described in (vi), the comparisons between pattern and board are performed as 2-bit bitwise operations. Therefore they can be performed in paralled, 16-at-a-time on a 32-bit machine.
Suppose the board is layed out as follows :
.X.O....OO XXXXO..... .X..OOOOOO X.X....... ....X...O.
which is internally stored internally in a 2d array (binary)
00 10 00 01 00 00 00 00 01 01 10 10 10 10 01 00 00 00 00 00 00 10 00 00 01 01 01 01 01 01 10 00 10 00 00 00 00 00 00 00 00 00 00 00 10 00 00 00 01 00
we can compile this to a composite array in which each element stores the state of a 4x4 grid of squares :
???????? ???????? ???????? ... ??001000 00100001 10000100 ??101010 10101010 10101001 ??001000 00100000 10000001 ??001000 00100001 ... ??101010 10101010 ??001000 00100000 ??001000 10001000 ... ??100010 ... ??000000 ???????? ????????
Where '??' is off the board.
We can store these 32-bit composites in a 2d merged-board array, substituting the illegal value %11 for '??'.
Similarly, for each pattern, mkpat produces appropriate 32-bit and-value masks for the pattern elements near the anchor. It is a simple matter to test the pattern with a similar test to (vi) above, but for 32-bits at a time.
GNU Go includes a joseki compiler in patterns/joseki.c. This processes an sgf file (with variations) and produces a sequence of patterns which can then be fed back into mkpat. The joseki database is in files in `patterns/' called `hoshi.sgf', `komoku.sgf', `sansan.sgf', `mokuhadzushi.sgf' and `takamoku.sgf'.
Not every node in the sgf file contributes a pattern. The nodes which contribute patterns have the joseki in the upper right corner, with the boundary marked with an `A' and the value given by a comment.
The joseki compiler is able to generate a constraint line in the
`.db'. The square symbol is a shortcut for oarea()
, the
triangle is xarea()
and the circle is
(!oarea()&&!xarea())
= an empty area.
The delimiter between value and classification in the SGF comment must be `;', for example:
81; D
Spaces and \n
may be omitted.
These features are experimental and are currently not used in the joseki files.
The patterns in `patterns/conn.db' are compiled separately from the other patterns. When a `B' pattern is found, a cutting point is set in the worm data structure and make eye space marginal for the connection inhibiting entries of the pattern. If it is a C pattern, amalgamate the dragons in the pattern.