3.4 Expansion strategies
3.4.2 Drop-out Expansion
The problem with best-first expansion was that in situations as in Figure 3.5 only the best move is considered for expansion, whereas it is not unlikely that an opponent will play the second-best move. To solve this we now consider all moves for expansion, and give each move a priority depending on the depth of the book following the move, and the difference between the best value and the value of the move. A successor has high expansion priority if it is a good move and/or it has a shallow subbook, and a successor has low expansion priority if it is a bad move and/or it has a deep subbook. To calculate the expansion priorities we add two new attributes, epbi and epoi, to the nodes.
epbi =
1 + min(eposj)
for all best successors sj, interior nodes
0 (leaf nodes)
(3.1)
epoi =
1 + min(epbsj +ω(pi−psj))
for all successors sj, interior nodes
0 (leaf nodes)
(3.2)
epbi is the expansion priority for when it is the book player’s move (Equa- tion 3.1). It is initialized to zero in leaf nodes, and depends only on the expansion priority of the best successors. The +1 is the depth penalty. It guarantees that shallow nodes have higher priorities.
epoi is the expansion priority for when it is the opponent’s move (Equa- tion 3.2). It is initialized to zero in leaf nodes, and depends on the expansion priority of all successors. Besides the depth penalty (+1), inferior moves
3.4. EXPANSION STRATEGIES 35 get an additional penalty which depends on the value difference to the best move.
ω ≥0 is the weight for the differencepi−psj between the best value and the value of successor sj. The right choice of ω is game specific and depends on the heuristic value resolution, i.e. +1 may mean “one piece ahead” or
“0.01 pieces ahead”. A low value for ω means higher priority for inferior moves. If ω = 0 then all successors will be expanded to the same depth, regardless of their values. On the other hand, if ω → ∞ then drop-out expansion degenerates into best-first expansion because only best moves will be expanded.
Figure 3.6 uses drop-out diagrams to show a graphical interpretation of the influence of the choice of ω on the expansion strategy. With ω → ∞ we get best-first expansion, and leaf nodes are expanded from left to right.
With ω = 0 we expand the shallowest nodes first, going from bottom to top. Drop-out expansion allows expansion from bottom-left to top-right at an arbitrary angle.
value
depth
drop-out
value
depth
best-first
value
depth
shallowest-first
Figure 3.6: The influence of ω on expansion. For ω → ∞ we get best- first expansion and for ω = 0 we get shallowest-best expansion. Drop-out expansion is the generalized expansion strategy.
Figures 3.7 and 3.8 show two examples of Othello opening books with 10,000 nodes each, calculated with different values of ω.
Figure 3.9 shows the pseudo-code for drop-out expansion. The recursion is started by calling either CalcEpb(nroot) or CalcEpo(nroot), depending on whether the book should be expanded from the first player’s point-of-view or from the second player’s point-of-view. The function Select() selects a node for expansion. The pseudo-code ignores the handling of exact values.
What do we gain if we use drop-out expansion? Obviously a move that is only slightly worse than the best move will not be ignored forever, even if the best value never decreases. With increasing depth of the best move, the priority for the expansion of suboptimal moves will increase too and
0 5 10 15 20
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 value
Figure 3.7: Drop-out diagram of an Othello book with 10,000 nodes calcu- lated with ω= 1.0.
0 5 10 15 20
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
value
Figure 3.8: Drop-out diagram of an Othello book with 10,000 nodes calcu- lated with ω= 2.0.
3.4. EXPANSION STRATEGIES 37
void CalcEpb(Node n) { if (isLeaf(n)) {
Select(n);
} else {
BestValue = BestSuccessorValue(n);
epbmax = +∞;
for (all successors si) { if (si.value == BestValue) {
if (1+si.epo < epbmax) { epbmax = 1+si.epo;
BestMove = i;
}/*if*/
}/*if*/
}/*for*/
CalcEpo(sBestMove);
}/*if*/
}/*CalcEpb*/
void CalcEpo(Node n) { if (isLeaf(n)) {
Select(n);
} else {
BestValue = BestSuccessorValue(n);
epomax = +∞;
for (all successors si) {
if (1+si.epb+ω(BestValue−si.value) < epomax) { epomax = 1+si.epb+ω(BestValue−si.value);
BestMove = i;
}/*if*/
}/*for*/
CalcEpb(sBestMove);
}/*if*/
}/*CalcEpo*/
Figure 3.9: Pseudo-code for drop-out expansion. The function Select() selects a node for expansion. The recursion is started by calling either CalcEpb() orCalcEpo() with the start node.
this will eventually lead to their expansion. For the same reason it will not happen that a suboptimal move gets stuck with a bad value, as was shown in Figure 3.5 for best-first strategy. Thus all the problems observed with best-first expansion have been solved.
An additional benefit from drop-out expansion is that the parameter ω can be used to control the shape of the opening book. The user can choose any shape between full expansion of every line and best-first expansion.
The benefit from drop-out expansion may also be understood as a kind of insurance: if the opponent wants to force a drop-out, he has to pay with a move that is so bad that it has not been considered for expansion yet. If he keeps playing good moves, we will not drop out of the book early. So, at the end of the opening, we have either a good position, or we have saved lots of time, or a combination of both.