In this thesis, we study to solve games and improve search performances with embedded combinatorial game knowledge. For this study, we investigate three combinatorial games, including Triangular Nim, XT Domineering and NoGo.
First, solving nine layer Triangular Nim has the following contributions. Using the retrograde methods, this thesis strongly solves 9 layer Triangular Nim. This thesis also proposes some methods to improve the performance, such as designing data structures in blocks, using the retrograde method, removing redundancy and selecting the block representation with the less number of inter-block updates. Especially, by removing redundancy, we reduce the memory by a factor of 5.72 and the computation time by a factor of 4.62.
Our experiment result also shows that the ratio of the number of P-positions to that of N-positions is low, 5.0% for 9 layer Triangular Nim. Due to the low ratio, the retrograde method does perform well when compared with the traditional forward checking.
Second, XT domineering has the following three major contributions. We present a new game, XT Domineering, which has higher game-tree complexity than Domineering.
We also have presented a mathematical approach to solve sums of 3×3 XT Domineering.
Again, this success demonstrates the potential of applying CGT to solving more of other intelligent games.
Third, NoGo endgame analysis has the following three major contributions. We calculate the game values of many NoGo positions with endgame search depth 6 and 8 in brute force search from empty 4×4 board. In NoGo 4×4 board, the game values include
integers such as 2, 1, 0, -1 and -2,and infinitesimals including star, ups and downs such as
*, ↑, ↑*, ↑↑, ↑↑*, and ↑2. In experiments, we find the maximum of temperature is 2 among all the above 4×4 board positions and the temperature is low in most of these positions.
Furthermore, we present three propositions to help understand the characteristics of NoGo game.
In this thesis, we use algebra characteristics of CGT to help solve and reduce the complexity of three combinatorial games. Based on the theory, playing or solving these combinatorial games may simply become mathematical calculations, such as summation, instead of a complex tree search. CGT also helps cut some unnecessary branches in search tree in some combinatorial games, e.g. NoGo.
References
[1] M. H. Albert, J. P. Grossman, R. J. Nowakowski and D. Wolfe, “An introduction to Clobber,” Integers, Electr. J of Combinat. Number Theory vol. 5, no. 2, #A01, 2005.
[2] M. H. Albert, R. J. Nowakowski, D. Wolfe, Lessons in Play: An Introduction to the Combinatorial Game Theory, A.K. Peters, Wellesley (2007)
[3] S.-Q. Bai, S.-S. Lin, “On the Study of 8 Layer Triangular Nim,” (in Chinese) National Computer Symposium (NCS 2009), Taipei, Taiwan, 2009.
[4] R. Balla, A. Fern, UCT for tactical assault planning in real-time strategy games, in: 21st International Joint Conference on Artificial Intelligence, pp. 40–45.
[5] E. R. Berlekamp, “Blockbusting and Domineering,” Journal of Combinatorial Theory Ser.
A, vol. 49, pp. 67-116, 1988.
[6] E. R. Berlekamp, D. Wolfe, Mathematical Go: Chilling Gets the Last Point, A K Peters, Wellesley, MA, 1994.
[7] E. R. Berlekamp, J. H. Conway, and R. K. Guy, Winning Ways for your Mathematical Plays, 1st edition, 2 vols, New York: Academic Press, 1982.
[8] C. L. Bouton, “Nim, A Game with a Complete Mathematical Theory,” the Annals of Mathematics, 2nd Ser., Vol.3, 1/4. (1901-1902), pp.35-39.
[9] D. M. Breuker, J. W. H. M. Uiterwijk and H. J. van den Herik, “Solving 8×8 Domineering,” Theoretical Computer Science, Vol. 230, pp. 195-206, 2000.
[10] B. Brian, A Mathematical Pandora’s Box, Cambridge Univ Pr, 1993.
[11] C. Browne, E. Powley, D. Whitehouse, S. Lucas, P. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, S. Colton, “A survey of Monte Carlo Tree Search,” IEEE Transactions On Computational Intelligence and AI in Games, Vol. 4, No. 1, March 2012.
[12] N. Bullock, “Domineering: Solving Large Combinatorial Search Spaces,” ICGA Journal, vol. 25, no. 2, pp. 67-84, 2002.
[13] A. K. Chandra, D. C. Kozen, and L. J. Stockmeyer. Alternation. J. ACM, 28(1):114–133, 1981.
[14] C.-Y. Chen. S.-S. Lin, On the Study and Improvement of 8 Layer and 9 Layer Triangular Nim, Master Thesis, National Taiwan Normal University, 2010.
[15] C.-W. Chou, O. Teytaud, S.-J. Yen, "Revisiting Monte-Carlo tree search on a normal form game: NoGo," in: The main European events on Evolutionary Computation (Evo⁄ 2011), Torino, Italy, April 27–29, 2011.
[16] A. Cincotti, “Three-Player Domineering,” Proceedings of World Academy of Science, Engineering, and Technology 36:92-95,December 2008.
[17] A. Cincotti, “Further Results on Three-Player Domineering,” Proceedings of World Academy of Science, Engineering and Technology 51:187-189, 2009.
[18] Computer game tournaments in the 2012 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2012),
http://idb.csie.ncku.edu.tw/taai2012conference/index.php/awards/tournaments.
[19] Computer Olympiad 2013, in Japan, Yokohama , http://icga.uvt.nl/?page_id=627.
[20] J. H. Conway, On Numbers and Games, New York: Academic Press, 1976.
[21] R. Coulom, “Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search”, in:
5th International Conference on Computer and Games, pp. 72–83.
[22] E. D. Demaine, “Playing games with algorithms: algorithmic combinatorial game theory,”
in: Mathematical Foundations of Computer Science (Mariánské Lázn˘e), Lecture Notes in Comput. Sci. 2136, Springer, Berlin (2001), pp. 18–32.
[23] N. D. Elkies. On numbers and endgames: combinatorial game theory in chess endgames.
In R. J. Nowakowski, editor, Games of No Chance, Proc. MSRI Workshop on Combinatorial Games, July, 1994, Berkeley, CA, MSRI Publ., volume 29, pages 135-150.
Cambridge University Press, Cambridge, 1996.
[24] T. S. Ferguson, Game Theory, lecture notes, UCLA 2008.
[25] H. Finnsson, Y. Björnsson, “Simulation-based approach to general game playing,” in:
23rd Conference on Artificial Intelligence, pp. 259–264.
[26] R. Fleischer and S. U. Khan. Xinagqi and combinatorial game theory. Technical Report HKUST-TCSC-2002-01, Hong Kong University of Science and Technology, February 2002.
[27] M. Gardner, “Mathematical games,” Scientific American, Vol. 230, pp. 106-108, 1974.
[28] S. Gelly, “A Contribution to Reinforcement Learning; Application to Computer Go”, Ph.D.
Thesis, University of South Paris, 2007.
[29] S. Gelly, D. Silver, “Achieving master level play in 9 x 9 computer Go,” in: 23rd Conference on Artificial Intelligence, 2008, pp. 1537–1540.
[30] S. Gelly, D. Silver, “Monte-Carlo Tree Search and Rapid Action Value Estimation in Computer Go,” Artificial Intelligence, Volume 175, Issue 11, pp. 1856-1875, 2011.
[31] P. M. Grundy, “Mathematics and games,” Eureka 2, pp.6-8, 1939, Reprint, Eureka 27, pp.
9-11, 1964.
[32] O. Hanner, (1959). “Mean Play for Sums of Positional Games,” Pacific J. Math. 9, pp.
81-99.
[33] S.-C. Hsu, “Solving the Problem of 7 Layer of Triangular Nim,” (in Chinese) National Computer Symposium (NCS 1985), pp.798-802, Dec. 1985.
[34] T.-S. Hsu. and P.-Y. Liu, "Verification of Endgame Databases," International Computer Game Association (ICGA) Journal, volume 25, number 3, pp. 132-144, 2002.
[35] H. J. van den Herik, J.W.H.M. Uiterwijk and J.V. Rijswijck, “Games solved: Now and in the future,” Artificial Intelligence, vol. 134 (1-2), pp. 277–311, 2002.
[36] K.-Y. Kao, “On Hot and Tepid Combinatorial Games,” Ph.D. Thesis, UNC Charlotte 1997.
[37] K.-Y. Kao, (1998). “Mean and Temperature Search for Combinatorial Games,” JCIS Proceedings, Vol I, pp. 389-392.
[38] K.-Y. Kao, “The game of un-impartial Kayles,” Proceedings of the 25th Workshop on Combinatorial Mathematics and Computation Theory, Chung Hua University, Hsinchu Hsien, Taiwan, April 25-26, 2008, pp. 151-153.
[39] K. Y. Kao, “Sumbers – Sums of Ups and Downs,” Integers, E. Journal of Combinatorial Number Theory, Vol. 2005.
[40] K.-Y. Kao, I-C. Wu, Y.-C. Shan and H.-H. Lin, “Chilled Domineering,” International Conference on Technologies and Applications of Artificial Intelligence (TAAI 2010), pp.427-432.
[41] K.-Y. Kao, I.-C. Wu, S.-J. Yen and Y.-C. Shan, "Incentive Learning in Monte Carlo Tree Search," IEEE Transactions on Computational Intelligence and AI in Games (IEEE
TCIAIG), Feb 2013.
[42] K.-Y. Kao, I.-C. Wu, Y.-C. Shan, and S.-J. Yen), "Selection Search for Mean and Temperature of Multi-Branch Combinatorial Games," ICGA Journal, Vol. 35, No. 3, pp.
157-176, Sep. 2012.
[43] K.-Y. Kao, I.-C. Wu, and Y.-C. Shan, "XT Domineering: A New Combinatorial Game,"
Knowledge-Based Systems, Volume 34, pp. 55-63, October 2012.
[44] M. Lachmann, C. Moore, I. Rapaport, “Who Wins Domineering on Rectangular Boards,”
in More Games of No Chance, R.J. Nowakowski, Ed. Cambridge University Press, 2002, pp. 307-315.
[45] C.-S. Lee, M.-H. Wang, Y.-J. Chen, H. Hagras, M.-J. Wua, O. Teytaud, "Genetic fuzzy markup language for game of NoGo," Knowledge-Based Systems, Volume 34, pp. 64-80, October 2012.
[46] L. Lew, and Coulom, R. (2010). “Simulation-based Search of Combinatorial Games,” in ICML Workshop on Machine Learning and Games, Israel.
[47] H.-H. Lin, I-C. Wu and Y.-C. Shan, "Solving Eight Layer Triangular Nim," (in Chinese) National Computer Symposium (NCS 2009), Taipei, Taiwan, November 2009.
[48] R. Lorentz, “Amazons discover Monte-Carlo,” in: 6th International Conference on Computers and Games, pp. 13–24.
[49] F. Mäser, Global threats in combinatorial games: a computational model with applications to chess endgames, in: More Games of No Chance, Proc. MSRI Workshop on Combinatorial Games, July, 2000, Berkeley, CA, MSRI Publ. (R. J. Nowakowski, ed.), Vol. 42, Cambridge University Press, Cambridge, pp. 137-149.
[50] S. K. McCurdy and R. Nowakowski, “Cutthroat, an all-small game on graphs,” Integers, Electr. J of Combinat. Number Theory vol. 5, no. 2, #A13, 2005.
[51] G. A. Mesdal, “Partzan Splittles,” Games of No Chance 3, Cambridge University Press, 2009, pp 447-461.
[52] J. Milnor, (1953). “Sums of Positional Games,” in Kuhn and Tucker (eds.) “Contributions to the Theory of Games,” Ann. Math. Studies #28, Princeton, pp. 291-301.
[53] M. Müller, “Computer Go,” Artificial Intelligence, Vol.134, No.1-2 (Special issue on Games, Computers and AI), January 2002, pp145-179.
[54] M. Müller, Enzenberger, M., and Schaeffer, J., (2004). “Temperature discovery search,” In AAAI, pp. 658–663, San Jose, CA.
[55] M. Müller, NoGo History and Competitions,
http://webdocs.cs.ualberta.ca/~mmueller/nogo/history.html
[56] J. Schäfer, “The UCT algorithm applied to games with imperfect information,” Diploma Thesis. Otto-von-Guericke-Universit¨at Magdeburg, 2008.
[57] Y.-C. Shan, I.-C. Wu, H.-H. Lin, and K.-Y. Kao, "Solving Nine Layer Triangular Nim,"
Journal of Information Science and Engineering, vol.28, No.1, pp.99-113, January, 2012.
[58] C. E. Shannon. "Programming a Computer for Playing Chess," in Philosophical Magazine, 7th series, 41, NO. 314 (March 1950): 256-75.
[59] A. N. Siegel, CGSuite, A java based toolkit for evaluating games, http://www.cgsuite.org/
[60] R. P. Sprague, "Ü ber mathematische Kampfspiele," Tohoku Mathematical Journal 41, pp.
438-444, 1936.
[61] N. Sturtevant, “An analysis of UCT in multi-player games,” in: 6th International Conference on Computers and Games, pp. 37–49, 2008.
[62] F. Teytaud, O. Teytaud, “Creating an Upper Confidence Tree program for Havannah,” in:
12th Advances in Computer Games Conference, pp. 65–74, 2010.
[63] K. Thompson. “Retrograde analysis of certain endgames,” ICCA Journal, Vol. 9, No. 3, pp.131-139, 1986.
[64] K. Thompson. “6-Piece Endgames,” ICCA Journal, Vol. 19, No. 4, pp. 215-226, 1996.
[65] A. Tucker, Applied Combinatiorics, 3rd edition, New York: Wiley, pp396-403, 1994.
[66] M. Winands, Y. Björnsson, “Evaluation function based Monte-Carlo LOA,” in: 12th Advances in Computer Games Conference, pp. 33–44.
[67] I-C. Wu, C.-P. Chen, P.-H. Lin, G.-Z. Huang, L.-P. Chen, D.-J. Sun, Y.-C. Chan, and H.-Y.
Tsou, "A Volunteer-Computing-Based Grid Environment for Connect6 Applications," The 12th IEEE International Conference on Computational Science and Engineering (CSE-09), August 29-31, Vancouver, Canada, 2009.
[68] I.-C. Wu, Y.-C. Shan, C.-H. Lin and S.-J. Yen, "LongCATMJ Wins Mahjong Tournament in TCGA 2011," ICGA Journal, vol. 34, no. 3, 2011, pp. 166–167.
[69] I.-C. Wu, C.-H. Lin, Y.-C. Shan, "Tournament Framework for Computer Mahjong Competitions," The International Workshop on Computer Games (IWCG 2011), Chunli, Taiwan, November 2011.
[70] P.-H. Wu, P.-Y. Liu and T.-S. Hsu, "An External-Memory Retrograde Analysis Algorithm," Proc. 4th International Conference on Computers and Games (CG), Springer-Verlag LNCS# 3846, pp. 145-160, 2004.
Appendix A The Derivations for XT Domineering Games Values of Positions
The power of using combinatorial theory is to derive the game value (or result) without tree search as many board games do. This is well described in many articles such as [1][50].
In this appendix, a simple Domineering example with C and E in Figure 18 as well as a XT Domineering example is illustrated to demonstrate the power of using combinatorial theory.
(a)
(b)
Figure 35. Deriving both game values of C and E of Figure 18 in (a) and (b) respectively.
{ | 0 } = – { 0 | } = –1
{ | }= 0
{ 0, –1 | 1 }= { 0 | 1} = 1/2
{ 0 | }= 1 { | }= 0 { | 0 }= –1
{ | }= 0 { | }= 0
First, let us investigate the game of Domineering. The game value of C, –1, is derived in Figure 35 (a). The negative game value indicates that Right wins the game. The game value of E, 1/2, is derived in Figure 35 (b). The positive value indicates that Left wins the game. In the derivation, a cross is used to indicate that Left does not choose –1 since choosing 0 is better to Left.
Figure 36. Deriving the game values of C + E.
If both C and E are left in a game, we can derive the game value, –1+1/2 = –1/2, by using the combinatorial theory, and easily conclude that Right wins the game due to the negative game value. However, in case of using tree search, we need to derive the same game value as shown in Figure 36, whose computational complexity grows exponentially as more are added.
(a)
(b)
Figure 37. Deriving both game values of C and E of Figure 18 in (a) and (b) respectively.
Now, let us investigate the game of XT Domineering. As described in Section 3.3, the game becomes more complex since 1×1 dominos are also allowed to be placed. For both games C and E, the derivations for both are shown in Figure 37 (a) and (b), respectively.
The game value of C, ↓ (a negative infinitesimal), indicates that Right still wins the game, {*,* | 0} = {* | 0 } = ↓
while the game value of E, ↑↑*, indicates that Left wins the game. The derivations for both are clearly much more complex, when compared with Figure 35.
Figure 38. Deriving the game values of C + E.
For simplicity, we choose the game * + E, as shown in Figure 38. Its game value is ↑↑
with atomic weight 2, which indicates that Left wins the game, as described in 3.2.4.
+
Appendix B The proof for XT Domineering inequalities
Proposition: The ups in the grids of Table 7 are the sufficient and necessary conditions
for
grid(i, j) + row(i) + col(j) > 0.
Proof:
Let (Gi,j) denote the inequality grid(i, j) + row(i) + col(j) > 0.
We first show the sufficiency of the conditions.
Since ↑ > ↑/2 > ↑2 > 0, we have (G3,1), ↑2 > 0 and ↑/2－↑2 > 0.
Since */2 + △* + ↑/2 > 0, we have (G7,1).
Since */2 + ↑ － ↑2 > 0 and */2 + ↑/2+2.↑2 > 0, we have (G8,1).
Since */2 + ↑/2 + ↑2 > ★, we have (G8,2).
Since △* + △*> *, we have (G1,3), and (G7,1) => (G5,3),
(G8,1) => (G6,3), and (G8,2) => (G6,4).
Since */2 + ↑/2 + ↑ + ↑2 > ★+*, we have (G8,4).
(G8,2) and (G8,4) => (G9,2) and (G9,4).
Since ↑2 > ★ and ↑ + ↑2 > ★ + *, we have (G3,2) and (G3,4).
(G3,2) and (G3,4) => (G4,2) and (G4,4).
Since ↑2 > ★, we have (G3,2), and (G4,2) => (G4,1),
(G9,2) => (G9,1), (G1,3) => (G1,4), (G4,4) => (G4,3), and (G9,4) => (G9,3).
Since △*> 0, we have (G2,1), and (G2,1) => (G1,1)
(G7,1) => (G6,1) => (G5,1), (G3,2) => (G2,2) => (G1,2),
(G8,2) => (G7,2) => (G6,2) => (G5,2), (G4,3) => (G3,3) => (G2,3),
(G9,3) => (G8,3) => (G7,3), (G3,4) => (G2,4),
(G6,4) => (G5,4), and (G8,4) => (G7,4).
This completes proof for the sufficiency of the conditions.
Next, we prove the necessary of the conditions. We need to show that any sums of ups less than or confused with the value in a corresponding grid will result in an insufficient condition. Note that the smallest increments of sums ups are ↑2 and ↑/2 – ↑2, and the only possible sums of ups confusing with 0 are ↑/2 – (n + 1).↑2, n > 0.
For (G3,1), (G6,3) and (G8,1), we only need to show that if the value in the corresponding grid reduced by ↑2, then the inequality will not hold. For all the other grids, in order to prove the necessary conditions, we need to show that if the value in a grid reduced by ↑2 or
↑/2 – ↑2, or, if the value in a grid increased or reduced by ↑/2 – (n + 1).↑2, n > 0, then the corresponding inequality will not hold. Since ↑/2 – 2.↑2 ≧ ↑/2 – (n + 1).↑2 > –(↑/2 – ↑2) > – (↑/2 – (n + 1).↑2), it is sufficient to show: if the value in a grid reduced by ↑2 or increase by
↑/2 – 2.↑2 then the corresponding inequality will not hold.
Consider (G3,1), ↑2 > 0 and ↑/2 – ↑2 > 0.
But 0 ≯ 0 and ↑/2 – 2.↑2 ≯ 0.
Thus ↑2 or ↑/2 – ↑2 is a necessary condition.
Consider (G6,3),
*/2 + 2.△* + ↑/2 + 2.↑2 > * and */2 + 2.△* +↑ – ↑2 > *.
But */2 + 2.△* + ↑/2 + ↑2 ≯ * and */2 + 2.△* + ↑ – 2.↑2 ≯ *.
Thus ↑/2 + 2.↑2 or ↑ – ↑2 is a necessary condition.
Note that, since 2.△* > *, the necessary condition of (G6,3) implies the necessary condition of (G8,1).
Consider (G1,2), (n + 1).△* + ↑2 > ★.
But (n + 1).△* ≯ ★ and (n + 1).△* + ↑/2 – ↑2 ≯ ★ Thus ↑2 is a necessary condition.
Consider (G1,4), (n + 1).△* + ↑2> ★ + *.
But (n + 1).△* ≯ ★ + * and (n + 1).△* + ↑/2 – ↑2 ≯ ★ + * Thus ↑2 is a necessary condition.
Consider (G2,3), △* + ↑ + 2.↑2 > *.
But △* + ↑ + ↑2 ≯ * and △* + ↑ + ↑/2≯ * Thus ↑ + 2.↑2 is a necessary condition.
Consider (G4,1), –n.△* + ↑ + 2.↑2 > 0.
But –n.△* + ↑ + ↑2 ≯ 0 and –n.△* + ↑ + ↑/2≯ 0 Thus ↑ + 2.↑2 is a necessary condition.
Consider (G5,2), */2 + n.△* + ↑/2 + ↑2 > ★.
But */2 + n.△* + ↑/2 ≯ ★ and */2 + n.△* + ↑ – ↑2 ≯ ★ Thus ↑/2 + ↑2 is a necessary condition.
Consider (G5,4), */2 + (n + 2).△* + ↑/2 + ↑2 > ★ + *.
But */2 + (n + 2).△* + ↑/2 ≯★ + * and
*/2 + (n + 2).△* + ↑ – ↑2 ≯ ★ + *.
Thus ↑/2 + ↑2 is a necessary condition.
Consider (G7,3), */2 + △* + ↑/2 + ↑ + 2.↑2 > *.
But */2 + △* + ↑/2 + ↑ + ↑2≯ * and */2 + △* + 2.↑ ≯ *.
Thus ↑/2 + ↑ + 2.↑2 is a necessary condition.
Consider (G9,1), */2 – n.△* + ↑/2 + ↑ + 2.↑2 > 0.
But */2 – n.△* + ↑/2 + ↑ + ↑2 ≯ 0 and */2 –n.△* + 2.↑≯ 0.
Thus ↑/2 + ↑ + 2.↑2 is a necessary condition.
Let (Gi,j)* denote the inequalities grid(i, j) + row(i) + col(j) – ↑2 ≯ 0, and grid(i, j) + row(i) + col(j) + ↑/2 – 2.↑2 ≯ 0
Since ↑2 > ★, we have (G1,2) * => (G1,1) *,
(G1,4) * => (G1,3) *, (G2,3) * => (G2,4) *, (G4,1) * => (G4,2) *, (G5,2) * => (G5,1) *, (G5,4) * => (G5,3) *, (G7,3) * => (G7,4) *, and (G9,1) * => (G9,2) *.
Since △*> 0, we have (G1,1) * => (G2,1) *,
(G5,1) * => (G6,1) * => (G7,1) *, (G1,2) * => (G2,2) * => (G3,2) *,
(G5,2) * => (G6,2) * => (G7,2) * => (G8,2) *, (G2,3) * => (G3,3) * => (G4,3) *,
(G7,3) * => (G8,3) * => (G9,3) *, (G2,4) * => (G3,4) * => (G4,4) *, (G5,4) * => (G6,4) *, and (G7,4) * => (G8,4) * => (G9,4) *.
This completes the proof for the necessary of the conditions.
Vita
Yi-Chang Shan was born in Taipei, Taiwan in 1969. He received the B.S.
and M.S. degrees in Computer Science from National Taiwan Normal University, in 1991 and 2002, respectively, and Ph.D. degree in Institute of Computer Science, National Chiao Tung University, Hsinchu, Taiwan, in 2013.
His research interests include computer game, combinatorial game theory, and cloud computing.