在本論文中，我們試圖透過基因規劃法自動化地產生一個適用於爐石戰記 三種牌組類型的出牌策略，對於評估時間過長的問題，我們使用昂貴優化中的 適應值近似法減少了 10% - 20% 的實驗時間。透過實驗結果發現，以基因規劃 法產生的策略在快攻與中速類型的牌組表現較好，先手對上 COG 2019 爐石比賽 第 1 名的動態前瞻策略時，平均勝率為 32%，後手約為 28% 。且在控制類型的 牌組表現較不穩定，可能的原因為是我們使用中速牌組來演化染色體，因此，
此策略較適合使用於中速類型牌組。在分析染色體結構後，我們推測另一個原 因是控制類型牌組的遊戲方式比較不同。
此外，我們將前述策略作為 MCTS 的模擬策略，在對上 COG 2019 爐石比賽 第 1 名的動態前瞻策略時先手勝率約為 41%，後手約為 33% 。相較於使用隨機 模擬策略的 MCTS 爐石 AI 上升約 20% 的勝率，且稍微改善了此策略使用控制 套牌勝率不穩定的缺點。由於 COG 2020 爐石比賽的每回合 30 秒時間限制，
MCTS 的迭代次數與模擬次數也受到一定的限制，在迭代與模擬次數充足的條件 下，我們預測 MCTS 的效能將會再往上提升。
我們希望未來能針對不同類型牌組分別產生最佳策略，可能會有較好的表 現。此外，我們會不斷地改進基因規劃法效能，試著產生一個真正適用於爐石 戰記各類型牌組的通用出牌策略。也希望能將此研究方法應用在其他遊戲，只 使用對遊戲基本的領域知識，就可以透過演化的精髓找出遊戲獲勝的關鍵。
參考文獻
[1] 遊戲王 - 怪獸之決鬥 (Yu-Gi-Oh! Duel Monsters) https://en.wikipedia.org/wiki/
Yu-Gi-Oh!_Duel_Monsters
[2] 魔法風雲會 (Magic: The Gathering) https://magic.wizards.com/zh-hant
[3] 爐石戰記 (Hearthstone) https://playhearthstone.com/zh-tw/
[4] J. R. Koza, “Genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems,” Report No. STANCS-90-1314, Stanford University, Stanford, CA, 1990.
[5] J. R. Koza, “Genetic programming: on the programming of computers by means of natural selection,” MIT Press, Cambridge, MA, 1992.
[6] M. Bhattacharya, “Evolutionary approaches to expensive optimisation,”
International Journal of Advanced Research in Artificial Intelligence, vol. 2, no. 3, pp. 53–59, 2013.
[7] A. Esparcia-Alcazar and J. Moravec, “Fitness approximation for bot evolution in genetic programming,” Soft Computing, vol. 17, no. 8, pp. 1479–1487, 2013.
[8] Unreal Tournament 2004 (UT2004) https://en.wikipedia.org/wiki/Unreal_Tourna-ment_2004
[9] R. Coulom, “Efficient selectivity and backup operators in Monte-Carlo tree search,”
International Conference on Computers and Games, pp. 29–31, 2006.
[10] C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P.
Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, “A survey of Monte-Carlo tree search methods,” IEEE Transactions on Computational Intelligence and AI in Games, vol. 4, no. 1, pp. 1–43, 2012.
[11] P. Auer, N. Cesa-Bianchi, and P.Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine Learning, vol. 47, no. 2, pp. 235–256, 2002.
[12] L. Kocsis and C. Szepesvári, “Bandit based Monte-Carlo planning,” European Conference on Machine Learning, pp. 282–293, 2006.
[13] A. Dockhorn, M. Frick, U. Akkaya, and R. Kruse, “Predicting opponent moves for improving hearthstone AI,” International Conference on Information Processing and Management of Uncertainty in KnowledgeBased Systems, pp. 621–632, 2018.
[14] A. Saffidine, T. Cazenave, and J. Méhat, “UCD: Upper confidence bound for rooted directed acyclic graphs,” Knowledge-Based Systems, vol. 34, pp. 26–33, 2012.
[15] J. S. B. Choe and J. Kim, “Enhancing monte carlo tree search for playing hearthstone,” IEEE Conference on Games, pp. 1–7, 2019.
[16] M. Kearns, Y. Mansour, and A. Y. Ng, “A sparse sampling algorithm for near-optimal planning in large Markov decision processes,” International Joint Conference on Artificial Intelligence, pp. 1224–1231, 1999.
[17] S. Zhang and M. Buro, “Improving hearthstone AI by learning high-level rollout policies and bucketing chance node events,” IEEE Conference on Computational Intelligence and Games, pp. 309–316, 2017.
[18] M. Swiechowski, T. Tajmajer, and A. Janusz, “Improving hearthstone AI by combining mcts and supervised learning algorithms,” IEEE Conference on Computational Intelligence and Games, pp. 1–8, 2018.
[19] A. Santos, P. A. Santos, and F. S. Melo, “Monte-Carlo tree search experiments in hearthstone,” IEEE Conference on Computational Intelligence and Games, pp.
272–279, 2017.
[20] P. García-Sánchez, A. Tonda, A. J. Fernández-Leiva, and C. Cotta, “Optimizing Hearthstone agents using an evolutionary algorithm,” Knowledge-Based Systems, 2019.
[21] M. A. Potter and K. A. De Jong, “Cooperative coevolution: An architecture for evolving coadapted subcomponents,” Evolutionary Computation, vol. 8, no. 1, pp.
1–29, 2000.
[22] CIG Hearthstone 2018, COG Hearthstone 2019, and 2020.
https://dockhorn.antares.uberspace.de/wordpress/
[23] C. D. Ward and P. I. Cowling, “Monte-Carlo search applied to card selection in magic: the gathering,” IEEE Symposium on Computational Intelligence in Games, pp. 9–16, 2009.
[24] P. I. Cowling, C. D. Ward, and E. J. Powley, “Ensemble determinization in monte carlo tree search for the imperfect information card game magic: The gathering,”
IEEE Transactions on Computational Intelligence and AI in Games, vol. 4, no. 4, pp. 241–257, 2012.
[25] B. E. Childs, J. H. Brodeur, and L. Kocsis, “Transpositions and move groups in Monte-Carlo tree search,” IEEE Symposium on Computational Intelligence and Games, Perth, Australia, pp. 389–395, 2008.
[26] P. García-Sánchez, A. Tonda, G. Squillero, A. Mora, and J. Merelo, “Evolutionary deckbuilding in hearthstone,” IEEE Conference on Computational Intelligence and Games, pp. 1–8, 2016.
[27] P. García-Sánchez, A. Tonda, A. M. Mora, G. Squillero, and J. J. Merelo,
“Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone,” Knowledge-Based Systems, vol. 153, pp. 133–146, 2018.
[28] SabberStone, https://github.com/HearthSim/SabberStone [29] L. Piotrowski, S. Mittenentzwei, and T. Heimbrodt,
https://dockhorn.antares.uberspace.de/wordpress/bot-downloads/
[30] E. Håkonsen, 2018, Available:
https://www.reddit.com/r/hearthstone/comments/7l1ob0/i_wrote_a_masters_thesi s_on_effective_hearthstone/
[31] A. M. Alhejali and S. M. Lucas, “Using genetic programming to evolve heuristics for a Monte-Carlo tree search Ms Pac-Man agent,” IEEE Conference on Computational Intelligence and Games, pp. 65–72, 2013.
[32] D.E. Knuth and R.W. Moore, “An analysis of alpha-beta pruning.” Artificial Intelligence, vol. 6, no. 4, pp. 293–326, 1975.
[33] A. Benbassat and M. Sipper, “EvoMCTS: Enhancing MCTSbased Players Through Genetic Programming.” IEEE Conference on Computational Intelligence and Games, pp. 57–64, 2013.