Off-campus WSU users: To download campus access dissertations, please use the following link to log into our proxy server with your WSU access ID and password, then click the "Off-campus Download" button below.
Non-WSU users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Access Type
WSU Access
Date of Award
January 2025
Degree Type
Dissertation
Degree Name
Ph.D.
Department
Industrial and Manufacturing Engineering
First Advisor
Ratna Babu Chinnam
Abstract
Monte-Carlo Tree Search (MCTS) is a versatile and adaptive heuristic tree-search algo-rithm, designed to uncover near-optimal actions by iteratively exploring decision-making points. Through its unique balance of exploration and exploitation, MCTS progressively constructs a search tree by gathering samples during each iteration, ultimately guiding the search towards regions of the decision space that yield higher rewards. This adaptability and efficiency have positioned MCTS as a leading technique within the realm of game theory, where it has demonstrated exceptional success in solving complex decision-making problems such as those found in Go, Chess, and other strategic games. In addition to its notable application in gaming, MCTS has proven its capability to ad- dress broader, more complex problems. Specifically, MCTS has shown significant promise when applied to NP-hard combinatorial optimization problems, which are central to many industrial and research applications. These include problems such as the Job-Shop Schedul- ing Problem (JSSP) and the Weighted Set-Cover Problem (WSCP), both of which require intelligent exploration of vast solution spaces to identify near-optimal solutions within reasonable timeframes. Given the increasing demand for solving larger and more complex problems, MCTS has been extended for distributed-memory parallel platforms, a crucial step in enabling scalability across high-performance computing systems. However, the adoption of MCTS in distributed-memory environments introduces two primary challenges: (1) the consid- erable communication overhead that arises from coordinating parallel processes, and (2) the difficulty in maintaining an even computational load across all processes to avoid bot- tlenecks and inefficiencies. In this work, we introduce a novel distributed-memory parallel MCTS algorithm, termed Parallel Partial-Backpropagation MCTS (PPB-MCTS). The primary innovation of PPB-MCTS lies in its approach to minimizing communication overhead while enhancing performance in combinatorial optimization contexts. Our algorithm leverages a technique known as partial backpropagation, which reduces the frequency and size of data transmitted be- tween processes by sending only essential backpropagation messages, rather than full state information. This minimizes the communication overhead without sacrificing the accuracy of the search. To address the load-balancing challenge, we introduce a shared transposition table, enabling parallel processes to share information regarding explored states. This strategy not only ensures that computational work is distributed more evenly across processes, but also reduces redundant computations, thereby improving the overall efficiency of the algorithm. Moreover, our approach addresses the issue of duplicate states in distributed-memory environments. As is common in sequential MCTS, duplicate states can cause the search tree to evolve into a Directed Acyclic Graph (DAG), complicating the search process. We adapt techniques from sequential MCTS to manage duplicate states effectively in a parallel context, thus preserving the integrity of the search structure and preventing unnecessary computational overhead. We evaluate the effectiveness of PPB-MCTS through an extensive experimental study, focusing on the Job-Shop Scheduling Problem (JSSP) and the Weighted Set-Cover Prob- lem (WSCP). Both problems are well-known for their computational difficulty and are commonly used as benchmarks for evaluating combinatorial optimization algorithms. The experiments are conducted on a large cluster of computers, each equipped with multiple cores, allowing us to fully test the scalability and efficiency of our approach. The empirical results demonstrate that our proposed algorithm significantly outper- forms existing distributed-memory parallel MCTS algorithms, particularly in terms of scal- ability and load balancing. As the number of processes increases, PPB-MCTS maintains high rollout efficiency and improves the distribution of computational load, leading to faster convergence to high-quality solutions. This performance improvement makes PPB- MCTS a valuable tool for solving large-scale NP-hard problems in both research and indus- try.
Recommended Citation
Naderzadeh Ardebili, Yashar, "Scalable Methods For Monte Carlo Tree Search" (2025). Wayne State University Dissertations. 4168.
https://digitalcommons.wayne.edu/oa_dissertations/4168