Enhancing Compiler Optimization with Reinforcement Learning and Monte Carlo Tree Search

Abstract

Reinforcement learning (RL) has shown promising performance in compiler optimization, demonstrating significant capabilities across various frameworks such as AutoPhase and CompilerGym.Despite its great success in compiler optimization tasks, challenges remain in training efficiency and optimization performance.This paper presents a novel approach that integrates reinforcement learning with Monte Carlo Tree Search (MCTS) to improve the optimization performance of the traditional Proximal Policy Optimization (PPO) algorithm.We also employ a lightweight search algorithm to reduce training time while maintaining comparable optimization performance.Experimental results show that our PPO-MCTS-Centor model achieves an average performance improvement of 4% over the traditional PPO algorithm.In terms of training efficiency, our PPO-Light model maintains 98% of the performance while reducing training costs by around 16%.Our approach offers a promising solution to enhance compiler optimization through improved efficiency and effectiveness.