Artificial intelligence, fueled by machine learning and deep learning techniques, is revolutionizing various domains. Reinforcement learning (RL) stands out as a potent method for training agents to navigate complex environments and make informed decisions. Our focus is on applying RL techniques, specifically Convolutional Neural Networks (CNNs) combined with policy gradient methods, to enhance the gameplay experience of Khmer chess. Our goal is to surpass the performance of traditional chess engines. The system employs deep neural networks to train AI agents, enabling self-play iterations for strategy refinement. Specifically, we utilize RL technology to iteratively enhance game strategies based on self-matching results, ultimately improving the system's chess proficiency. Our approach entails developing a CNN-based RL system tailored for Khmer chess, encompassing strategies, value evaluation mechanisms, and rule adaptations specific to the game. We utilize deep neural networks to facilitate agent training through self-play iterations, leveraging RL techniques for continual strategy refinement. To enhance training efficiency, we introduce a segmentation method for Khmer chess stages, optimizing the neural network's learning process by mapping game situations to optimal actions based on cumulative rewards. Furthermore, we integrate RL principles to guide action selection towards maximizing reward values, employing Deep Q-Learning with policy gradient for optimal decision-making. With the experimental validation demonstrates the efficacy of our CNN-based RL system in enhancing Khmer chess gameplay. The system exhibits self-improvement, adaptability, and human-like gameplay characteristics, enriching player experience and entertainment value. Moreover, the proposed approach showcases improved training efficiency compared to conventional RL-based chess systems, highlighting its efficacy and scalability for AI-driven game enhancements.