Abstract
This thesis investigates the application of active inference framework on different reinforcement learning (RL) tasks. We specifically consider the following OpenAI Gym environments: CartPole, MountainCar, and LunarLander. In this thesis, our primary goal is to explore whether active inference could provide better performance and stability compared to traditional RL methods. The proposed model consisted of three main components: a variational autoencoder (VAE) model to infer hidden states, a transition model predicting latent states, and a Double deep Q-network (Double DQN) as the actor selecting optimal actions. To achieve this, extensive experiments are carried out using grid searches across several hyperparameters, including learning rate, discount factor gamma, KL-divergence weights and soft update factor tau. Models achieving stable and rapid convergence across multiple trials were selected as optimal. Custom reward shaping techniques were implemented for more challenging environments such as MountainCar and LunarLander. The experimental results demonstrated that while the active inference agent successfully achieved the desired performance thresholds in each environment, its performance was not stable, often increasing early before subsequently decreasing. This behavior suggested issues related to catastrophic forgetting where the agent might implicitly treat different state regions as separate tasks, continuously overwriting previously beneficial policy parameters. Elastic weight consolidation (EWC) was explored to solve the instability issue. However, incorporating EWC yielded limited improvement, suggesting that the instability could originate from factors beyond traditional catastrophic forgetting. These results indicate that active inference, combined with Double DQN, is capable of effectively solving standard RL tasks. However, challenges remain in terms of policy stability. Therefore, it is important to conduct further research to understand and overcome these instabilities as it potentially deliver great utilities to solving more complex tasks with active inference.
Library of Congress Subject Headings
Reinforcement learning; Artificial intelligence--Data processing; Inference; Active learning
Publication Date
8-2025
Document Type
Thesis
Student Type
Graduate
College
Golisano College of Computing and Information Sciences
Advisor
None provided
Recommended Citation
Hu, Xinyu, "An Investigation of Active Inference for Reinforcement Learning Control" (2025). Thesis. Rochester Institute of Technology. Accessed from
https://repository.rit.edu/theses/12294
Campus
RIT – Main Campus
Plan Codes
COMPSCI-MS
