Abstract

A microgrid is a small-scale power grid that can operate in either the grid-connected mode or the islanded mode. It consists of energy generation and storage devices for both power and thermal demands, where the thermal energy from the exhaust heat that cannot be transmitted over long distances can be utilized locally. The supply side may include dispatchable generation units with adjustable outputs and energy storage, alongside non-dispatchable renewables that depend entirely on resource availability. On the demand side, thermal loads are satisfied through waste heat recovery from exhaust gas (to maximize overall system efficiency) and direct fuel combustion. Electrical loads are classified as shiftable and non-shiftable. Shiftable appliances are flexible and can be shifted according to the time-of-use electricity prices to reduce the total energy cost.  Managing the daily operation of microgrids is essential to reduce costs and emissions, as well as to avoid possible severe damage to the system or propagating faults on the main grid. The problem is challenging due to the intermittency of renewable generation and the uncertainty of demand, especially at the sub-hourly level and during days with highly variable weather conditions. Also, multi-objective problems yield a set of optimal trade-off solutions instead of a single one, requiring higher computational complexity. Moreover, the interdependency of supply and demand management problems leads to tighter coupling or numerical instability. In this research, to address the above three challenges, the microgrid operation problem is studied in three phases. In phase 1, the photovoltaic (PV) power output is predicted based on weather conditions. Then, in phase 2, operating strategies of energy generation and storage devices are determined based on predicted PV power and demand. In phase 3, the demand-side management (DSM) is incorporated along with the supply-side to further reduce operational costs and emissions by shifting loads with flexible operation.  Phase 1 focuses on variable renewable generation forecasting to predict the PV power. To address the challenge of high forecasting errors in the PV power during variable weather conditions at the sub-hourly level, a multi-layer weather classification-based regression model (MWCR) is developed to predict the PV output power. The key idea is to classify weather conditions at each time step to capture the rapid PV power fluctuations. To address the absence of irradiance sensors, the power ratio of predicted power obtained by a Long-Short Term Memory (LSTM) model over the maximum power calculated by the mathematical model in a clear sky is used to reflect the sky condition. The k-nearest neighbor classification model classifies weather conditions based on the power ratio, local irradiance, temperature, wind speed, and humidity. Then, for each class, a unique regression neural network model with the same inputs is developed. To improve the accuracy of the model, the LSTM model's predictive power is integrated as an additional input to the regression model to develop the enhanced MWCR model (E-MWCR). The testing data is collected at a 5-minute time step from a 2-MW solar farm located at the Rochester Institute of Technology in New York. Prediction results of three datasets at 1-hour and 5-minute time steps demonstrate superior accuracy for MWCR and E-MWCR models compared to a single regression, conventional neural network, gated recurrent unit and LSTM models in all testing datasets, especially on a day with high PV power fluctuations.  Phase 2 focuses on supply-side management to predict operation strategies of generation devices and storage based on the predicted PV power obtained in phase 1. To address the intermittency of renewable generation and the uncertainty of demand at the sub-hourly level, a Managed Q-Learning Long Short-Term Memory (MQLSTM) model is developed to predict the operation strategy by the synergistic integration of LSTM and Q-learning, where the predicted PV data obtained in phase 1 are used as input data. This double-layer model includes an LSTM layer that works in a time series manner and allows the use of historical environmental data of renewables and demand to predict the future states of generation devices and storage. The Q-learning layer predicts the operation strategy to minimize operation cost and emissions based on states predicted by the LSTM. In addition, to mitigate the propagation of forecasting error, a moving window training method is implemented to update the input data at each time step when the actual data is available. Furthermore, to incorporate multiple objectives, the Q-learning model policy is managed to obtain the Pareto Frontier (set of trade-offs between objectives) by predicting a vector of Q-values instead of a single one. The model has been tested on a virtual microgrid consisting of combined cooling, heating and power, heat pump, photovoltaic, battery, and thermal storage systems. The testing dataset consists of the same PV data collected in phase 1, along with 15-minute power and thermal load collected from a large hotel in New York. MQLSTM performance has been compared with two time series models (Gated Recurrent Unit and LSTM), two on-policy reinforcement learning models (peer-to-peer reinforcement learning and state-action-reward-state), and two off-policy reinforcement learning models (twin delayed deep deterministic policy gradient and soft actor critic). Results show that the MQLSTM model outperforms the above by predicting operation strategies at a 15-minute time step closer to strategies obtained by the mathematical optimization using actual PV and demand data.   To incorporate the demand-side management (DSM), phase 3 focuses on supply-demand management to predict operation strategies and load shifting decisions based on the MQLSTM developed in Phase 2. To address the tight coupling and the long training time caused by the interdependencies between supply and demand management, a particle swarm optimization (PSO) based recurrent Q-learning model for predicting operation with load shifting strategies is established. Instead of a double-layer MQLSTM model that requires going through a loop to process the whole input sequence, the LSTM is embedded into the deep Q-learning to estimate the Q-function (instead of the deep neural network) to establish a single-layer Recurrent Q-learning model. The established model predicts the operation strategy at one shot to reduce the training time. Also, the domain knowledge is incorporated into the Q-learning through constraint satisfaction to speed up convergence and improve stability. Furthermore, to handle multiple objectives, PSO is integrated into the Recurrent Q-learning to restrict the training process to non-dominated operating policies by applying Pareto-based selection to PSO Particles defined as the learned policies by the Recurrent Q-learning. The model has been tested on the same virtual microgrid of phase 2 with the same PV data, and the power and thermal demand is collected from a large residential home in New York at a 15-minute time step. Results show that the PSO-based recurrent Q-learning model outperforms four state-of-the-art multi-objective reinforcement learning models, Deep Deterministic Policy Gradient, Twin Delayed Deep Deterministic Policy Gradient, Soft Actor-Critic, and Proximal Policy Optimization in achieving lower costs and emissions compared to existing models. Also, integrating PSO into the recurrent Q-learning shows better results than integrating other methods like scalarization, Genetic Algorithm (GA), and Ant Colony Optimization (ACO) in terms of prediction accuracy at shorter training time.  In this dissertation, the day-ahead sub-hourly operation optimization of microgrids considering the renewable uncertainty, multi-objective, and demand-side management has been investigated. The developed learning-based operation method combines a weather-classification-based forecasting model for renewable generation, a recurrent reinforcement learning model for supply-demand management, and a particle swarm optimization for handling multi-objective operation. This approach can scale to handle interconnected microgrid networks, as its low computational complexity and one-shot decision-making allow it to simultaneously coordinate massive amounts of decentralized data from renewables and demand in real time.

Publication Date

7-2026

Document Type

Dissertation

Student Type

Graduate

Degree Name

Electrical and Computer Engineering (Ph.D)

College

Kate Gleason College of Engineering

Advisor

Bing Yan

Advisor/Committee Member

Abdulla Ismail

Advisor/Committee Member

Katie McConky

Campus

RIT – Main Campus

Download

Share

COinS