Transformer network for data imputation in electricity demand data
Ashkan Lotfipoor, Sandhya Patidar, David Jenkins
Load forecasting necessitates a significant amount of smart meter data. Several elements in this process, including device malfunctions and signal transmission issues, produce missing data gaps. Missing values in the dataset significantly influence the learning ability of machine learning algorithms, and they must be infilled before proceeding with any statistical analysis. This paper investigates the handling of missing values in demand data, and a new approach is developed for improving the performance of demand analytics, such as energy forecasting. The proposed model uses a transformer neural network to impute the missing values at various rates in the demand profile. Our model uses a k-means algorithm to fill in the missing values with proxy values in the dataset. The model is applied to two case-study residential house located in Cornwall and Fintry, United Kingdom. The developed…