ICDVAE: An Effective Self-supervised Reinforcement Learning Method for behavior decisions of Non-player Characters in Metaverse Games
Fangze Lin, Ning Wei, Haoyu Zhai
Reinforcement Learning (RL) is widely recognized as a powerful approach to constructing intelligent NPCs. Unlike general games, metaverse games tend to build up the larger, more complex, and more diverse virtual scenes with scores of various NPCs. In this case, it is challenging and hig-cost to design the reward functions for large-scale RL-based NPCs. To solve this challenge, we propose an effective distribution-aware curiosity mechanism termed Intrinsic Curiosity-Driven Variational Auto-Encoder (ICDVAE). It constructs the state distribution to rep-resent novelty by integrating the Intrinsic Curiosity Mechanism (ICM) and Generative Model (GM). In addition, we develop two effective techniques termed Inverse Dynamic Mechanism (IDM) and Variational Encoder Bottleneck (VEB) to extract the ego-motion feature and prevent overfitting. The empirical results show that our algorithm is more…