Author

Luchen Li

8 papers · 2018–2026

2018peak 3 in 20262026

Frequent co-authors

Papers

showing 1–8 of 8 RSS

The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making

Luchen Li, Matthieu Komorowski, Aldo A. Faisal

cs.AI · May 2018 · cited 21× · arXiv ↗ · more like this ·

Off-policy reinforcement learning enables near-optimal policy from suboptimal experience, thereby provisions opportunity for artificial intelligence applications in healthcare. Previous works have mainly framed patient-clinician interactions as Markov decision processes, while true physiological states are not necessarily fully observable from clinical data. We capture this situation with partially observable Markov decision process, in which an agent optimises its actions in a belief represented as a distribution of patient states inferred from individual history trajectories. A Gaussian mixture model is fitted for the observed data. Moreover, we take into account the fact that nuance in pharmaceutical dosage could presumably result in significantly different effect by modelling a continuous policy through a Gaussian approximator directly in the policy space, i.e. the actor. To address…

Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPs

Luchen Li, Matthieu Komorowski, Aldo A. Faisal

cs.AI · 17 May 2019 · cited 9× · arXiv ↗ · more like this ·

Health-related data is noisy and stochastic in implying the true physiological states of patients, limiting information contained in single-moment observations for sequential clinical decision making. We model patient-clinician interactions as partially observable Markov decision processes (POMDPs) and optimize sequential treatment based on belief states inferred from history sequence. To facilitate inference, we build a variational generative model and boost state representation with a recurrent neural network (RNN), incorporating an auxiliary loss from sequence auto-encoding. Meanwhile, we optimize a continuous policy of drug levels with an actor-critic method where policy gradients are obtained from a stablized off-policy estimate of advantage function, with the value of belief state backed up by parallel best-first suffix trees. We exploit our methodology in optimizing dosages of…

Bayesian Distributional Policy Gradients

Luchen Li, A. Aldo Faisal

cs.LG · 23 Mar 2021 · cited 7× · arXiv ↗ · more like this ·

Distributional Reinforcement Learning (RL) maintains the entire probability distribution of the reward-to-go, i.e. the return, providing more learning signals that account for the uncertainty associated with policy performance, which may be beneficial for trading off exploration and exploitation and policy learning in general. Previous works in distributional RL focused mainly on computing the state-action-return distributions, here we model the state-return distributions. This enables us to translate successful conventional RL algorithms that are based on state values into distributional RL. We formulate the distributional Bellman operation as an inference-based auto-encoding process that minimises Wasserstein metrics between target/model return distributions. The proposed algorithm, BDPG (Bayesian Distributional Policy Gradients), uses adversarial training in joint-contrastive…

Tactile-Guided Robot Learning: From Representation Learning to Foundation Models

Kiyanoush Nazari, Luchen Li, Haoshu Fang +4 more

cs.RO · 14 Oct 2026 · cited 1× · arXiv ↗ · more like this ·

Tactile sensing is becoming central to robust, adaptive, and dexterous robotic manipulation. Beyond its traditional role in contact detection and reactive control, touch is increasingly used to learn transferable representations, condition manipulation policies, and support embodied physical reasoning. Yet existing surveys largely emphasize tactile hardware and perception, leaving the rapid shift toward learning-based tactile manipulation insufficiently synthesized. This paper reviews tactile-guided skill learning from a learning-centric perspective, organized by the functional role of touch across the learning and control pipeline. We examine tactile representation and policy learning, tactile foundation models, predictive and adaptive control, and generalization across objects, tasks, sensors, embodiments, and simulation-to-real transfer. Across these directions, we identify common…

Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention

Shengchuan Gao, Teng Hu, Bohao Feng +4 more

cs.CV · 16 Aug 2026 · cited 1× · arXiv ↗ · more like this ·

Recent audio-visual generation models can synthesize synchronized video and sound in a unified diffusion process, but their inference cost remains high because long video token sequences require repeated attention computation across denoising steps.A variety of acceleration techniques have been developed for video generation models, including low-bit quantization, attention sparsification, and feature caching.However, since these methods are originally designed for video generation, directly applying them to audio-visual models overlooks the interactions between the audio and video branches and may therefore disrupt audio-video synchronization.We present a synchronization-aware acceleration framework for efficient audio-visual generation.Our key observation is that bidirectional audio-video cross-attention reveals structured interactions between the two branches, with high responses…

Who Gets Left Out of Digital Banking in Later Life? Barriers and Opportunities in Hong Kong's Silver Population

Chi San Cheung, Jianan Johanna Liu, Lulin Chen +5 more

Demography · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 12 Jun 2026 · cited 0× · DOI ↗ · more like this ·

As digital banking increasingly replaces face-to-face financial services, older adults face growing challenges in navigating self-service and mobile platforms. This issue is particularly salient in Hong Kong, where a highly digitalized yet fragmented multi-channel banking ecosystem combines branches, ATMs, mobile apps, and phone banking. While prior research has identified general barriers such as usability and trust, less is known about how banking practices, challenges, and support needs differ across stages of later life. We address this gap through a mixed-methods study in Hong Kong, combining an in-person survey with 151 adults aged 60+ and semi-structured interviews with older adults and frontline bank staff. Participants were grouped into young-old (60–69), old-old (70–79), and oldest-old (80+) cohorts. Our findings reveal clear age-related patterns: young-old adults actively use…

Bag of Policies for Distributional Deep Exploration

Asen Nachkov, Luchen Li, Giulia Luise +2 more

cs.LG · 3 Aug 2023 · cited 0× · arXiv ↗ · more like this ·

Efficient exploration in complex environments remains a major challenge for reinforcement learning (RL). Compared to previous Thompson sampling-inspired mechanisms that enable temporally extended exploration, i.e., deep exploration, we focus on deep exploration in distributional RL. We develop here a general purpose approach, Bag of Policies (BoP), that can be built on top of any return distribution estimator by maintaining a population of its copies. BoP consists of an ensemble of multiple heads that are updated independently. During training, each episode is controlled by only one of the heads and the collected state-action pairs are used to update all heads off-policy, leading to distinct learning signals for each head which diversify learning and behaviour. To test whether optimistic ensemble method can improve on distributional RL as did on scalar RL, by e.g. Bootstrapped DQN, we…

Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation

Luchen Li, Ignacio Albert-Smet, Aldo A. Faisal

cs.LG · 18 Mar 2020 · arXiv ↗ · more like this ·

Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies in a clinical deployment. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics, and the clinically debated, highly-divergent set of intervention policies required by each individual patient, yet intensive care units are naturally data rich. In our work, we build on RL approaches in healthcare ("AI Clinicians"), and learn off-policy continuous dosing policy of pharmaceuticals for sepsis treatment using historical intensive care data under partially observable MDPs (POMDPs). POMPDs capture uncertainty in patient state better by taking in all historical information…