arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Pascal Poupart

185 papers · 2001–2026

2001: 12002: 42003: 22004: 12005: 72006: 52007: 22008: 72009: 32010: 22011: 132012: 152013: 62014: 72015: 82016: 102017: 82018: 82019: 72020: 122021: 82022: 112023: 52024: 112025: 112026: 11
2001peak 15 in 20122026

Frequent co-authors

Papers

showing 1–25 of 185 RSS

Representation Learning for Dynamic Graphs: A Survey

Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain +4 more

cs.LG · · cited 674× · arXiv ↗ · more like this ·

Graphs arise naturally in many real-world applications including social networks, recommender systems, ontologies, biology, and computational finance. Traditionally, machine learning models for graphs have been mostly designed for static graphs. However, many applications involve evolving graphs. This introduces important challenges for learning and inference since nodes, attributes, and edges change over time. In this survey, we review the recent advances in representation learning for dynamic graphs, including dynamic knowledge graphs. We describe existing models from an encoder-decoder perspective, categorize these encoders and decoders based on the techniques they employ, and analyze the approaches in each category. We also review several prominent applications and widely used datasets and highlight directions for future research.

Time2Vec: Learning a Vector Representation of Time

Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali +7 more

cs.LG · · cited 532× · arXiv ↗ · more like this ·

Time is an important feature in many applications involving events that occur synchronously and/or asynchronously. To effectively consume time information, recent studies have focused on designing new architectures. In this paper, we take an orthogonal but complementary approach by providing a model-agnostic vector representation for time, called Time2Vec, that can be easily imported into many existing and future architectures and improve their performances. We show on a range of models and problems that replacing the notion of time with its Time2Vec representation improves the performance of the final model.

Diachronic Embedding for Temporal Knowledge Graph Completion

Rishab Goel, Seyed Mehran Kazemi, Marcus Brubaker +1 more

cs.LG · · cited 471× · arXiv ↗ · more like this ·

Knowledge graphs (KGs) typically contain temporal facts indicating relationships among entities at different times. Due to their incompleteness, several approaches have been proposed to infer new facts for a KG based on the existing ones-a problem known as KG completion. KG embedding approaches have proved effective for KG completion, however, they have been developed mostly for static KGs. Developing temporal KG embedding models is an increasingly important problem. In this paper, we build novel models for temporal KG completion through equipping static models with a diachronic entity embedding function which provides the characteristics of entities at any point in time. This is in contrast to the existing temporal KG embedding approaches where only static entity features are provided. The proposed embedding function is model-agnostic and can be potentially combined with any static…

An analytic solution to discrete Bayesian reinforcement learning

Pascal Poupart, Nikos Vlassis, Jesse Hoey +1 more

Artificial Intelligence · UvA-DARE (University of Amsterdam) · · cited 296× · DOI ↗ · more like this ·

Reinforcement learning (RL) was originally proposed as a framework to allow agents to learn in an online fashion as they interact with their environment. Existing RL algorithms come short of achieving this goal because the amount of exploration required is often too costly and/or too time consuming for online learning. As a result, RL is mostly used for offline learning in simulated environments. We propose a new algorithm, called BEETLE, for effective online learning that is computationally efficient while minimizing the amount of exploration. We take a Bayesian model-based approach, framing RL as a partially observable Markov decision process. Our two main contributions are the analytical derivation that the optimal value function is the upper envelope of a set of multivariate polynomials, and an efficient point-based value iteration algorithm that exploits this simple…

Point-Based Value Iteration for Continuous POMDPs

Josep M. Porta, Nikos Vlassis, Matthijs T. J. Spaan +1 more

Artificial Intelligence · DIGITAL.CSIC (Spanish National Research Council (CSIC)) · · cited 246× · DOI ↗ · more like this ·

We propose a novel approach to optimize Partially Observable Markov Decisions Processes (POMDPs) defined on continuous spaces. To date, most algorithms for model-based POMDPs are restricted to discrete states, actions, and observations, but many real-world problems such as, for instance, robot navigation, are naturally defined on continuous spaces. In this work, we demonstrate that the value function for continuous POMDPs is convex in the beliefs over continuous state spaces, and piecewise-linear convex for the particular case of discrete observations and actions but still continuous states. We also demonstrate that continuous Bellman backups are contracting and isotonic ensuring the monotonic convergence of value-iteration algorithms. Relying on those properties, we extend the PERSEUS algorithm, originally developed for discrete POMDPs, to work in continuous state spaces by…

Exploiting structure to efficiently solve large scale partially observable markov decision processes

Pascal Poupart

Artificial Intelligence · · cited 209× · OpenAlex ↗ · more like this ·

Partially observable Markov decision processes (POMDPs) provide a natural and principled framework to model a wide range of sequential decision making problems under uncertainty. To date, the use of POMDPs in real-world problems has been limited by the poor scalability of existing solution algorithms, which can only solve problems with up to ten thousand states. In fact, the complexity of finding an optimal policy for a finite-horizon discrete POMDP is PSPACE-complete. In practice, two important sources of intractability plague most solution algorithms: large policy spaces and large state spaces. On the other hand,

Self-Adaptive Hierarchical Sentence Model

Han Zhao, Zhengdong Lu, Pascal Poupart

cs.CL · · cited 191× · arXiv ↗ · more like this ·

The ability to accurately model a sentence at varying stages (e.g., word-phrase-sentence) plays a central role in natural language processing. As an effort towards this goal we propose a self-adaptive hierarchical sentence model (AdaSent). AdaSent effectively forms a hierarchy of representations from words to phrases and then to sentences through recursive gated local composition of adjacent segments. We design a competitive mechanism (through gating networks) to allow the representations of the same sentence to be engaged in a particular learning task (e.g., classification), therefore effectively mitigating the gradient vanishing problem persistent in other recursive models. Both qualitative and quantitative analysis shows that AdaSent can automatically form and select the representations suitable for the task at hand during training, yielding superior classification performance over…

Bounded Finite State Controllers

Pascal Poupart, Craig Boutilier

Artificial Intelligence · Neural Information Processing Systems · · cited 162× · OpenAlex ↗ · more like this ·

We describe a new approximation algorithm for solving partially observable MDPs. Our bounded policy iteration approach searches through the space of bounded-size, stochastic finite state controllers, combining several advantages of gradient ascent (efficiency, search through restricted controller space) and policy iteration (less vulnerability to local optima). 1

A decision-theoretic approach to task assistance for persons with dementia

Jennifer Boger, Pascal Poupart, Jesse Hoey +3 more

Artificial Intelligence · Discovery Research Portal (University of Dundee) · · cited 160× · DOI ↗ · more like this ·

Heart failure (HF) is a common disease that causes significant limitations on the organism's capacity and, in extreme cases, leads to death. Clinically, iron deficiency (ID) plays an essential role in heart failure by deteriorating the patient's condition and is a prognostic marker indicating poor clinical outcomes. Therefore, in HF patients, supplementation of iron is recommended. However, iron treatment may cause adverse effects by increasing iron-related apoptosis and the production of oxygen radicals, which may cause additional heart damage. Furthermore, many knowledge gaps exist regarding the complex interplay between iron deficiency and heart failure. Here, we describe the current, comprehensive knowledge about the role of the proteins involved in iron metabolism. We will focus on the molecular and clinical aspects of iron deficiency in HF. We believe that summarizing the new…

Assisting persons with dementia during handwashing using a partially observable Markov decision process.

Jesse Hoey, Axel von Bertoldi, Pascal Poupart +1 more

Artificial Intelligence · PUB – Publications at Bielefeld University (Bielefeld University) · · cited 145× · DOI ↗ · more like this ·

This paper presents a real-time system to assist a person with dementia wash their hands. Assistance is given in the form of verbal and/or visual prompts, or through the enlistment of a human caregiver's help. The system uses only video inputs, and combines a Bayesian sequential estimation framework for tracking hands and towel, with a decision theoretic framework for computing policies of action -- specifically a partially observable Markov decision process (POMDP). A key element of the system is the ability to estimate and adapt to user states, such as awareness, responsiveness and overall dementia level. We demonstrate the system in a set of simulation experiments, and we show examples of real-time interactions with actors.

Variational Attention for Sequence-to-Sequence Models

Hareesh Bahuleyan, Lili Mou, Olga Vechtomova +1 more

cs.CL · · cited 122× · arXiv ↗ · more like this ·

The variational encoder-decoder (VED) encodes source information as a set of random variables using a neural network, which in turn is decoded into target data using another neural network. In natural language processing, sequence-to-sequence (Seq2Seq) models typically serve as encoder-decoder networks. When combined with a traditional (deterministic) attention mechanism, the variational latent space may be bypassed by the attention model, and thus becomes ineffective. In this paper, we propose a variational attention mechanism for VED, where the attention vector is also modeled as Gaussian distributed random variables. Results on two experiments show that, without loss of quality, our proposed method alleviates the bypassing phenomenon as it increases the diversity of generated sentences.

Order-Planning Neural Text Generation From Structured Data

Lei Sha, Lili Mou, Tianyu Liu +4 more

cs.CL · · cited 117× · arXiv ↗ · more like this ·

Generating texts from structured data (e.g., a table) is important for various natural language processing tasks such as question answering and dialog systems. In recent studies, researchers use neural language models and encoder-decoder frameworks for table-to-text generation. However, these neural network-based approaches do not model the order of contents during text generation. When a human writes a summary based on a given table, he or she would probably consider the content order before wording. In a biography, for example, the nationality of a person is typically mentioned before occupation in a biography. In this paper, we propose an order-planning text generation model to capture the relationship between different fields and use such relationship to make the generated text more fluent and smooth. We conducted experiments on the WikiBio dataset and achieve significantly higher…

Factored partially observable Markov decision processes for dialogue management

J. D. Williams, Pascal Poupart, SJ Young

Artificial Intelligence · Cambridge University Engineering Department Publications Database · · cited 114× · OpenAlex ↗ · more like this ·

This work shows how a dialogue model can be represented as a factored Partially Observable Markov Decision Process (POMDP). The factored representation has several benefits, such as enabling more nuanced reward functions to be specified. Although our dialogue model is significantly larger than past work using POMDPs, experiments on a small testbed problem demonstrate that recent optimisation techniques scale well and produce policies which outperform a traditional fully-observable Markov Decision Process. This work then shows how a dialogue manager produced with a POMDP optimisation technique may be directly compared to a handcrafted dialogue manager. Experiments on the testbed problem show that automatically generated dialogue managers outperform several handcrafted dialogue managers, and that automatically generated dialogue managers for the testbed problem successfully adapt to…

Value-Directed Compression of POMDPs

Pascal Poupart, Craig Boutilier

Computational Theory and Mathematics · Neural Information Processing Systems · · cited 99× · OpenAlex ↗ · more like this ·

We examine the problem of generating state-space compressions of POMDPs in a way that minimally impacts decision quality. We analyze the impact of compressions on decision quality, observing that compressions that allow accurate policy evaluation (prediction of expected future reward) will not affect decision quality. We derive a set of sufficient conditions that ensure accurate prediction in this respect, illustrate interesting mathematical properties these confer on lossless linear compressions, and use these to derive an iterative procedure for finding good linear lossy compressions. We also elaborate on how structured representations of a POMDP can be used to find such compressions. 1

Solving POMDPs with continuous or large discrete observation spaces

Jesse Hoey, Pascal Poupart

Artificial Intelligence · Discovery Research Portal (University of Dundee) · · cited 98× · OpenAlex ↗ · more like this ·

We describe methods to solve partially observable Markov decision processes (POMDPs) with continuous or large discrete observation spaces. Realistic problems often have rich observation spaces, posing significant problems for standard POMDP algorithms that require explicit enumeration of the observations. This problem is usually approached by imposing an a priori discretisation on the observation space, which can be sub-optimal for the decision making task. However, since only those observations that would change the policy need to be distinguished, the decision problem itself induces a lossless partitioning of the observation space. This paper demonstrates how to find this partition while computing a policy, and how the resulting discretisation of the observation space reveals the relevant features of the application domain. The algorithms are demonstrated on a toy example and on a…

On the Relationship between Sum-Product Networks and Bayesian Networks

Han Zhao, Mazen Melibari, Pascal Poupart

cs.AI · · cited 97× · arXiv ↗ · more like this ·

In this paper, we establish some theoretical connections between Sum-Product Networks (SPNs) and Bayesian Networks (BNs). We prove that every SPN can be converted into a BN in linear time and space in terms of the network size. The key insight is to use Algebraic Decision Diagrams (ADDs) to compactly represent the local conditional probability distributions at each node in the resulting BN by exploiting context-specific independence (CSI). The generated BN has a simple directed bipartite graphical structure. We show that by applying the Variable Elimination algorithm (VE) to the generated BN with ADD representations, we can recover the original SPN where the SPN can be viewed as a history record or caching of the VE inference process. To help state the proof clearly, we introduce the notion of {\em normal} SPN and present a theoretical analysis of the consistency and decomposability…

Online flow size prediction for improved network routing

Pascal Poupart, Zhitang Chen, Priyank Jaini +6 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · · cited 92× · DOI ↗ · more like this ·

We describe an emerging application of data mining in the context of computer networks. This application concerns the problem of predicting the size of a flow and detecting elephant flows (very large flows). Flow size is a very important statistic that can be used to improve routing, load balancing and scheduling in computer networks. Flow size prediction is particularly challenging since flow patterns continuously change and predictions must be done in real time (milliseconds) to avoid delays. We describe how to formulate the problem as an online machine learning task to continuously adjust to changes in flow traffic. We evaluate the predictive nature of a set of features and the accuracy of three online predictors based on neural networks, Gaussian process regression and online Bayesian Moment Matching on three datasets of real traffic. We also demonstrate how to use such online…

Learning Dynamic Belief Graphs to Generalize on Text-Based Games

Ashutosh Adhikari, Xingdi Yuan, Marc-Alexandre Côté +7 more

cs.CL · · cited 91× · arXiv ↗ · more like this ·

Playing text-based games requires skills in processing natural language and sequential decision making. Achieving human-level performance on text-based games remains an open challenge, and prior research has largely relied on hand-crafted structured representations and heuristics. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text. We propose a novel graph-aided transformer agent (GATA) that infers and updates latent belief graphs during planning to enable effective action selection by capturing the underlying game dynamics. GATA is trained using a combination of reinforcement and self-supervised learning. Our work demonstrates that the learned graph-based representations help agents converge to better policies than their text-only counterparts and facilitate effective…

Hierarchical POMDP Controller Optimization by Likelihood Maximization

Marc Toussaint, Laurent Charlin, Pascal Poupart

cs.AI · · cited 89× · arXiv ↗ · more like this ·

Planning can often be simpli ed by decomposing the task into smaller tasks arranged hierarchically. Charlin et al. [4] recently showed that the hierarchy discovery problem can be framed as a non-convex optimization problem. However, the inherent computational di culty of solving such an optimization problem makes it hard to scale to realworld problems. In another line of research, Toussaint et al. [18] developed a method to solve planning problems by maximumlikelihood estimation. In this paper, we show how the hierarchy discovery problem in partially observable domains can be tackled using a similar maximum likelihood approach. Our technique rst transforms the problem into a dynamic Bayesian network through which a hierarchical structure can naturally be discovered while optimizing the policy. Experimental results demonstrate that this approach scales better than previous techniques…

Minimal Sufficient Explanations for Factored Markov Decision Processes

Omar Khan, Pascal Poupart, James P. Black

Artificial Intelligence · Proceedings of the International Conference on Automated Planning and Scheduling · · cited 75× · DOI ↗ · more like this ·

Explaining policies of Markov Decision Processes (MDPs) is complicated due to their probabilistic and sequential nature. We present a technique to explain policies for factored MDP by populating a set of domain-independent templates. We also present a mechanism to determine a minimal set of templates that, viewed together, completely justify the policy. Our explanations can be generated automatically at run-time with no additional effort required from the MDP designer. We demonstrate our technique using the problems of advising undergraduate students in their course selection and assisting people with dementia in completing the task of handwashing. We also evaluate our explanations for course-advising through a user study involving students.

Unsupervised Video Object Segmentation for Deep Reinforcement Learning

Vik Goel, Jameson Weng, Pascal Poupart

cs.CV · · cited 72× · arXiv ↗ · more like this ·

We present a new technique for deep reinforcement learning that automatically detects moving objects and uses the relevant information for action selection. The detection of moving objects is done in an unsupervised way by exploiting structure from motion. Instead of directly learning a policy from raw images, the agent first learns to detect and segment moving objects by exploiting flow information in video sequences. The learned representation is then used to focus the policy of the agent on the moving objects. Over time, the agent identifies which objects are critical for decision making and gradually builds a policy based on relevant moving objects. This approach, which we call Motion-Oriented REinforcement Learning (MOREL), is demonstrated on a suite of Atari games where the ability to detect moving objects reduces the amount of interaction needed with the environment to obtain a…

Multi Type Mean Field Reinforcement Learning

Sriram Ganapathi Subramanian, Pascal Poupart, Matthew E. Taylor +1 more

cs.MA · · cited 70× · arXiv ↗ · more like this ·

Mean field theory provides an effective way of scaling multiagent reinforcement learning algorithms to environments with many agents that can be abstracted by a virtual mean agent. In this paper, we extend mean field multiagent algorithms to multiple types. The types enable the relaxation of a core assumption in mean field reinforcement learning, which is that all agents in the environment are playing almost similar strategies and have the same goal. We conduct experiments on three different testbeds for the field of many agent reinforcement learning, based on the standard MAgents framework. We consider two different kinds of mean field environments: a) Games where agents belong to predefined types that are known a priori and b) Games where the type of each agent is unknown and therefore must be learned based on observations. We introduce new algorithms for each type of game and…

A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?

Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta +3 more

cs.LG · · cited 69× · arXiv ↗ · more like this ·

Automation is one of the cornerstones of contemporary material discovery. Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior domain knowledge into efficient exploration of a large molecular space. While such prior knowledge can take many forms, there has been significant fanfare around the ancillary scientific knowledge encapsulated in large language models (LLMs). However, existing work thus far has only explored LLMs for heuristic materials searches. Indeed, recent work obtains the uncertainty estimate -- an integral part of BO -- from point-estimated, non-Bayesian LLMs. In this work, we study the question of whether LLMs are actually useful to accelerate principled Bayesian optimization in the molecular space. We take a sober, dispassionate stance in answering this question. This is done by carefully (i) viewing LLMs as fixed…

SPFlow: An Easy and Extensible Library for Deep Probabilistic Learning using Sum-Product Networks

Alejandro Molina, Antonio Vergari, Karl Stelzner +5 more

cs.LG · · cited 69× · arXiv ↗ · more like this ·

We introduce SPFlow, an open-source Python library providing a simple interface to inference, learning and manipulation routines for deep and tractable probabilistic models called Sum-Product Networks (SPNs). The library allows one to quickly create SPNs both from data and through a domain specific language (DSL). It efficiently implements several probabilistic inference routines like computing marginals, conditionals and (approximate) most probable explanations (MPEs) along with sampling as well as utilities for serializing, plotting and structure statistics on an SPN. Moreover, many of the algorithms proposed in the literature to learn the structure and parameters of SPNs are readily available in SPFlow. Furthermore, SPFlow is extremely extensible and customizable, allowing users to promptly distill new inference and learning routines by injecting custom code into a lightweight…

Exponential Recency Weighted Average Branching Heuristic for SAT Solvers

Liang Jia, Vijay Ganesh, Pascal Poupart +1 more

Computational Theory and Mathematics · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 67× · DOI ↗ · more like this ·

Modern conflict-driven clause-learning SAT solvers routinely solve large real-world instances with millions of clauses and variables in them. Their success crucially depends on effective branching heuristics. In this paper, we propose a new branching heuristic inspired by the exponential recency weighted average algorithm used to solve the bandit problem. The branching heuristic, we call CHB, learns online which variables to branch on by leveraging the feedback received from conflict analysis. We evaluated CHB on 1200 instances from the SAT Competition 2013 and 2014 instances, and showed that CHB solves significantly more instances than VSIDS, currently the most effective branching heuristic in widespread use. More precisely, we implemented CHB as part of the MiniSat and Glucose solvers, and performed an apple-to-apple comparison with their VSIDS-based variants. CHB-based MiniSat (resp.…