arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Victor Dibia

23 papers · 2008–2026

2008: 12009: 02010: 02011: 02012: 02013: 02014: 02015: 22016: 02017: 02018: 42019: 12020: 12021: 02022: 12023: 22024: 72025: 22026: 2
2008peak 7 in 20242026

Frequent co-authors

Papers

showing 1–23 of 23 RSS

Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networks

Victor Dibia, Çağatay Demiralp

cs.HC · · cited 243× · arXiv ↗ · more like this ·

Rapidly creating effective visualizations using expressive grammars is challenging for users who have limited time and limited skills in statistics and data visualization. Even high-level, dedicated visualization tools often require users to manually select among data attributes, decide which transformations to apply, and specify mappings between visual encoding variables and raw or transformed attributes. In this paper we introduce Data2Vis, a neural translation model for automatically generating visualizations from given datasets. We formulate visualization generation as a sequence to sequence translation problem where data specifications are mapped to visualization specifications in a declarative language (Vega-Lite). To this end, we train a multilayered attention-based recurrent neural network (RNN) with long short-term memory (LSTM) units on a corpus of visualization…

LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models

Victor Dibia

cs.AI · · cited 202× · arXiv ↗ · more like this ·

Systems that support users in the automatic creation of visualizations must address several subtasks - understand the semantics of data, enumerate relevant visualization goals and generate visualization specifications. In this work, we pose visualization generation as a multi-stage generation problem and argue that well-orchestrated pipelines based on large language models (LLMs) such as ChatGPT/GPT-4 and image generation models (IGMs) are suitable to addressing these tasks. We present LIDA, a novel tool for generating grammar-agnostic visualizations and infographics. LIDA comprises of 4 modules - A SUMMARIZER that converts data into a rich but compact natural language summary, a GOAL EXPLORER that enumerates visualization goals given the data, a VISGENERATOR that generates, refines, executes and filters visualization code and an INFOGRAPHER module that yields data-faithful stylized…

Interactive Debugging and Steering of Multi-Agent AI Systems

Will Epperson, Gagan Bansal, Victor Dibia +4 more

cs.MA · · cited 106× · arXiv ↗ · more like this ·

Fully autonomous teams of LLM-powered AI agents are emerging that collaborate to perform complex tasks for users. What challenges do developers face when trying to build and debug these AI agent teams? In formative interviews with five AI agent developers, we identify core challenges: difficulty reviewing long agent conversations to localize errors, lack of support in current tools for interactive debugging, and the need for tool support to iterate on agent configuration. Based on these needs, we developed an interactive multi-agent debugging tool, AGDebugger, with a UI for browsing and sending messages, the ability to edit and reset prior agent messages, and an overview visualization for navigating complex message histories. In a two-part user study with 14 participants, we identify common user strategies for steering agents and highlight the importance of interactive message resets…

AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systems

Victor Dibia, Jingya Chen, Gagan Bansal +5 more

cs.SE · · cited 43× · arXiv ↗ · more like this ·

Multi-agent systems, where multiple agents (generative AI models + tools) collaborate, are emerging as an effective pattern for solving long-running, complex tasks in numerous domains. However, specifying their parameters (such as models, tools, and orchestration mechanisms etc,.) and debugging them remains challenging for most developers. To address this challenge, we present AUTOGEN STUDIO, a no-code developer tool for rapidly prototyping, debugging, and evaluating multi-agent workflows built upon the AUTOGEN framework. AUTOGEN STUDIO offers a web interface and a Python API for representing LLM-enabled agents using a declarative (JSON-based) specification. It provides an intuitive drag-and-drop UI for agent workflow specification, interactive evaluation and debugging of workflows, and a gallery of reusable agent components. We highlight four design principles for no-code multi-agent…

Beyond Heuristics: Learning Visualization Design

Bahador Saket, Dominik Moritz, Halden Lin +3 more

cs.HC · · cited 39× · arXiv ↗ · more like this ·

In this paper, we describe a research agenda for deriving design principles directly from data. We argue that it is time to go beyond manually curated and applied visualization design guidelines. We propose learning models of visualization design from data collected using graphical perception studies and build tools powered by the learned models. To achieve this vision, we need to 1) develop scalable methods for collecting training data, 2) collect different forms of training data, 3) advance interpretability of machine learning models, and 4) develop adaptive models that evolve as more data becomes available.

Exploring the Potential of Wearables to Support Employment for People with Mild Cognitive Impairment

Victor Dibia, Shari Trewin, Maryam Ashoori +1 more

Demography · · cited 31× · DOI ↗ · more like this ·

We interviewed adults with mild cognitive impairment working in a sheltered workshop and three support professionals about competitive employment in the community, and what concerns or barriers they may have. While individuals were most concerned about managing their health, professionals also described personal safety and productivity challenges. Families' worries about the safety of their family members were reported to be a barrier to competitive employment. Using a smart watch as a technology probe, we identified three potential applications of wearable technology in tackling these challenges: health support, family support, and productivity support.

Aligning Offline Metrics and Human Judgments of Value for Code Generation Models

Victor Dibia, Adam Fourney, Gagan Bansal +3 more

cs.SE · · cited 9× · arXiv ↗ · more like this ·

Large language models have demonstrated great potential to assist programmers in generating code. For such human-AI pair programming scenarios, we empirically demonstrate that while generated code is most often evaluated in terms of their functional correctness (i.e., whether generations pass available unit tests), correctness does not fully capture (e.g., may underestimate) the productivity gains these models may provide. Through a user study with N = 49 experienced programmers, we show that while correctness captures high-value generations, programmers still rate code that fails unit tests as valuable if it reduces the overall effort needed to complete a coding task. Finally, we propose a hybrid metric that combines functional correctness and syntactic similarity and show that it achieves a 14% stronger correlation with value and can therefore better represent real-world gains when…

An Embodied Cognitive Assistant for Visualizing and Analyzing Exoplanet Data

Jeffrey O. Kephart, Victor Dibia, Jason Ellis +3 more

Artificial Intelligence · IEEE Internet Computing · · cited 9× · DOI ↗ · more like this ·

In this paper, we describe a cognitive assistant that helps astrophysicists visualize and analyze exoplanet data via natural interactions. When users speak about and point to objects of interest, the assistant finds relevant data or performs requested computations, shows the results on a large display wall, and explains them with synthesized speech.

A Cognitive Assistant for Visualizing and Analyzing Exoplanets

Jeffrey O. Kephart, Victor Dibia, Jason Ellis +3 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 8× · DOI ↗ · more like this ·

We demonstrate an embodied cognitive agent that helps scientists visualize and analyze exo-planets and their host stars. The prototype is situated in a room equipped with a large display, microphones, cameras, speakers, and pointing devices. Users communicate with the agent via speech, gestures, and combinations thereof, and it responds by displaying content and generating synthesized speech. Extensive use of context facilitates natural interaction with the agent.

Data Analysis in the Era of Generative AI

Jeevana Priya Inala, Chenglong Wang, Steven Drucker +6 more

cs.AI · · cited 4× · arXiv ↗ · more like this ·

This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow by translating high-level user intentions into executable code, charts, and insights. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps. Finally, we discuss the research challenges that impede the development of these AI-based systems such as enhancing model capabilities, evaluating and benchmarking, and understanding end-user needs.

Designing for Democratization: Introducing Novices to Artificial Intelligence Via Maker Kits

Victor Dibia, Aaron Cox, Justin Weisz

cs.HC · · cited 4× · arXiv ↗ · more like this ·

Existing research highlight the myriad of benefits realized when technology is sufficiently democratized and made accessible to non-technical or novice users. However, democratizing complex technologies such as artificial intelligence (AI) remains hard. In this work, we draw on theoretical underpinnings from the democratization of innovation, in exploring the design of maker kits that help introduce novice users to complex technologies. We report on our work designing TJBot: an open source cardboard robot that can be programmed using pre-built AI services. We highlight principles we adopted in this process (approachable design, simplicity, extensibility and accessibility), insights we learned from showing the kit at workshops (66 participants) and how users interacted with the project on GitHub over a 12-month period (Nov 2016 - Nov 2017). We find that the project succeeds in attracting…

Magentic-UI: Towards Human-in-the-loop Agentic Systems

Hussein Mozannar, Gagan Bansal, Cheng Tan +17 more

cs.AI · · cited 3× · arXiv ↗ · more like this ·

AI agents powered by large language models are increasingly capable of autonomously completing complex, multi-step tasks using external tools. Yet, they still fall short of human-level performance in most domains including computer use, software development, and research. Their growing autonomy and ability to interact with the outside world, also introduces safety and security risks including potentially misaligned actions and adversarial manipulation. We argue that human-in-the-loop agentic systems offer a promising path forward, combining human oversight and control with AI efficiency to unlock productivity from imperfect systems. We introduce Magentic-UI, an open-source web interface for developing and studying human-agent interaction. Built on a flexible multi-agent architecture, Magentic-UI supports web browsing, code execution, and file manipulation, and can be extended with…

Concept Distillation from Strong to Weak Models via Hypotheses-to-Theories Prompting

Emmanuel Aboah Boateng, Cassiano O. Becker, Nabiha Asghar +5 more

cs.AI · · cited 2× · arXiv ↗ · more like this ·

Hand-crafting high quality prompts to optimize the performance of language models is a complicated and labor-intensive process. Furthermore, when migrating to newer, smaller, or weaker models (possibly due to latency or cost gains), prompts need to be updated to re-optimize the task performance. We propose Concept Distillation (CD), an automatic prompt optimization technique for enhancing weaker models on complex tasks. CD involves: (1) collecting mistakes made by weak models with a base prompt (initialization), (2) using a strong model to generate reasons for these mistakes and create rules/concepts for weak models (induction), and (3) filtering these rules based on validation set performance and integrating them into the base prompt (deduction/verification). We evaluated CD on NL2Code and mathematical reasoning tasks, observing significant performance boosts for small and weaker…

Axiomatic Preference Modeling for Longform Question Answering

Corby Rosset, Guoqing Zheng, Victor Dibia +2 more

cs.AI · · cited 2× · arXiv ↗ · more like this ·

The remarkable abilities of large language models (LLMs) like GPT-4 partially stem from post-training processes like Reinforcement Learning from Human Feedback (RLHF) involving human preferences encoded in a reward model. However, these reward models (RMs) often lack direct knowledge of why, or under what principles, the preferences annotations were made. In this study, we identify principles that guide RMs to better align with human preferences, and then develop an axiomatic framework to generate a rich variety of preference signals to uphold them. We use these axiomatic signals to train a model for scoring answers to longform questions. Our approach yields a Preference Model with only about 220M parameters that agrees with gold human-annotated preference labels more often than GPT-4. The contributions of this work include: training a standalone preference model that can score human-…

NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets

Victor Dibia

cs.CL · · cited 1× · arXiv ↗ · more like this ·

Existing tools for Question Answering (QA) have challenges that limit their use in practice. They can be complex to set up or integrate with existing infrastructure, do not offer configurable interactive interfaces, and do not cover the full set of subtasks that frequently comprise the QA pipeline (query expansion, retrieval, reading, and explanation/sensemaking). To help address these issues, we introduce NeuralQA - a usable library for QA on large datasets. NeuralQA integrates well with existing infrastructure (e.g., ElasticSearch instances and reader models trained with the HuggingFace Transformers API) and offers helpful defaults for QA subtasks. It introduces and implements contextual query expansion (CQE) using a masked language model (MLM) as well as relevant snippets (RelSnip) - a method for condensing large documents into smaller passages that can be speedily processed by a…

Understanding the Disclosure of Private Healthcare Information within Online Quantified Self 2.0 Platforms

Jun Zhang, Victor Dibia, Alexey Sodnomov +1 more

Sociology and Political Science · SSRN Electronic Journal · · cited 1× · OpenAlex ↗ · more like this ·

The quantified self-movement encourages a continuous tracking of data points regarding a person’s daily activities through wearable sensors, and thus has important implications for health and wellness. With the advent of sophisticated low-cost wearable computing devices, online communities that facilitate social interaction and exchange of wearable data (Quantified Self 2.0 platforms) have also emerged. Although security and privacy disclosure has been studied within online social networks and online health communities, little has been done to understand how individual and group characteristics influence the disclosure behaviour regarding highly sensitive personal information gathered from wearable sensors (e.g., sleep, nutrition, mood, performance, ambient conditions). Using data collected from 43 Fitbit groups which consist of 5300 Asian users within the Fitbit online community, we…

"Is This Book AI-Generated?" How Authorship Suspicion Manifests in Marketplace Reviews

Victor Dibia

cs.HC · · cited 0× · arXiv ↗ · more like this ·

As AI becomes part of how books are authored, reader response to suspected AI authorship grows more consequential, yet remains unexamined. We analyze 863 low-star reviews of 78 Amazon bestsellers across 8 categories at three levels of proximity to AI. Suspicion concentrates in Generative AI books (35.1%) but appears in every category, including Gardening (5.7%). Reviews citing AI authorship complain more about shallow content and poor presentation than other critical reviews. Suspicion takes two forms: ambient, where "AI-generated" is a generic complaint about formulaic writing, and corroborated, where reviewers of the same book independently cite concrete evidence. We propose two mechanisms by which suspicion arises: topical concentration, where a book's AI subject matter supplies vocabulary for quality complaints, and artifact detection, where readers notice ChatGPT-style formatting…

Services Provisioning on Mobile Devices via Bluetooth in a Localized Setting Using a Proposed System - BTServ

Oludele Awodele, Victor Dibia, Oghenerukevwe Onoruvie +1 more

Computer Networks and Communications · Informing Science and IT Education Conference · · cited 0× · DOI ↗ · more like this ·

Conventionally, enterprise services that can be utilized by mobile devices such as E - mail, instant messaging, data transfer, news and event notification have been typically achieved using GPRS (General Packet Radio Service) connections to access internet servers that provide these services. Thus, even in a local area, disadvantages such as full dependence on GPRS service providers, incurred GPRS costs and slow data/communication rates constitute formidable drawbacks. This paper describes a system /Architecture (BTServ) in which these same services can be supplied via Bluetooth, with notable benefits such as high data rates, minimal dependence on external (GPRS) networks, higher security (internet attacks are totally avoided) and reduced cost, in a localized setting such as a corporate house.

Agents in the Wild: Where Research Meets Deployment

Grace Hui Yang, Pranav N. Venkit, Hooman Sedghamiz +3 more

cs.AI · · arXiv ↗ · more like this ·

Agentic systems large language model (LLM) based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents are rapidly transitioning from research prototypes to production scale deployments across domains such as software engineering, scientific discovery, and finance. While academic work has emphasized benchmarks and algorithmic innovation, deployment raises new challenges around robustness, safety, and reliability. This tutorial brings together researchers and practitioners to explore advances in reasoning and planning, multi agent coordination, and evaluation, highlighting open challenges arising from deployment experience. Through applied case studies in pharmaceutical discovery and financial systems, we analyze common design patterns that make agentic systems successful, and discuss practical mitigation strategies for failure modes, such as…

Challenges in Human-Agent Communication

Gagan Bansal, Jennifer Wortman Vaughan, Saleema Amershi +5 more

cs.HC · · arXiv ↗ · more like this ·

Remarkable advancements in modern generative foundation models have enabled the development of sophisticated and highly capable autonomous agents that can observe their environment, invoke tools, and communicate with other agents to solve problems. Although such agents can communicate with users through natural language, their complexity and wide-ranging failure modes present novel challenges for human-AI interaction. Building on prior research and informed by a communication grounding perspective, we contribute to the study of \emph{human-agent communication} by identifying and analyzing twelve key communication challenges that these systems pose. These include challenges in conveying information from the agent to the user, challenges in enabling the user to convey information to the agent, and overarching challenges that need to be considered across all human-agent communication. We…

Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks

Adam Fourney, Gagan Bansal, Hussein Mozannar +17 more

cs.AI · · arXiv ↗ · more like this ·

Modern AI agents, driven by advances in large foundation models, promise to enhance our productivity and transform our lives by augmenting our knowledge and capabilities. To achieve this vision, AI agents must effectively plan, perform multi-step reasoning and actions, respond to novel observations, and recover from errors, to successfully complete complex tasks across a wide range of scenarios. In this work, we introduce Magentic-One, a high-performing open-source agentic system for solving such tasks. Magentic-One uses a multi-agent architecture where a lead agent, the Orchestrator, plans, tracks progress, and re-plans to recover from errors. Throughout task execution, the Orchestrator directs other specialized agents to perform tasks as needed, such as operating a web browser, navigating local files, or writing and executing Python code. We show that Magentic-One achieves…

Towards better Human-Agent Alignment: Assessing Task Utility in LLM-Powered Applications

Negar Arabzadeh, Julia Kiseleva, Qingyun Wu +5 more

cs.CL · · arXiv ↗ · more like this ·

The rapid development in the field of Large Language Models (LLMs) has led to a surge in applications that facilitate collaboration among multiple agents to assist humans in their daily tasks. However, a significant gap remains in assessing whether LLM-powered applications genuinely enhance user experience and task execution efficiency. This highlights the pressing need for methods to verify utility of LLM-powered applications, particularly by ensuring alignment between the application's functionality and end-user needs. We introduce AgentEval provides an implementation for the math problems, a novel framework designed to simplify the utility verification process by automatically proposing a set of criteria tailored to the unique purpose of any given application. This allows for a comprehensive assessment, quantifying the utility of an application against the suggested criteria. We…

The Aleph & Other Metaphors for Image Generation

Gonzalo Ramos, Rick Barraza, Victor Dibia +1 more

cs.HC · · arXiv ↗ · more like this ·

In this position paper, we reflect on fictional stories dealing with the infinite and how they connect with the current, fast-evolving field of image generation models. We draw attention to how some of these literary constructs can serve as powerful metaphors for guiding human-centered design and technical thinking in the space of these emerging technologies and the experiences we build around them. We hope our provocations seed conversations about current and yet-to-be developed interactions with these emerging models in ways that may amplify human agency.