arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Procheta Sen

38 papers · 2016–2026

2016: 12017: 12018: 22019: 12020: 32021: 52022: 32023: 102024: 52025: 42026: 3
2016peak 10 in 20232026

Frequent co-authors

Papers

showing 1–25 of 38 RSS

Explainable Information Retrieval

Avishek Anand, Procheta Sen, Sourav Saha +2 more

Artificial Intelligence · Research Repository (Delft University of Technology) · · cited 40× · DOI ↗ · more like this ·

This tutorial presents explainable information retrieval (ExIR), an emerging area focused on fostering responsible and trustworthy deployment of machine learning systems in the context of information retrieval. As the field has rapidly evolved in the past 4-5 years, numerous approaches have been proposed that focus on different access modes, stakeholders, and model development stages. This tutorial aims to introduce IR-centric notions, classification, and evaluation styles in ExIR, while focusing on IR-specific tasks such as ranking, text classification, and learning-to-rank systems. We will delve into method families and their adaptations to IR, extensively covering post-hoc methods, axiomatic and probing approaches, and recent advances in interpretability-by-design approaches. We will also discuss ExIR applications for different stakeholders, such as researchers, practitioners, and…

The Curious Case of IR Explainability: Explaining Document Scores within and across Ranking Models

Procheta Sen, Debasis Ganguly, Manisha Verma +1 more

Artificial Intelligence · · cited 15× · DOI ↗ · more like this ·

It is often useful for an IR practitioner to analyze the similarity function of an IR model, or for a non-technical search engine user to understand why a document was shown at a certain rank, in terms of the three fundamental aspects of a similarity function, namely the a) frequency of a term in a document, b) frequency of a term in a collection and c) the length of a document. We propose a general methodology of approximating an IR model as the coefficients of a linear function of these three fundamental aspects (and an additional aspect of semantic similarity between terms for neural models), which potentially can help IR practitioners to optimize the relative importance of each aspect on specific document collection and types of queries. Our analysis shows that the coefficients, which represent the relative importance of the three fundamental aspects, are useful to compare a model's…

A case study for automated attribute extraction from legal documents using large language models

Subinay Adhikary, Procheta Sen, Dwaipayan Roy +1 more

Artificial Intelligence · Artificial Intelligence and Law · · cited 14× · DOI ↗ · more like this ·

Abstract The escalating number of pending cases is a growing concern worldwide. Recent advancements in digitization have opened up possibilities for leveraging artificial intelligence (AI) tools in the processing of legal documents. Adopting a structured representation for legal documents, as opposed to a mere bag-of-words flat text representation, can significantly enhance processing capabilities. With the aim of achieving this objective, we put forward a set of diverse attributes for criminal case proceedings. To enhance the effectiveness of automatically extracting these attributes from legal documents within a sequence labeling framework, we propose the utilization of a few-shot learning approach based on Large Language Models (LLMs). Moreover, we demonstrate the efficacy of the extracted attributes in downstream tasks, such as legal judgment prediction and legal statute prediction .

Tempo-Lexical Context Driven Word Embedding for Cross-Session Search Task Extraction

Procheta Sen, Debasis Ganguly, Gareth J. F. Jones

Artificial Intelligence · Dublin City University Open Access Institutional Repository (Dublin City University) · · cited 11× · DOI ↗ · more like this ·

Procheta Sen, Debasis Ganguly, Gareth Jones. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.

I Know What You Need: Investigating Document Retrieval Effectiveness with Partial Session Contexts

Procheta Sen, Debasis Ganguly, Gareth J. F. Jones

Information Systems · ACM Transactions on Information Systems · · cited 8× · DOI ↗ · more like this ·

Reducing user effort in finding relevant information is one of the key objectives of search systems. Existing approaches have been shown to effectively exploit the context from the current search session of users for automatically suggesting queries to reduce their search efforts. However, these approaches do not accomplish the end goal of a search system—that of retrieving a set of potentially relevant documents for the evolving information need during a search session. This article takes the problem of query prediction one step further by investigating the problem of contextual recommendation within a search session. More specifically, given the partial context information of a session in the form of a small number of queries, we investigate how a search system can effectively predict the documents that a user would have been presented with had he continued the search session by…

Overview of RCD-2020, the FIRE-2020 track on Retrieval from Conversational Dialogues

Debasis Ganguly, Dipasree Pal, Manisha Verma +1 more

Artificial Intelligence · Forum for Information Retrieval Evaluation · · cited 8× · DOI ↗ · more like this ·

This paper describes an overview of the track - ’Retrieval from Conversational Dialogues’ (RCD) organized as a part of Forum of Information Retrieval and Evaluation (FIRE), 2020. The motivation of the track is to develop a dataset towards a controlled and reproducible laboratory based experimental setup for investigating the effectiveness if conversational assistance systems. Specifically, the manner of conversational assistance which this track addresses is contextualization of certain concepts within the content either written (e.g. a chat system) or uttered (e.g. in an audio or video communication) by a user about which the other users participating in the communication are not well versed. To study the problem under a laboratory-based reproducible setting, we took a collection of four movie scripts and manually annotated spans of text that may require contextualization. The two…

Procrastination is the Thief of Time

Procheta Sen, Debasis Ganguly, Gareth J. F. Jones

Information Systems · Dublin City University Open Access Institutional Repository (Dublin City University) · · cited 7× · DOI ↗ · more like this ·

Users of current search systems actively interact with the system to complete their search task. This can encompass formulating and reformulating a series queries expressing evolving of different information needs. We believe that the next generation of search systems will see a shift towards proactive understanding of user intent based on analysis of user activities. Such a proactive search system could start recommending documents that are likely to help users accomplish their tasks without requiring them to explicitly submit queries to the system. We propose a framework to evaluate such a search system. The key idea behind our proposed metric is to aggregate a correlation measure over a search session between the expected outcome, which in this case refers to the list of documents retrieved with a true user query, and the predicted outcome, which refers to the list of documents…

Measuring and Comparing the Consistency of IR Models for Query Pairs with Similar and Different Information Needs

Procheta Sen, Sourav Saha, Debasis Ganguly +2 more

Information Systems · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · · cited 6× · DOI ↗ · more like this ·

A widespread use of supervised ranking models has necessitated an investigation on how consistent their outputs align with user expectations. While a match between the user expectations and system outputs can be sought at different levels of granularity, we study this alignment for search intent transformation across a pair of queries. Specifically, we propose a consistency metric, which for a given pair of queries - one reformulated from the other with at least one term in common, measures if the change in the set of the top-retrieved documents induced by this reformulation is as per a user's expectation. Our experiments led to a number of observations, such as DRMM (an early interaction based IR model) exhibits better alignment with set-level user expectations, whereas transformer-based neural models (e.g., MonoBERT) agree more consistently with the content and rank-based expectations…

Word-

Procheta Sen, Debasis Ganguly, Gareth J. F. Jones

Artificial Intelligence · · cited 4× · DOI ↗ · more like this ·

Procheta Sen, Debasis Ganguly, Gareth Jones. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.

CROWD: Certified Robustness via Weight Distribution for Smoothed Classifiers against Backdoor Attack

Siqi Sun, Procheta Sen, Wenjie Ruan

Artificial Intelligence · · cited 2× · DOI ↗ · more like this ·

Language models are vulnerable to clandestinely modified data and manipulation by attackers.Despite considerable research dedicated to enhancing robustness against adversarial attacks, the realm of provable robustness for backdoor attacks remains relatively unexplored.In this paper, we initiate a pioneering investigation into the certified robustness of NLP models against backdoor triggers.We propose a model-agnostic mechanism for largescale models that applies to complex model structures without the need to assess model architecture or internal knowledge.More importantly, we take recent advances in randomized smoothing theory and propose a novel weightbased distribution algorithm to enable semantic similarity and provide theoretical robustness guarantees.Experimentally, we demonstrate the efficacy of our approach across a diverse range of datasets and tasks, highlighting its utility in…

Explainability of Text Processing and Retrieval Methods

Avishek Anand, Sourav Saha, Procheta Sen +1 more

Artificial Intelligence · · cited 2× · DOI ↗ · more like this ·

This tutorial presents explainability of text processing and retrieval methods, an emerging area focused on fostering responsible and trustworthy deployment of machine learning systems in the context of information retrieval. As the field has rapidly evolved in the past 4-5 years, numerous approaches have been proposed that focus on different access modes, stakeholders, and model development stages. This tutorial aims to introduce IR-centric notions, classification, and evaluation styles in explainable information retrieval (ExIR) while focusing on IR-specific tasks such as ranking, text classification, and learning-to-rank systems. We will delve into method families and their adaptations to IR, extensively covering post-hoc methods, axiomatic and probing approaches, and recent advances in interpretability-by-design approaches. We will also discuss ExIR applications for different…

Automated Argument Generation from Legal Facts

Oscar Tuvey, Procheta Sen

cs.AI · · cited 2× · arXiv ↗ · more like this ·

The count of pending cases has shown an exponential rise across nations (e.g., with more than 10 million pending cases in India alone). The main issue lies in the fact that the number of cases submitted to the law system is far greater than the available number of legal professionals present in a country. Given this worldwide context, the utilization of AI technology has gained paramount importance to enhance the efficiency and speed of legal procedures. In this study we partcularly focus on helping legal professionals in the process of analyzing a legal case. Our specific investigation delves into harnessing the generative capabilities of open-sourced large language models to create arguments derived from the facts present in legal cases. Experimental results show that the generated arguments from the best performing method have on average 63% overlap with the benchmark set gold…

Task2KB: A Public Task-Oriented Knowledge Base

Procheta Sen, Xi Wang, Ruiqing Xu +1 more

cs.IR · · cited 2× · arXiv ↗ · more like this ·

Search engines and conversational assistants are commonly used to help users complete their every day tasks such as booking travel, cooking, etc. While there are some existing datasets that can be used for this purpose, their coverage is limited to very few domains. In this paper, we propose a novel knowledge base, 'Task2KB', which is constructed using data crawled from WikiHow, an online knowledge resource offering instructional articles on a wide range of tasks. Task2KB encapsulates various types of task-related information and attributes, such as requirements, detailed step description, and available methods to complete tasks. Due to its higher coverage compared to existing related knowledge graphs, Task2KB can be highly useful in the development of general purpose task completion assistants

A Word Sense Distribution-based approach for Semantic Change Prediction

Xiaohang Tang, Yi Zhou, Taichi Aida +2 more

Artificial Intelligence · ORCA Online Research @Cardiff (Cardiff University) · · cited 2× · DOI ↗ · more like this ·

Semantic Change Detection of words is an important task for various NLP applications that must make time-sensitive predictions. Some words are used over time in novel ways to express new meanings, and these new meanings establish themselves as novel senses of existing words. On the other hand, Word Sense Disambiguation (WSD) methods associate ambiguous words with sense ids, depending on the context in which they occur. Given this relationship between WSD and SCD, we explore the possibility of predicting whether a target word has its meaning changed between two corpora collected at different time steps, by comparing the distributions of senses of that word in each corpora. For this purpose, we use pretrained static sense embeddings to automatically annotate each occurrence of the target word in a corpus with a sense id. Next, we compute the distribution of sense ids of a target word in a…

Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning

Procheta Sen, Debasis Ganguly

cs.CY · · cited 2× · arXiv ↗ · more like this ·

Human society had a long history of suffering from cognitive biases leading to social prejudices and mass injustice. The prevalent existence of cognitive biases in large volumes of historical data can pose a threat of being manifested as unethical and seemingly inhuman predictions as outputs of AI systems trained on such data. To alleviate this problem, we propose a bias-aware multi-objective learning framework that given a set of identity attributes (e.g. gender, ethnicity etc.) and a subset of sensitive categories of the possible classes of prediction outputs, learns to reduce the frequency of predicting certain combinations of them, e.g. predicting stereotypes such as `most blacks use abusive language', or `fear is a virtue of women'. Our experiments conducted on an emotion prediction task with balanced class priors shows that a set of baseline bias-agnostic models exhibit cognitive…

Evaluation of Personalised Information Retrieval at CLEF 2017 (PIR-CLEF): Towards a Reproducible Evaluation Framework for PIR.

Gabriella Pasi, Gareth J. F. Jones, Stefania Marrara +3 more

Artificial Intelligence · BOA (University of Milano-Bicocca) · · cited 2× · OpenAlex ↗ · more like this ·

The Personalised Information Retrieval (PIR-CLEF) Lab workshop at CLEF 2017 is designed to provide a forum for the exploration of methodologies for the repeatable evaluation of personalised information retrieval (PIR). The PIR-CLEF 2017 Lab provides a preliminary pilot edition of a Lab task dedicated to personalised search, while the workshop at the conference is intended to provide a forum for the discussion of strategies for the evaluation of PIR and extension of the pilot Lab task. The PIR-CLEF 2017 Pilot Task is the first evaluation benchmark based on the Cranfield paradigm, with the potential benefits of producing evaluation results that are easily reproducible. The task is based on search sessions over a subset of the ClueWeb12 collection, undertaken by 10 users by using a clearly defined and novel methodology. The collection provides data gathered by the activities undertaken…

Adaptive Retrieval-Augmented Generation for Conversational Systems

Xi Wang, Procheta Sen, Ruizhe Li +1 more

cs.CL · · cited 1× · arXiv ↗ · more like this ·

Despite the success of integrating large language models into the development of conversational systems, many studies have shown the effectiveness of retrieving and augmenting external knowledge for informative responses. Hence, many existing studies commonly assume the always need for Retrieval Augmented Generation (RAG) in a conversational system without explicit control. This raises a research question about such a necessity. In this study, we propose to investigate the need for each turn of system response to be augmented with external knowledge. In particular, by leveraging human judgements on the binary choice of adaptive augmentation, we develop RAGate, a gating model, which models conversation context and relevant inputs to predict if a conversational system requires RAG for improved responses. We conduct extensive experiments on devising and applying RAGate to conversational…

Lexical Entrainment for Conversational Systems

Zhengxiang Shi, Procheta Sen, Aldo Lipani

cs.CL · · cited 1× · arXiv ↗ · more like this ·

Conversational agents have become ubiquitous in assisting with daily tasks, and are expected to possess human-like features. One such feature is lexical entrainment (LE), a phenomenon in which speakers in human-human conversations tend to naturally and subconsciously align their lexical choices with those of their interlocutors, leading to more successful and engaging conversations. As an example, if a digital assistant replies 'Your appointment for Jinling Noodle Pub is at 7 pm' to the question 'When is my reservation for Jinling Noodle Bar today?', it may feel as though the assistant is trying to correct the speaker, whereas a response of 'Your reservation for Jinling Noodle Bar is at 7 pm' would likely be perceived as more positive. This highlights the importance of LE in establishing a shared terminology for maximum clarity and reducing ambiguity in conversations. However, we…

Report on the 2nd Symposium on Artificial Intelligence and Law (SAIL) 2022

Saptarshi Ghosh, Kripabandhu Ghosh, Debasis Ganguly +9 more

Political Science and International Relations · ACM SIGIR Forum · · cited 1× · DOI ↗ · more like this ·

This report describes the 2 nd edition of the Symposium on Artificial Intelligence and Law (SAIL) organized as a virtual event during June 6--9, 2022. The aim of SAIL is to bring together experts from the industry and the academia to discuss the scope and future of AI as applied to the legal domain. The symposium is also meant to foster collaborations between researchers of the following communities: Law, Artificial Intelligence, Data Mining, Information Retrieval, and Natural Language Processing. Eminent researchers working on AI and Law in both the academia and the industry were invited to deliver talks at this symposium. Date: 6--9 June, 2022. Website: https://sites.google.com/view/sail-2022/.

Overview of the Supporting and Understanding of Conversational Dialogues (SUD) Workshop

Debasis Ganguly, Manisha Verma, Procheta Sen +2 more

Artificial Intelligence · · cited 1× · DOI ↗ · more like this ·

The workshop on Supporting and Understanding of (multi-party) conversational Dialogues (SUD) seeks to encourage researchers to investigate automated methods to analyze and understand conversations, and also explore methodologies for proactively providing assistance to the communicating parties during conversations, ranging from summarizing the minutes of meetings to automatically keeping track of action items etc. The workshop will have (1) a regular research paper track, and a more focused (2) data challenge track, inviting papers on a specific task of contextualizing entities of interest from conversation dialogues.

Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Models

Lingfang Li, Procheta Sen, Shubham Das +1 more

cs.AI · · cited 0× · arXiv ↗ · more like this ·

Fine-tuning has emerged as a widely adopted approach for adapting LLMs to a variety of downstream tasks. However, how it reshapes their internal mechanisms remains poorly understood. To address this, we investigate how fine-tuning alters internal representations in LLMs, including attention patterns and layer-wise activations, and examine whether these changes are linked to task-relevant components identified by EAP (e.g., attention heads and logit-level activations) that drive task performance. We find that EAP-identified components are concentrated within specific layers, indicating a degree of functional localisation in how models internalise task-specific behavior. Notably, the distribution of these components across layers is largely uncorrelated with the layers undergoing the most substantial representational changes during fine-tuning. Furthermore, we observe that overlap in…

Report on the 3rd Symposium on NLP for Social Good (NSG 2025)

Procheta Sen, Tulika Saha, Danushka Bollegala

Sociology and Political Science · ACM SIGIR Forum · · cited 0× · DOI ↗ · more like this ·

Artificial intelligence (AI), specifically, Natural Language Processing (NLP) is being hailed as a new breeding ground for immense innovation potential. Researchers believe that NLP-based technologies could help to solve societal issues such as equality and inclusion, education, health, and hunger, climate action etc., and many more. Tackling these questions requires a concerted, collaborative effort across all sectors of society. The 3rd Symposium on NLP for Social Good (NSG) was a novel effort that aimed to enable NLP researchers and scholars from inter-disciplinary field who want to think about the societal implications of their work for solving humanitarian and environmental challenges. The objective of the symposium was to support fundamental research and engineering efforts and empower the social sector with tools and resources, while collaborating with partners from all sectors…

MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses

Tong Chen, Zimu Wang, Yiyi Miao +6 more

cs.CL · · cited 0× · arXiv ↗ · more like this ·

Medical fact-checking has become increasingly critical as more individuals seek medical information online. However, existing datasets predominantly focus on human-generated content, leaving the verification of content generated by large language models (LLMs) relatively unexplored. To address this gap, we introduce MedFact, the first evidence-based Chinese medical fact-checking dataset of LLM-generated medical content. It consists of 1,321 questions and 7,409 claims, mirroring the complexities of real-world medical scenarios. We conduct comprehensive experiments in both in-context learning (ICL) and fine-tuning settings, showcasing the capability and challenges of current LLMs on this task, accompanied by an in-depth error analysis to point out key directions for future research. Our dataset is publicly available at https://github.com/AshleyChenNLP/MedFact.

Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective

Bhavik Chandna, Zubair Bashir, Procheta Sen

cs.CL · · cited 0× · arXiv ↗ · more like this ·

Large Language Models (LLMs) are known to exhibit social, demographic, and gender biases, often as a consequence of the data on which they are trained. In this work, we adopt a mechanistic interpretability approach to analyze how such biases are structurally represented within models such as GPT-2 and Llama2. Focusing on demographic and gender biases, we explore different metrics to identify the internal edges responsible for biased behavior. We then assess the stability, localization, and generalizability of these components across dataset and linguistic variations. Through systematic ablations, we demonstrate that bias-related computations are highly localized, often concentrated in a small subset of layers. Moreover, the identified components change across fine-tuning settings, including those unrelated to bias. Finally, we show that removing these components not only reduces biased…

Unraveling the Influence of Training Data and Internal Structures in Large Language Models for Enhanced Explainability (Student Abstract)

Lingfang Li, Procheta Sen

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 0× · DOI ↗ · more like this ·

Recent advances in deep learning have expanded the application of large language models (LLMs) across fields such as medicine, finance, and education. Understanding the mechanisms underlying these models is essential to mitigate issues like hallucinations and bias. This study provides deep learning practitioners with insights into how specific training data points and internal structures influence model behaviour. Using influence functions and mechanistic interpretability, we will analyze the impact of data on model predictions across various tasks. Preliminary findings indicate that semantic search techniques, such as FAISS, enable efficient identification of influential training points in GPT-2 small. Future work will extend these methods to additional tasks and more complex models, with a focus on further elucidating LLM structures to improve interpretability.