arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Ronay Ak

18 papers · 2017–2026

2017: 12018: 12019: 02020: 02021: 32022: 42023: 02024: 22025: 52026: 2
2017peak 5 in 20252026

Frequent co-authors

Papers

showing 1–18 of 18 RSS

Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation

Gabriel de Souza Pereira Moreira, Sara Rabhi, Jeong Min Lee +2 more

Artificial Intelligence · · cited 171× · DOI ↗ · more like this ·

Much of the recent progress in sequential and session-based recommendation has been driven by improvements in model architecture and pretraining techniques originating in the field of Natural Language Processing. Transformer architectures in particular have facilitated building higher-capacity models and provided data augmentation and training techniques which demonstrably improve the effectiveness of sequential recommendation. But with a thousandfold more research going on in NLP, the application of transformers for recommendation understandably lags behind. To remedy this we introduce Transformers4Rec, an open-source library built upon HuggingFace’s Transformers library with a similar goal of opening up the advances of NLP based Transformers to the recommender system community and making these advancements immediately accessible for the tasks of sequential and session-based…

Gaussian Process Regression (GPR) Representation in Predictive Model Markup Language (PMML)

Jinwoo Park, David Lechevalier, Ronay Ak +4 more

Artificial Intelligence · Smart and Sustainable Manufacturing Systems · · cited 53× · DOI ↗ · more like this ·

This paper describes Gaussian process regression (GPR) models presented in predictive model markup language (PMML). PMML is an extensible-markup-language (XML) -based standard language used to represent data-mining and predictive analytic models, as well as pre- and post-processed data. The previous PMML version, PMML 4.2, did not provide capabilities for representing probabilistic (stochastic) machine-learning algorithms that are widely used for constructing predictive models taking the associated uncertainties into consideration. The newly released PMML version 4.3, which includes the GPR model, provides new features: confidence bounds and distribution for the predictive estimations. Both features are needed to establish the foundation for uncertainty quantification analysis. Among various probabilistic machine-learning algorithms, GPR has been widely used for approximating a target…

Building and Deploying a Multi-Stage Recommender System with Merlin

Karl Higley, Even Oldridge, Ronay Ak +2 more

Information Systems · · cited 11× · DOI ↗ · more like this ·

Newcomers to recommender systems often face challenges related to their lack of understanding of how these systems operate in real life. In most online content related to this topic, the focus is on models and algorithms that score items based on the user’s preferences. However, the recommender model alone does not comprise everything needed for serving optimized recommender systems that meet the company’s business objectives. An industry-standard recommender system involves a number of steps, including data preprocessing, defining and training recommender models, as well as filtering and business logic for serving. In this work, we propose the four-stage recommender system, an industry-wide design pattern we have identified for production recommender systems. The four-stage pipeline includes an item retrieval step that prepares a small subset of relevant items for scoring. The…

Transformers with multi-modal features and post-fusion context for e-commerce session-based recommendation

Gabriel de Souza P. Moreira, Sara Rabhi, Ronay Ak +2 more

cs.IR · · cited 9× · arXiv ↗ · more like this ·

Session-based recommendation is an important task for e-commerce services, where a large number of users browse anonymously or may have very distinct interests for different sessions. In this paper we present one of the winning solutions for the Recommendation task of the SIGIR 2021 Workshop on E-commerce Data Challenge. Our solution was inspired by NLP techniques and consists of an ensemble of two Transformer architectures - Transformer-XL and XLNet - trained with autoregressive and autoencoding approaches. To leverage most of the rich dataset made available for the competition, we describe how we prepared multi-model features by combining tabular events with textual and image vectors. We also present a model prediction analysis to better understand the effectiveness of our architectures for the session-based recommendation.

Training and Deploying Multi-Stage Recommender Systems

Ronay Ak, Benedikt Schifferer, Sara Rabhi +1 more

Artificial Intelligence · · cited 4× · DOI ↗ · more like this ·

Industrial recommender systems are made up of complex pipelines requiring multiple steps including feature engineering and preprocessing, a retrieval model for candidate generation, filtering, a feature store query, a ranking model for scoring, and an ordering stage. These pipelines need to be carefully deployed as a set, requiring coordination during their development and deployment. Data scientists, ML engineers, and researchers might focus on different stages of recommender systems, however they share a common desire to reduce the time and effort searching for and combining boilerplate code coming from different sources or writing custom code from scratch to create their own RecSys pipelines.

End-to-End Session-Based Recommendation on GPU

Gabriel de Souza Pereira Moreira, Sara Rabhi, Ronay Ak +1 more

Information Systems · · cited 4× · DOI ↗ · more like this ·

In recent years, several deep learning-based algorithms have been proposed for recommendation systems while its adoption in industry deployments have been steeply growing. In particular, NLP-inspired approaches have been successfully adapted for sequential and session-based recommendation problems, which are important for many domains like e-commerce, news and streaming media. In this regard, this hands-on tutorial will offer to the participants: (I) an introduction on the main concepts and algorithms for session-based recommendation, (II) how to build, train and evaluate a session-based recommendation model based on RNN and Transformer architectures, and (III) how to speed up with GPUs the entire RecSys pipeline which encompasses feature engineering, preprocessing, training, evaluation and inference using NVIDIA Merlin - an open source ecosystem for large-scale deep learning…

Beyond Semantic Similarity: Performance and Costs of Agentic Retrieval for Complex Tasks

Reza Esfandiarpoor, Radek Osmulski, Yauhen Babakhin +8 more

cs.IR · · cited 2× · arXiv ↗ · more like this ·

Modern information systems, including many agentic workflows, use dense retrieval to explore large amounts of unstructured data. However, dense retrieval relies on surface-level semantic similarity, which is insufficient for increasingly complex search applications. Here, we investigate agentic retrieval that combines the reasoning capabilities of Large Language Models (LLMs) with the efficient corpus exploration of retrievers in a ReAct agentic loop to solve complex retrieval tasks. In our experiments, we show that agentic retrieval is more effective than standard retrieval, improving nDCG@10 by 8.7 points using the same embedding model. Moreover, while specialized retrieval methods struggle on out-of-domain tasks, agentic retrieval is highly generalizable: the same pipeline achieves competitive results on both the ViDoRe v3 and BRIGHT leaderboards. However, this improvement comes at a…

A Diverse Models Ensemble for Fashion Session-Based Recommendation

Benedikt Schifferer, Jiwei Liu, Sara Rabhi +5 more

Information Systems · · cited 2× · DOI ↗ · more like this ·

Session-based recommendation is an important task for domains like e-commerce, that suffer from the user cold-start problem due to anonymous browsing and for which users preferences might change considerably over time. The RecSys Challenge 2022, organized by Dressipi, is focused on the session-based recommendation problem for the fashion e-commerce domain. In this paper, the NVIDIA RAPIDS and NVIDIA Merlin teams present their solution that placed 3rd in the challenge. Among the most effective techniques we found sessions augmentation and ensembling a very diverse set of statistical, machine learning and deep learning models. Our recommendation pipeline is composed of three stages, where the first level is focused on candidate generation and the others refine the recommendation ranking for more robust and accurate recommendations.

Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG

Gabriel de Souza P. Moreira, Ronay Ak, Benedikt Schifferer +3 more

cs.IR · · cited 1× · arXiv ↗ · more like this ·

Ranking models play a crucial role in enhancing overall accuracy of text retrieval systems. These multi-stage systems typically utilize either dense embedding models or sparse lexical indices to retrieve relevant passages based on a given query, followed by ranking models that refine the ordering of the candidate passages by its relevance to the query. This paper benchmarks various publicly available ranking models and examines their impact on ranking accuracy. We focus on text retrieval for question-answering tasks, a common use case for Retrieval-Augmented Generation systems. Our evaluation benchmarks include models some of which are commercially viable for industrial applications. We introduce a state-of-the-art ranking model, NV-RerankQA-Mistral-4B-v3, which achieves a significant accuracy increase of ~14% compared to pipelines with other rerankers. We also provide an ablation study…

NV-Retriever: Improving text embedding models with effective hard-negative mining

Gabriel de Souza P. Moreira, Radek Osmulski, Mengyao Xu +3 more

cs.IR · · cited 1× · arXiv ↗ · more like this ·

Text embedding models have been popular for information retrieval applications such as semantic search and Question-Answering systems based on Retrieval-Augmented Generation (RAG). Those models are typically Transformer models that are fine-tuned with contrastive learning objectives. One of the challenging aspects of fine-tuning embedding models is the selection of high quality hard-negative passages for contrastive learning. In this paper we introduce a family of positive-aware mining methods that use the positive relevance score as an anchor for effective false negative removal, leading to faster training and more accurate retrieval models. We provide an ablation study on hard-negative mining methods over their configurations, exploring different teacher and base models. We further demonstrate the efficacy of our proposed mining methods at scale with the NV-Retriever-v1 model, which…

Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model for Text, Image, Audio, and Video

Mengyao Xu, Wenfei Zhou, Yauhen Babakhin +6 more

cs.CL · · cited 0× · arXiv ↗ · more like this ·

We present Omni-Embed-Nemotron, a unified multimodal retrieval embedding model developed to handle the increasing complexity of real-world information needs. While Retrieval-Augmented Generation (RAG) has significantly advanced language models by incorporating external knowledge, existing text-based retrievers rely on clean, structured input and struggle with the visually and semantically rich content found in real-world documents such as PDFs, slides, or videos. Recent work such as ColPali has shown that preserving document layout using image-based representations can improve retrieval quality. Building on this, and inspired by the capabilities of recent multimodal models such as Qwen2.5-Omni, we extend retrieval beyond text and images to also support audio and video modalities. Omni-Embed-Nemotron enables both cross-modal (e.g., text - video) and joint-modal (e.g., text - video+audio)…

Boost the Performance of Tabular Data Models with GPU Accelerated Feature Engineering

Chris Deotte, Ronay Ak

Artificial Intelligence · · cited 0× · DOI ↗ · more like this ·

Feature engineering remains a crucial technique for improving the performance of models trained on tabular data. Unlike computer vision and natural language processing, where deep learning models automatically extract hierarchical features from raw data, the most accurate tabular models, such as gradient boosted decision trees, still benefit significantly from manually crafted features. This is demonstrated in Team NVIDIA's many first-place data science competition victories.

Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model

Mengyao Xu, Gabriel Moreira, Ronay Ak +5 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Motivated by the growing demand for retrieval systems that operate across modalities, we introduce llama-nemoretriever-colembed, a unified text-image retrieval model that delivers state-of-the-art performance across multiple benchmarks. We release two model variants, 1B and 3B. The 3B model achieves state of the art performance, scoring NDCG@5 91.0 on ViDoRe V1 and 63.5 on ViDoRe V2, placing first on both leaderboards as of June 27, 2025. Our approach leverages the NVIDIA Eagle2 Vision-Language model (VLM), modifies its architecture by replacing causal attention with bidirectional attention, and integrates a ColBERT-style late interaction mechanism to enable fine-grained multimodal retrieval in a shared embedding space. While this mechanism delivers superior retrieval accuracy, it introduces trade-offs in storage and efficiency. We provide a comprehensive analysis of these trade-offs.…

MIRACL-VISION: A Large, multilingual, visual document retrieval benchmark

Radek Osmulski, Gabriel de Souza P. Moreira, Ronay Ak +3 more

cs.IR · · cited 0× · arXiv ↗ · more like this ·

Document retrieval is an important task for search and Retrieval-Augmented Generation (RAG) applications. Large Language Models (LLMs) have contributed to improving the accuracy of text-based document retrieval. However, documents with complex layout and visual elements like tables, charts and infographics are not perfectly represented in textual format. Recently, image-based document retrieval pipelines have become popular, which use visual large language models (VLMs) to retrieve relevant page images given a query. Current evaluation benchmarks on visual document retrieval are limited, as they primarily focus only English language, rely on synthetically generated questions and offer a small corpus size. Therefore, we introduce MIRACL-VISION, a multilingual visual document retrieval evaluation benchmark. MIRACL-VISION covers 18 languages, and is an extension of the MIRACL dataset, a…

Reducing the Friction for Building Recommender Systems with Merlin

Sara Rabhi, Ronay Ak, Marc Romeijn +2 more

Information Systems · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · · cited 0× · DOI ↗ · more like this ·

Recommender Systems (RecSys) are the engine of the modern internet and the catalyst for human decisions. The goal of a recommender system is to generate relevant recommendations for users from a collection of items or services that might interest them. Building a recommendation system is challenging because it requires multiple stages (item retrieval, filtering, ranking, ordering) to work together seamlessly and efficiently during training and inference. The biggest challenges faced by new practitioners are the lack of understanding around what RecSys look like in the real world and the difficulty in transitioning from the simple Matrix Factorization (MF) to more complex deep learning architectures with multiple input features, neural components and prediction heads.

Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrieval

Gabriel de Souza P. Moreira, Ronay Ak, Mengyao Xu +9 more

cs.IR · · arXiv ↗ · more like this ·

Retrieval-Augmented Generation (RAG) systems have been popular for generative applications, powering language models by injecting external knowledge. Companies have been trying to leverage their large catalog of documents (e.g. PDFs, presentation slides) in such RAG pipelines, whose first step is the retrieval component. Dense retrieval has been a popular approach, where embedding models are used to generate a dense representation of the user query that is closer to relevant content embeddings. More recently, VLM-based embedding models have become popular for visual document retrieval, as they preserve visual information and simplify the indexing pipeline compared to OCR text extraction. Motivated by the growing demand for visual document retrieval, we introduce Nemotron ColEmbed V2, a family of models that achieve state-of-the-art performance on the ViDoRe benchmarks. We release three…

Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks

Yauhen Babakhin, Radek Osmulski, Ronay Ak +5 more

cs.CL · · arXiv ↗ · more like this ·

We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show strong performance, their training data or methodologies are often not fully disclosed. We aim to address this by developing a fully open-source model, publicly releasing its weights and detailed ablation studies, and planning to share the curated training datasets. Our model demonstrates superior performance across all major embedding tasks -- including retrieval, classification and semantic textual similarity (STS) -- and excels in challenging multilingual scenarios, such as low-resource languages and cross-lingual setups. This state-of-the-art performance is driven by a novel data mix of 16.1 million query-document pairs, split between 7.7 million…

Detection and Segmentation of Manufacturing Defects with Convolutional Neural Networks and Transfer Learning

Max Ferguson, Ronay Ak, Yung-Tsun Tina Lee +1 more

cs.CV · · arXiv ↗ · more like this ·

Quality control is a fundamental component of many manufacturing processes, especially those involving casting or welding. However, manual quality control procedures are often time-consuming and error-prone. In order to meet the growing demand for high-quality products, the use of intelligent visual inspection systems is becoming essential in production lines. Recently, Convolutional Neural Networks (CNNs) have shown outstanding performance in both image classification and localization tasks. In this article, a system is proposed for the identification of casting defects in X-ray images, based on the Mask Region-based CNN architecture. The proposed defect detection system simultaneously performs defect detection and segmentation on input images, making it suitable for a range of defect detection tasks. It is shown that training the network to simultaneously perform defect detection and…