arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Vikas

4,856 papers · 1951–2026

1967: 01968: 01969: 01970: 01971: 01972: 01973: 01974: 01975: 11976: 01977: 01978: 01979: 01980: 01981: 01982: 11983: 01984: 11985: 01986: 01987: 01988: 31989: 11990: 11991: 21992: 11993: 21994: 11995: 31996: 61997: 61998: 21999: 22000: 92001: 92002: 242003: 292004: 242005: 432006: 392007: 472008: 562009: 642010: 852011: 1022012: 1332013: 1702014: 2012015: 1952016: 1642017: 2092018: 2202019: 2322020: 3152021: 3672022: 3452023: 4312024: 6212025: 5272026: 153
1967peak 621 in 20242026

Frequent co-authors

Papers

showing 1–25 of 4,856 RSS

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Gemini Team, Petko Georgiev, Ving Ian Lei +1134 more

cs.CL · · cited 4,033× · arXiv ↗ · more like this ·

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's…

Federated Learning with Non-IID Data

Yue Zhao, Meng Li, Liangzhen Lai +3 more

cs.LG · · cited 3,495× · arXiv ↗ · more like this ·

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55% for neural networks trained for highly skewed non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve…

Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples

Mikhail Belkin, Partha Niyogi, Vikas Sindhwani

Artificial Intelligence · Journal of Machine Learning Research · · cited 3,273× · OpenAlex ↗ · more like this ·

We propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution. We focus on a semi-supervised framework that incorporates labeled and unlabeled data in a general-purpose learner. Some transductive graph learning algorithms and standard methods including Support Vector Machines and Regularized Least Squares can be obtained as special cases. We utilize properties of Reproducing Kernel Hilbert spaces to prove new Representer theorems that provide theoretical basis for the algorithms. As a result (in contrast to purely graph-based approaches) we obtain a natural out-of-sample extension to novel examples and so are able to handle both transductive and truly semi-supervised settings. We present experimental evidence suggesting that our semi-supervised algorithms are able to use unlabeled data effectively.…

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

BigScience Workshop, :, Teven Le Scao +391 more

cs.CL · · cited 3,043× · arXiv ↗ · more like this ·

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we…

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448 more

cs.CL · · cited 2,670× · arXiv ↗ · more like this ·

Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 450 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, childhood development, math, common-sense reasoning, biology, physics, social bias, software development, and beyond. BIG-bench focuses on tasks that are believed to be beyond the capabilities of…

Gemma 2: Improving Open Language Models at a Practical Size

Gemma Team, Morgane Riviere, Shreya Pathak +195 more

cs.CL · · cited 2,513× · arXiv ↗ · more like this ·

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

A Survey on IoT Security: Application Areas, Security Threats, and Solution Architectures

Vikas Hassija, Vinay Chamola, Vikas Saxena +3 more

Computer Networks and Communications · IEEE Access · · cited 1,371× · DOI ↗ · more like this ·

The Internet of Things (IoT) is the next era of communication. Using the IoT, physical objects can be empowered to create, receive, and exchange data in a seamless manner. Various IoT applications focus on automating different tasks and are trying to empower the inanimate physical objects to act without any human intervention. The existing and upcoming IoT applications are highly promising to increase the level of comfort, efficiency, and automation for the users. To be able to implement such a world in an ever-growing fashion requires high security, privacy, authentication, and recovery from attacks. In this regard, it is imperative to make the required changes in the architecture of the IoT applications for achieving end-to-end secure IoT environments. In this paper, a detailed review of the security-related challenges and sources of threat in the IoT applications is presented. After…

InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

Fan-Yun Sun, Jordan Hoffmann, Vikas Verma +1 more

cs.LG · · cited 1,051× · arXiv ↗ · more like this ·

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the…

Learning From Crowds

Vikas C. Raykar, Shipeng Yu, Linda Zhao +4 more

Computer Science Applications · Journal of Machine Learning Research · · cited 1,009× · DOI ↗ · more like this ·

For many supervised learning tasks it may be infeasible (or very expensive) to obtain objective and reliable labels. Instead, we can collect subjective (possibly noisy) labels from multiple experts or annotators. In practice, there is a substantial amount of disagreement among the annotators, and hence it is of great practical interest to address conventional supervised learning problems in this scenario. In this paper we describe a probabilistic approach for supervised learning when we have multiple annotators providing (possibly noisy) labels but no absolute gold standard. The proposed algorithm evaluates the different experts and also gives an estimate of the actual hidden labels. Experimental results indicate that the proposed method is superior to the commonly used majority voting baseline.

Interpolation Consistency Training for Semi-Supervised Learning

Vikas Verma, Kenji Kawaguchi, Alex Lamb +4 more

stat.ML · · cited 942× · arXiv ↗ · more like this ·

We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets. Our theoretical analysis shows that ICT corresponds to a certain type of data-adaptive regularization with unlabeled points which reduces overfitting to labeled points under high confidence values.

Gemini: A Family of Highly Capable Multimodal Models

Gemini Team, Rohan Anil, Sebastian Borgeaud +1348 more

cs.CL · · cited 816× · arXiv ↗ · more like this ·

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models…

Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention

Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty +4 more

cs.CL · · cited 788× · arXiv ↗ · more like this ·

Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences -- a topic being actively studied in the community. To address this limitation, we propose Nyströmformer -- a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nyström method to approximate standard self-attention with $O(n)$ complexity. The scalability of Nyströmformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and…

Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language

Andy Zeng, Maria Attarian, Brian Ichter +10 more

cs.CV · · cited 760× · arXiv ↗ · more like this ·

Large pretrained (e.g., "foundation") models exhibit distinct capabilities depending on the domain of data they are trained on. While these domains are generic, they may only barely overlap. For example, visual-language models (VLMs) are trained on Internet-scale image captions, but large language models (LMs) are further trained on Internet-scale text with no images (e.g., spreadsheets, SAT questions, code). As a result, these models store different forms of commonsense knowledge across different domains. In this work, we show that this diversity is symbiotic, and can be leveraged through Socratic Models (SMs): a modular framework in which multiple pretrained models may be composed zero-shot i.e., via multimodal-informed prompting, to exchange information with each other and capture new multimodal capabilities, without requiring finetuning. With minimal engineering, SMs are not only…

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Jun Chen, Deyao Zhu, Xiaoqian Shen +7 more

cs.CV · · cited 750× · arXiv ↗ · more like this ·

Large language models have shown their remarkable capabilities as a general interface for various language-related applications. Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others. The challenge is to use a single model for performing diverse vision-language tasks effectively with simple multi-modal instructions. Towards this objective, we introduce MiniGPT-v2, a model that can be treated as a unified interface for better handling various vision-language tasks. We propose using unique identifiers for different tasks when training the model. These identifiers enable our model to better distinguish each task instruction effortlessly and also improve the model learning efficiency for each task. After the three-stage training, the experimental results…

A cautionary perspective on cross-layer design

Vikas Kawadia, P. Rajesh Kumar

Computer Networks and Communications · IEEE Wireless Communications · · cited 744× · DOI ↗ · more like this ·

Recently, in an effort to improve the performance of wireless networks, there has been increased interest in protocols that rely on interactions between different layers. However, such cross-layer design can run at cross purposes with sound and longer-term architectural principles, and lead to various negative consequences. This motivates us to step back and reexamine holistically the issue of cross-layer design and its architectural ramifications. We contend that a good architectural design leads to proliferation and longevity, and illustrate this with some historical examples. Even though the wireless medium is fundamentally different from the wired one, and can offer undreamt of modalities of cooperation, we show that the conventional layered architecture is a reasonable way to operate wireless networks, and is in fact optimal up to an order. However the temptation and perhaps even…

Understanding the Role of Artificial Intelligence in Personalized Engagement Marketing

Vikas Kumar, Bharath Rajan, Rajkumar Venkatesan +1 more

Sociology and Political Science · California Management Review · · cited 711× · DOI ↗ · more like this ·

This article explores the role of artificial intelligence (AI) in aiding personalized engagement marketing—an approach to create, communicate, and deliver personalized offerings to customers. It proposes that consumers are ready for a new journey in which AI is a tool for endless options and information that are narrowed and curated in a personalized way. It also provides predictions for managers regarding the AI-driven environment on branding and customer management practices in both developed and developing countries.

Business as usual: The acceptance and perpetuation of corruption in organizations

Vikas Anand, Blake E. Ashforth, Mahendra Joshi

Sociology and Political Science · Academy of Management Perspectives · · cited 698× · DOI ↗ · more like this ·

Executive Summary Many of the recent corruption scandals at formerly venerated organizations such as Enron, WorldCom, and Parmalat have some noteworthy features in common. In most instances, the fraudulent acts involved knowing cooperation among numerous employees who were upstanding community members, givers to charity, and caring parents—far removed from the prototypical image of a criminal. The involvement of such individuals in corrupt acts, and the persistence of the acts over time, is both disturbing and puzzling. We argue that the above phenomenon can be explained in part by the rationalization tactics used by individuals committing unethical or fraudulent acts. Rationalizations are mental strategies that allow employees (and others around them) to view their corrupt acts as justified. Employees may collectively use rationalizations to neutralize any regrets or negative feelings…

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

Vikas Yadav, Steven Bethard

cs.CL · · cited 678× · arXiv ↗ · more like this ·

Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. NER systems have been studied and developed widely for decades, but accurate systems using deep neural networks (NN) have only been introduced in the last few years. We present a comprehensive survey of deep neural network architectures for NER, and contrast them with previous approaches to NER based on feature engineering and other supervised or semi-supervised learning algorithms. Our results highlight the improvements achieved by neural networks, and show how incorporating some of the lessons learned from past work on feature-based NER systems can yield further improvements.

How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation

Amr Hendy, Mohamed Abdelrehim, Amr Sharaf +6 more

cs.CL · · cited 630× · arXiv ↗ · more like this ·

Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoroughly investigated. In this paper, we present a comprehensive evaluation of GPT models for machine translation, covering various aspects such as quality of different GPT models in comparison with state-of-the-art research and commercial systems, effect of prompting strategies, robustness towards domain shifts and document-level translation. We experiment with eighteen different translation directions involving high and low resource languages, as well as non English-centric translations, and evaluate the performance of three GPT models: ChatGPT, GPT3.5 (text-davinci-003), and text-davinci-002. Our results show that GPT models achieve very competitive translation quality for high resource languages, while having…

Clock rate versus IPC

Vikas Agarwal, M.S. Hrishikesh, Stephen W. Keckler +1 more

Hardware and Architecture · · cited 611× · DOI ↗ · more like this ·

The doubling of microprocessor performance every three years has been the result of two factors: more transistors per chip and superlinear scaling of the processor clock with technology generation. Our results show that, due to both diminishing improvements in clock rates and poor wire scaling as semiconductor devices shrink, the achievable performance growth of conventional microarchitectures will slow substantially. In this paper, we describe technology-driven models for wire capacitance, wire delay, and microarchitectural component delay. Using the results of these models, we measure the simulated performance-estimating both clock rate and IPCof an aggressive out-of-order microarchitecture as it is scaled from a 250nm technology to a 35nm technology. We perform this analysis for three clock scaling targets and two microarchitecture scaling strategies: pipeline scaling and capacity…

Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets

Tara N. Sainath, Brian Kingsbury, Vikas Sindhwani +2 more

Signal Processing · · cited 607× · DOI ↗ · more like this ·

While Deep Neural Networks (DNNs) have achieved tremendous success for large vocabulary continuous speech recognition (LVCSR) tasks, training of these networks is slow. One reason is that DNNs are trained with a large number of training parameters (i.e., 10–50 million). Because networks are trained with a large number of output targets to achieve good performance, the majority of these parameters are in the final weight layer. In this paper, we propose a low-rank matrix factorization of the final weight layer. We apply this low-rank technique to DNNs for both acoustic modeling and language modeling. We show on three different LVCSR tasks ranging between 50–400 hrs, that a low-rank factorization reduces the number of parameters of the network by 30–50%. This results in roughly an equivalent reduction in training time, without a significant loss in final recognition accuracy, compared to…

Power control and clustering in ad hoc networks

Vikas Kawadia, P.R. Kumar

Computer Networks and Communications · · cited 591× · DOI ↗ · more like this ·

In this paper, we consider the problem of power control when nodes are nonhomogeneously dispersed in space. In such situations, one seeks to employ per packet power control depending on the source and destination of the packet. This gives rise to a joint problem which involves not only power control but also clustering. We provide three solutions for joint clustering and power control. The first protocol, CLUSTERPOW, aims to increase the network capacity by increasing spatial reuse. We provide a simple and modular architecture to implement CLUSTERPOW at the network layer. The second, Tunnelled CLUSTERPOW, allows a finer optimization by using encapsulation, but we do not know of an efficient way to implement it. The last, MINPOW, whose basic idea is not new, provides an optimal routing solution with respect to the total power consumed in communication. Our contribution includes a clean…

Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks

Hardik Sharma, Jongse Park, Naveen Suda +5 more

cs.NE · · cited 586× · arXiv ↗ · more like this ·

Fully realizing the potential of acceleration for Deep Neural Networks (DNNs) requires understanding and leveraging algorithmic properties. This paper builds upon the algorithmic insight that bitwidth of operations in DNNs can be reduced without compromising their classification accuracy. However, to prevent accuracy loss, the bitwidth varies significantly across DNNs and it may even be adjusted for each layer. Thus, a fixed-bitwidth accelerator would either offer limited benefits to accommodate the worst-case bitwidth requirements, or lead to a degradation in final accuracy. To alleviate these deficiencies, this work introduces dynamic bit-level fusion/decomposition as a new dimension in the design of DNN accelerators. We explore this dimension by designing Bit Fusion, a bit-flexible accelerator, that constitutes an array of bit-level processing elements that dynamically fuse to match…

Evaluating Public Space

Vikas Mehta

Sociology and Political Science · Journal of Urban Design · · cited 558× · DOI ↗ · more like this ·

Public space plays an important role in sustaining the public realm. There is a renewed interest in public space with a growing belief that while modern societies no longer depend on the town square or the piazza for basic needs, good public space is required for the social and psychological health of modern communities. New public spaces are emerging around the world and old public space typologies are being retrofitted to contemporary needs. Good public space is responsive, democratic and meaningful. However, few comprehensive instruments exist to measure the quality of public space. Based on an extensive review of literature and empirical work, this paper creates a public space index to assess the quality of public space by empirically evaluating its inclusiveness, meaningfulness, safety, comfort and pleasurability. Four public spaces in downtown Tampa, Florida, are examined using…

Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks

Naveen Suda, Vikas Chandra, Ganesh Dasika +5 more

Computer Vision and Pattern Recognition · · cited 551× · DOI ↗ · more like this ·

Convolutional Neural Networks (CNNs) have gained popularity in many computer vision applications such as image classification, face detection, and video analysis, because of their ability to train and classify with high accuracy. Due to multiple convolution and fully-connected layers that are compute-/memory-intensive, it is difficult to perform real-time classification with low power consumption on today?s computing systems. FPGAs have been widely explored as hardware accelerators for CNNs because of their reconfigurability and energy efficiency, as well as fast turn-around-time, especially with high-level synthesis methodologies. Previous FPGA-based CNN accelerators, however, typically implemented generic accelerators agnostic to the CNN configuration, where the reconfigurable capabilities of FPGAs are not fully leveraged to maximize the overall system throughput. In this work, we…