arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Yusuf

4,997 papers · 1896–2026

1967: 21968: 01969: 11970: 31971: 01972: 01973: 01974: 11975: 11976: 11977: 01978: 11979: 21980: 31981: 21982: 11983: 21984: 01985: 11986: 51987: 51988: 51989: 31990: 21991: 51992: 71993: 71994: 91995: 61996: 151997: 81998: 51999: 142000: 202001: 162002: 502003: 422004: 402005: 452006: 502007: 792008: 832009: 982010: 1272011: 1512012: 2332013: 2562014: 2812015: 3272016: 3822017: 4772018: 5982019: 6302020: 2432021: 762022: 642023: 682024: 1352025: 1422026: 163
1967peak 630 in 20192026

Frequent co-authors

Papers

showing 1–25 of 4,997 RSS

On the Opportunities and Risks of Foundation Models

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111 more

cs.LG · · cited 7,452× · arXiv ↗ · more like this ·

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent…

SoundNet: Learning Sound Representations from Unlabeled Video

Yusuf Aytar, Carl Vondrick, Antonio Torralba

cs.CV · · cited 1,125× · arXiv ↗ · more like this ·

We learn rich natural sound representations by capitalizing on large amounts of unlabeled sound data collected in the wild. We leverage the natural synchronization between vision and sound to learn an acoustic representation using two-million unlabeled videos. Unlabeled video has the advantage that it can be economically acquired at massive scales, yet contains useful signals about natural sound. We propose a student-teacher training procedure which transfers discriminative visual knowledge from well established visual recognition models into the sound modality using unlabeled video as a bridge. Our sound representation yields significant performance improvements over the state-of-the-art results on standard benchmarks for acoustic scene/object classification. Visualizations suggest some high-level semantics automatically emerge in the sound network, even though it is trained without…

Analysis and Interpretation of Treatment Effects in Subgroups of Patients in Randomized Clinical Trials

Salim Yusuf

Statistics and Probability · JAMA · · cited 975× · DOI ↗ · more like this ·

A key principle for interpretation of subgroup results is that quantitative interactions (differences in degree) are much more likely than qualitative interactions (differences in kind). Quantitative interactions are likely to be truly present whether or not they are apparent, whereas apparent qualitative interactions should generally be disbelieved as they have usually not been replicated consistently. Therefore, the overall trial result is usually a better guide to the direction of effect in subgroups than the apparent effect observed within a subgroup. Failure to specify prior hypotheses, to account for multiple comparisons, or to correct P values increases the chance of finding spurious subgroup effects. Conversely, inadequate sample size, classification of patients into the wrong subgroup, and low power of tests of interaction make finding true subgroup effects difficult. We…

Genie: Generative Interactive Environments

Jake Bruce, Michael Dennis, Ashley Edwards +22 more

cs.LG · · cited 875× · arXiv ↗ · more like this ·

We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of…

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions

Terry Yue Zhuo, Minh Chien Vu, Jenny Chim +30 more

cs.SE · · cited 701× · arXiv ↗ · more like this ·

Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks ranging from software engineering development to general-purpose reasoning. While current benchmarks have shown that LLMs can solve tasks using programs like human developers, the majority of their evaluations are limited to short and self-contained algorithmic tasks or standalone function calls. Solving challenging and practical tasks requires the capability of utilizing diverse function calls as tools to efficiently implement functionalities like data analysis and web development. In addition, using multiple tools to solve a task needs compositional reasoning by accurately understanding complex instructions. Fulfilling both of these characteristics can pose a great challenge for LLMs.To assess how well LLMs can solve challenging and practical tasks via…

OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

Anas Awadalla, Irena Gao, Josh Gardner +13 more

cs.CV · · cited 660× · arXiv ↗ · more like this ·

We introduce OpenFlamingo, a family of autoregressive vision-language models ranging from 3B to 9B parameters. OpenFlamingo is an ongoing effort to produce an open-source replication of DeepMind's Flamingo models. On seven vision-language datasets, OpenFlamingo models average between 80 - 89% of corresponding Flamingo performance. This technical report describes our models, training data, hyperparameters, and evaluation suite. We share our models and code at https://github.com/mlfoundations/open_flamingo.

With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations

Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson +2 more

cs.CV · · cited 563× · arXiv ↗ · more like this ·

Self-supervised learning algorithms based on instance discrimination train encoders to be invariant to pre-defined transformations of the same instance. While most methods treat different views of the same image as positives for a contrastive loss, we are interested in using positives from other instances in the dataset. Our method, Nearest-Neighbor Contrastive Learning of visual Representations (NNCLR), samples the nearest neighbors from the dataset in the latent space, and treats them as positives. This provides more semantic variations than pre-defined transformations. We find that using the nearest-neighbor as positive in contrastive losses improves performance significantly on ImageNet classification, from 71.7% to 75.6%, outperforming previous state-of-the-art methods. On semi-supervised learning benchmarks we improve performance significantly when only 1% ImageNet labels are…

Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher Education

Ramteja Sajja, Yusuf Sermet, Muhammed Cikmaz +2 more

cs.AI · · cited 556× · arXiv ↗ · more like this ·

This paper presents a novel framework, Artificial Intelligence-Enabled Intelligent Assistant (AIIA), for personalized and adaptive learning in higher education. The AIIA system leverages advanced AI and Natural Language Processing (NLP) techniques to create an interactive and engaging learning platform. This platform is engineered to reduce cognitive load on learners by providing easy access to information, facilitating knowledge assessment, and delivering personalized learning support tailored to individual needs and learning styles. The AIIA's capabilities include understanding and responding to student inquiries, generating quizzes and flashcards, and offering personalized learning pathways. The research findings have the potential to significantly impact the design, implementation, and evaluation of AI-enabled Virtual Teaching Assistants (VTAs) in higher education, informing the…

THE EFFECT OF SOCIAL MEDIA MARKETING ACTIVITIES ON BRAND AWARENESS, BRAND IMAGE AND BRAND LOYALTY

Yusuf Bilgin

Sociology and Political Science · Business And Management Studies An International Journal · · cited 542× · DOI ↗ · more like this ·

The aim of this research is to examine the effect of social media marketing activities on brand awareness, brand image and brand loyalty. In addition, it has been aimed to analyze the effect of brand awareness and brand image on brand loyalty in this research. The population of the research consists of the consumers who actively follow five brands with the highest social score according to the Marketing Turkey social media brand performance data on social media communication channels such as Facebook, Twitter and Instagram. In this research, quantitative method has been used and research data has been obtained via online questionnaires shared on social media from 547 brand followers with applying convenience sampling method. The obtained data have been analyzed by structural equation modeling (SEM). As a result of the analysis, social media marketing activities have been found as…

Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development

Kexin Huang, Tianfan Fu, Wenhao Gao +7 more

cs.LG · · cited 527× · arXiv ↗ · more like this ·

Therapeutics machine learning is an emerging field with incredible opportunities for innovatiaon and impact. However, advancement in this field requires formulation of meaningful learning tasks and careful curation of datasets. Here, we introduce Therapeutics Data Commons (TDC), the first unifying platform to systematically access and evaluate machine learning across the entire range of therapeutics. To date, TDC includes 66 AI-ready datasets spread across 22 learning tasks and spanning the discovery and development of safe and effective medicines. TDC also provides an ecosystem of tools and community resources, including 33 data functions and types of meaningful data splits, 23 strategies for systematic model evaluation, 17 molecule generation oracles, and 29 public leaderboards. All resources are integrated and accessible via an open Python library. We carry out extensive experiments…

Overview of Results of Randomized Clinical Trials in Heart Disease

Salim Yusuf

Statistics and Probability · JAMA · · cited 497× · DOI ↗ · more like this ·

WHEN Daniel<sup>1</sup>requested that he and his compatriots be served pulse and water while Nebuchadnezzar's other young servants be served the king's food and wine, he was inaugurating the first recorded clinical trial. Today we would criticize Daniel's attempt to demonstrate the superiority of his simple diet. He failed to assign the two diets at random; neither the experimental subjects nor the eunuchs who were to measure the ten-day outcome, fairness of countenance, were blinded to the treatment; and the sample size, four subjects on the experimental diet and a larger unspecified number on kingly fare, was too small to be likely to detect a difference in outcome. Despite, or perhaps because of, the faulty design, the results supported Daniel's hypothesis. "And at the end of ten days their countenances appeared fairer, and they were fatter in flesh, than all the youths that did eat…

Single-Channel Multi-Speaker Separation using Deep Clustering

Yusuf Isik, Jonathan Le Roux, Zhuo Chen +2 more

cs.LG · · cited 458× · arXiv ↗ · more like this ·

Deep clustering is a recently introduced deep learning architecture that uses discriminatively trained embeddings as the basis for clustering. It was recently applied to spectrogram segmentation, resulting in impressive results on speaker-independent multi-speaker separation. In this paper we extend the baseline system with an end-to-end signal approximation objective that greatly improves performance on a challenging speech separation. We first significantly improve upon the baseline system performance by incorporating better regularization, larger temporal context, and a deeper architecture, culminating in an overall improvement in signal to distortion ratio (SDR) of 10.3 dB compared to the baseline of 6.0 dB for two-speaker separation, as well as a 7.1 dB SDR improvement for three-speaker separation. We then extend the model to incorporate an enhancement layer to refine the signal…

Learning Cross-Modal Embeddings for Cooking Recipes and Food Images

Amaia Salvador, Nicholas Hynes, Yusuf Aytar +4 more

Computer Vision and Pattern Recognition · DSpace@MIT (Massachusetts Institute of Technology) · · cited 408× · DOI ↗ · more like this ·

In this paper, we introduce Recipe1M, a new large-scale, structured corpus of over 1m cooking recipes and 800k food images. As the largest publicly available collection of recipe data, Recipe1M affords the ability to train high-capacity models on aligned, multi-modal data. Using these data, we train a neural network to find a joint embedding of recipes and images that yields impressive results on an image-recipe retrieval task. Additionally, we demonstrate that regularization via the addition of a high-level classification objective both improves retrieval performance to rival that of humans and enables semantic vector arithmetic. We postulate that these embeddings will provide a basis for further exploration of the Recipe1M dataset and food and cooking in general. Code, data and models are publicly available.

Perception Test: A Diagnostic Benchmark for Multimodal Video Models

Viorica Pătrăucean, Lucas Smaira, Ankush Gupta +21 more

cs.CV · · cited 389× · arXiv ↗ · more like this ·

We propose a novel multimodal video benchmark - the Perception Test - to evaluate the perception and reasoning skills of pre-trained multimodal models (e.g. Flamingo, SeViLA, or GPT-4). Compared to existing benchmarks that focus on computational tasks (e.g. classification, detection or tracking), the Perception Test focuses on skills (Memory, Abstraction, Physics, Semantics) and types of reasoning (descriptive, explanatory, predictive, counterfactual) across video, audio, and text modalities, to provide a comprehensive and efficient evaluation tool. The benchmark probes pre-trained models for their transfer capabilities, in a zero-shot / few-shot or limited finetuning regime. For these purposes, the Perception Test introduces 11.6k real-world videos, 23s average length, designed to show perceptually interesting situations, filmed by around 100 participants worldwide. The videos are…

TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement

Carl Doersch, Yi Yang, Mel Vecerik +5 more

cs.CV · · cited 360× · arXiv ↗ · more like this ·

We present a novel model for Tracking Any Point (TAP) that effectively tracks any queried point on any physical surface throughout a video sequence. Our approach employs two stages: (1) a matching stage, which independently locates a suitable candidate point match for the query point on every other frame, and (2) a refinement stage, which updates both the trajectory and query features based on local correlations. The resulting model surpasses all baseline methods by a significant margin on the TAP-Vid benchmark, as demonstrated by an approximate 20% absolute average Jaccard (AJ) improvement on DAVIS. Our model facilitates fast inference on long and high-resolution video sequences. On a modern GPU, our implementation has the capacity to track points faster than real-time, and can be flexibly extended to higher-resolution videos. Given the high-quality trajectories extracted from a large…

Tabula rasa: Model transfer for object category detection

Yusuf Aytar, Andrew Zisserman

Artificial Intelligence · Oxford University Research Archive (ORA) (University of Oxford) · · cited 344× · DOI ↗ · more like this ·

Our objective is transfer training of a discriminatively trained object category detector, in order to reduce the number of training images required. To this end we propose three transfer learning formulations where a template learnt previously for other categories is used to regularize the training of a new category. All the formulations result in convex optimization problems. Experiments (on PASCAL VOC) demonstrate significant performance gains by transfer learning from one class to another (e.g. motorbike to bicycle), including one-shot learning, specialization from class to a subordinate class (e.g. from quadruped to horse) and transfer using multiple components. In the case of multiple training samples it is shown that a detection performance approaching that of the state of the art can be achieved with substantially fewer training samples.

Temporal Cycle-Consistency Learning

Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson +2 more

cs.CV · · cited 338× · arXiv ↗ · more like this ·

We introduce a self-supervised representation learning method based on the task of temporal alignment between videos. The method trains a network using temporal cycle consistency (TCC), a differentiable cycle-consistency loss that can be used to find correspondences across time in multiple videos. The resulting per-frame embeddings can be used to align videos by simply matching frames using the nearest-neighbors in the learned embedding space. To evaluate the power of the embeddings, we densely label the Pouring and Penn Action video datasets for action phases. We show that (i) the learned embeddings enable few-shot classification of these action phases, significantly reducing the supervised training requirements; and (ii) TCC is complementary to other methods of self-supervised learning in videos, such as Shuffle and Learn and Time-Contrastive Networks. The embeddings are also used for…

Should Physicians Prescribe Religious Activities?

Richard P. Sloan, Emilia Bagiella, Larry VandeCreek +6 more

Health · New England Journal of Medicine · · cited 337× · DOI ↗ · more like this ·

There is increasing interest among the general public and the medical community in the role of religion in medicine. Polls indicate that the U.S. population is highly religious; most people believe in heaven and hell,1 the healing power of prayer,2 and the capacity of faith to aid in the recovery from disease.3 The popular press has published many articles in which religious faith and practice have been said to promote comfort, healing, or both. A report that 77 percent of hospitalized patients wanted physicians to consider their spiritual needs is consistent with this trend.4 Interest in the connection between religion . . .

TAP-Vid: A Benchmark for Tracking Any Point in a Video

Carl Doersch, Ankush Gupta, Larisa Markeeva +6 more

cs.CV · · cited 336× · arXiv ↗ · more like this ·

Generic motion understanding from video involves not only tracking objects, but also perceiving how their surfaces deform and move. This information is useful to make inferences about 3D shape, physical properties and object interactions. While the problem of tracking arbitrary physical points on surfaces over longer video clips has received some attention, no dataset or benchmark for evaluation existed, until now. In this paper, we first formalize the problem, naming it tracking any point (TAP). We introduce a companion benchmark, TAP-Vid, which is composed of both real-world videos with accurate human annotations of point tracks, and synthetic videos with perfect ground-truth point tracks. Central to the construction of our benchmark is a novel semi-automatic crowdsourced pipeline which uses optical flow estimates to compensate for easier, short-term motion like camera shake, allowing…

An Alternative Method for the Evaluation of Docking Performance: RSR vs RMSD

Dilmurat Yusuf, A. M. Davis, Gerard J. Kleywegt +1 more

Computational Theory and Mathematics · Journal of Chemical Information and Modeling · · cited 314× · DOI ↗ · more like this ·

A new assessment criterion for docking poses is proposed in which experimental electron density is taken into account when evaluating the ability of docking programs to reproduce experimentally observed binding modes. Three docking programs (Gold, Glide, and Fred) were used to generate poses for a set of 88 protein-ligand complexes for which the crystal structure is known. The new criterion is based on the real space R-factor (RSR), which measures how well a group of atoms-in our case the ligand-fits the experimental electron density by comparing that density to the expected density, calculated from the model (i.e., the predicted ligand pose). The RSR-based measure is compared to the traditional criterion, the root-mean-square distance (RMSD) between the docking pose and the binding configuration in the crystallographic model. The results highlight several shortcomings of the RMSD…

Deep ROC Analysis and AUC as Balanced Average Accuracy to Improve Model Selection, Understanding and Interpretation

André M. Carrington, Douglas G. Manuel, Paul W. Fieguth +9 more

stat.ME · · cited 300× · arXiv ↗ · more like this ·

Optimal performance is critical for decision-making tasks from medicine to autonomous driving, however common performance measures may be too general or too specific. For binary classifiers, diagnostic tests or prognosis at a timepoint, measures such as the area under the receiver operating characteristic curve, or the area under the precision recall curve, are too general because they include unrealistic decision thresholds. On the other hand, measures such as accuracy, sensitivity or the F1 score are measures at a single threshold that reflect an individual single probability or predicted risk, rather than a range of individuals or risk. We propose a method in between, deep ROC analysis, that examines groups of probabilities or predicted risks for more insightful analysis. We translate esoteric measures into familiar terms: AUC and the normalized concordant partial AUC are balanced…

Comparison of E-mail, Fax, and Postal Surveys of Pediatricians

Shawn R. McMahon, Martha Iwamoto, Mehran S. Massoudi +5 more

Sociology and Political Science · PEDIATRICS · · cited 291× · DOI ↗ · more like this ·

OBJECTIVES: To compare 3 communication modes (postal, fax, and e-mail) in a rotavirus vaccine physician survey. METHODS: We used 3 communication modes to distribute a survey to physicians listed in the membership directory of the Georgia Chapter of the American Academy of Pediatrics. The directory listed 1391 members; however, 404 were deemed ineligible on the basis of their listing as a specialist, retiree, resident in training, or government public health employee. Of the 987 members expected to administer vaccines, 150 were selected randomly to receive the postal survey (postal group). Of the remaining listings, 488 (58%) of 837 listed a fax number; 150 members were selected randomly and faxed a survey (fax group). Of the remaining members, 266 (39%) of 687 had e-mail addresses listed; 150 members were selected randomly for the e-mail survey (e-mail group). A follow-up survey was…

Underimmunization Among Children: Effects of Vaccine Safety Concerns on Immunization Status

Deborah A. Gust, Tara W. Strine, Emmanuel Maurice +6 more

Health · PEDIATRICS · · cited 290× · DOI ↗ · more like this ·

OBJECTIVE: To examine the attitudes, beliefs, and behaviors of parents whose children were underimmunized with respect to > or =2 vaccines that have recently received negative attention, compared with parents whose children were fully immunized with respect to the recommended vaccines. DESIGN: Case-control study. SETTING: A sample of households that participated in the National Immunization Survey were recontacted in 2001. MAIN OUTCOME MEASURE: Vaccination status was assessed. Case subjects were underimmunized with respect to > or =2 of 3 vaccines (diphtheria-tetanus-pertussis or diphtheria-tetanus-acellular pertussis, hepatitis B, or measles-containing vaccines), and control subjects were fully immunized. RESULTS: The response rate was 52.1% (2315 of 4440 subjects). Compared with control households, case households were more likely to make 0 dollar to 30,000 dollars (adjusted odds…

Project Aria: A New Tool for Egocentric Multi-Modal AI Research

Jakob Engel, Kiran Somasundaram, Michael Goesele +71 more

cs.HC · · cited 286× · arXiv ↗ · more like this ·

Egocentric, multi-modal data as available on future augmented reality (AR) devices provides unique challenges and opportunities for machine perception. These future devices will need to be all-day wearable in a socially acceptable form-factor to support always available, context-aware and personalized AI applications. Our team at Meta Reality Labs Research built the Aria device, an egocentric, multi-modal data recording and streaming device with the goal to foster and accelerate research in this area. In this paper, we describe the Aria device hardware including its sensor configuration and the corresponding software tools that enable recording and processing of such data.

Influence of e-WOM engagement on consumer purchase intention in social commerce

Ali Sahabi Yusuf, Ab Razak Che Hussin, Abdelsalam Busalim

Sociology and Political Science · Journal of Services Marketing · · cited 282× · DOI ↗ · more like this ·

Purpose As a business paradigm, social commerce (s-commerce) has brought about a new stage of innovation, and by extension, has transmuted the power from seller to buyer. S-commerce is a combination a commercial and social activities in which individuals may spread word of mouth (WOM) about their shopping experiences and knowledge and provide information about product and services to their to their friends. This kind of social interactions among individuals has increased the potentials of eWOM communication. Given such a backdrop, this paper aims to look into the influence of eWOM engagement on consumers’ purchase intention in s-commerce, which may complement the current effort of the research community in this area. Design/methodology/approach This study used elaboration likelihood model, theory of reasoned action and social support theory to investigate the influence of eWOM…