Author

Ryan Farrell

42 papers · 1968–2026

1968peak 11 in 20222026

Frequent co-authors

Papers

showing 1–25 of 42 RSS

Birdlets: Subordinate categorization using volumetric primitives and pose-normalized appearance

Ryan Farrell, Om Oza, Ning Zhang +3 more

Computer Vision and Pattern Recognition · November 2011 · cited 212× · DOI ↗ · more like this ·

Subordinate-level categorization typically rests on establishing salient distinctions between part-level characteristics of objects, in contrast to basic-level categorization, where the presence or absence of parts is determinative. We develop an approach for subordinate categorization in vision, focusing on an avian domain due to the fine-grained structure of the category taxonomy for this domain. We explore a pose-normalized appearance model based on a volumetric poselet scheme. The variation in shape and appearance properties of these parts across a taxonomy provides the cues needed for subordinate categorization. Training pose detectors requires a relatively large amount of training data per category when done from scratch; using a subordinate-level approach, we exploit a pose classifier trained at the basic-level, and extract part appearance and shape information to build…

Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction

Ning Zhang, Ryan Farrell, Forrest Iandola +1 more

Computer Vision and Pattern Recognition · December 2013 · cited 203× · DOI ↗ · more like this ·

Recognizing objects in fine-grained domains can be extremely challenging due to the subtle differences between subcategories. Discriminative markings are often highly localized, leading traditional object recognition approaches to struggle with the large pose variation often present in these domains. Pose-normalization seeks to align training exemplars, either piecewise by part or globally for the whole object, effectively factoring out differences in pose and in viewing angle. Prior approaches relied on computationally-expensive filter ensembles for part localization and required extensive supervision. This paper proposes two pose-normalized descriptors based on computationally-efficient deformable part models. The first leverages the semantics inherent in strongly-supervised DPM parts. The second exploits weak semantic annotations to learn cross-component correspondences, computing…

The designers' outpost

Scott Klemmer, Mark Newman, Ryan Farrell +2 more

Human-Computer Interaction · 11 Nov 2001 · cited 198× · DOI ↗ · more like this ·

In our previous studies into web design, we found that pens, paper, walls, and tables were often used for explaining, developing, and communicating ideas during the early phases of design. These wall-scale paper-based design practices inspired The Designers' Outpost, a tangible user interface that combines the affordances of paper and large physical workspaces with the advantages of electronic media to support information design. With Outpost, users collaboratively author web site information architectures on an electronic whiteboard using physical media (Post-it notes and images), structuring and annotating that information with electronic pens. This interaction is enabled by a touch-sensitive SMART Board augmented with a robust computer vision system, employing a rear-mounted video camera for capturing movement and a front-mounted high-resolution camera for capturing ink. We conducted…

Pose pooling kernels for sub-category recognition

Ning Zhang, Ryan Farrell, T. Darrell

Artificial Intelligence · June 2012 · cited 135× · DOI ↗ · more like this ·

The ability to normalize pose based on super-category landmarks can significantly improve models of individual categories when training data are limited. Previous methods have considered the use of volumetric or morphable models for faces and for certain classes of articulated objects. We consider methods which impose fewer representational assumptions on categories of interest, and exploit contemporary detection schemes which consider the ensemble of responses of detectors trained for specific pose-keypoint configurations. We develop representations for poselet-based pose normalization using both explicit warping and implicit pooling as mechanisms. Our method defines a pose normalized similarity or kernel function that is suitable for nearest-neighbor or kernel-based learning methods.

Open-vocabulary Object Retrieval

Sergio Guadarrama, Erik Rodner, Ryan Farrell +3 more

Computer Vision and Pattern Recognition · 12 Jul 2014 · cited 77× · DOI ↗ · more like this ·

Abstract—In this paper, we address the problem of retrieving objects based on open-vocabulary natural language queries: Given a phrase describing a specific object, e.g., “the corn flakes box”, the task is to find the best match in a set of images containing candidate objects. When naming objects, humans tend to use natural language with rich semantics, including basic-level categories, fine-grained categories, and instance-level concepts such as brand names. Existing approaches to large-scale object recognition fail in this scenario, as they expect queries that map directly to a fixed set of pre-trained visual categories, e.g. ImageNet synset tags. We address this limitation by introducing a novel object retrieval method. Given a candidate object image, we first map it to a set of words that are likely to describe it, using several learned image-to-text projections. We also propose a…

Aligned to the Object, not to the Image: A Unified Pose-aligned Representation for Fine-grained Recognition

Pei Guo, Ryan Farrell

cs.CV · January 2018 · cited 38× · arXiv ↗ · more like this ·

Dramatic appearance variation due to pose constitutes a great challenge in fine-grained recognition, one which recent methods using attention mechanisms or second-order statistics fail to adequately address. Modern CNNs typically lack an explicit understanding of object pose and are instead confused by entangled pose and appearance. In this paper, we propose a unified object representation built from a hierarchy of pose-aligned regions. Rather than representing an object by regions aligned to image axes, the proposed representation characterizes appearance relative to the object's pose using pose-aligned patches whose features are robust to variations in pose, scale and rotation. We propose an algorithm that performs pose estimation and forms the unified object representation as the concatenation of hierarchical pose-aligned regions features, which is then fed into a classification…

Decentralized discovery of camera network topology

Ryan Farrell, Larry S. Davis

Computer Vision and Pattern Recognition · September 2008 · cited 23× · DOI ↗ · more like this ·

One of the primary uses of camera networks is the observation and tracking of objects within some domain. Substantial research has gone into tracking objects within single and multiple views. However, few such approaches scale to large numbers of sensors, and those that do require an understanding of the network topology. Camera network topology models camera adjacency in the context of tracking: when an object/entity leaves one camera, which cameras could it appear at next? This paper presents a decentralized approach for estimating a camera networkpsilas topology based on sequential Bayesian estimation using a modified multinomial distribution. Central to this method is an information-theoretic appearance model for observation weighting. The distributed nature of the approach utilizes all of the sensors as processing agents in collectively recovering the network topology. Experimental…

Learning Higher-order Transition Models in Medium-scale Camera Networks

Ryan Farrell, David Doermann, Larry S. Davis

Artificial Intelligence · 2007 · cited 20× · DOI ↗ · more like this ·

We present a Bayesian framework for learning higher- order transition models in video surveillance networks. Such higher-order models describe object movement between cameras in the network and have a greater predictive power for multi-camera tracking than camera adjacency alone. These models also provide inherent resilience to camera failure, filling in gaps left by single or even multiple non-adjacent camera failures. Our approach to estimating higher-order transition models relies on the accurate assignment of camera observations to the underlying trajectories of objects moving through the network. We addresses this data association problem by gathering the observations and evaluating alternative partitions of the observation set into individual object trajectories. Searching the complete partition space is intractable, so an incremental approach is taken, iteratively adding…

Fair Comparison: Quantifying Variance in Resultsfor Fine-grained Visual Categorization

Matthew Gwilliam, Adam Teuscher, Connor Anderson +1 more

cs.CV · 8 Sep 2021 · cited 15× · arXiv ↗ · more like this ·

For the task of image classification, researchers work arduously to develop the next state-of-the-art (SOTA) model, each bench-marking their own performance against that of their predecessors and of their peers. Unfortunately, the metric used most frequently to describe a model's performance, average categorization accuracy, is often used in isolation. As the number of classes increases, such as in fine-grained visual categorization (FGVC), the amount of information conveyed by average accuracy alone dwindles. While its most glaring weakness is its failure to describe the model's performance on a class-by-class basis, average accuracy also fails to describe how performance may vary from one trained model of the same architecture, on the same dataset, to another (both averaged across all categories and at the per-class level). We first demonstrate the magnitude of these variations across…

Semantic Network Interpretation

Pei Guo, Ryan Farrell

cs.CV · May 2018 · cited 9× · arXiv ↗ · more like this ·

Network interpretation as an effort to reveal the features learned by a network remains largely visualization-based. In this paper, our goal is to tackle semantic network interpretation at both filter and decision level. For filter-level interpretation, we represent the concepts a filter encodes with a probability distribution of visual attributes. The decision-level interpretation is achieved by textual summarization that generates an explanatory sentence containing clues behind a network's decision. A Bayesian inference algorithm is proposed to automatically associate filters and network decisions with visual attributes. Human study confirms that the semantic interpretation is a beneficial alternative or complement to visualization methods. We demonstrate the crucial role that semantic network interpretation can play in understanding a network's failure patterns. More importantly…

Facing the Hard Problems in FGVC

Connor Anderson, Matt Gwilliam, Adam Teuscher +2 more

cs.CV · 24 Jun 2020 · cited 7× · arXiv ↗ · more like this ·

In fine-grained visual categorization (FGVC), there is a near-singular focus in pursuit of attaining state-of-the-art (SOTA) accuracy. This work carefully analyzes the performance of recent SOTA methods, quantitatively, but more importantly, qualitatively. We show that these models universally struggle with certain "hard" images, while also making complementary mistakes. We underscore the importance of such analysis, and demonstrate that combining complementary models can improve accuracy on the popular CUB-200 dataset by over 5%. In addition to detailed analysis and characterization of the errors made by these SOTA methods, we provide a clear set of recommended directions for future FGVC researchers.

Training with Confusion for Fine-Grained Visual Classification.

Abhimanyu Dubey, Otkrist Gupta, Pei Guo +3 more

Computer Vision and Pattern Recognition · arXiv (Cornell University) · 22 May 2017 · cited 7× · OpenAlex ↗ · more like this ·

Research in Fine-Grained Visual Classification has focused on tackling the variations in pose, lighting, and viewpoint using sophisticated localization and segmentation techniques, and the usage of robust texture features to improve performance. In this work, we look at the fundamental optimization of neural network training for fine-grained classification tasks with minimal inter-class variance, and attempt to learn features with increased generalization to prevent overfitting. We introduce Training-with-Confusion, an optimization procedure for fine-grained classification tasks that regularizes training by introducing confusion in activations. Our method can be generalized to any fine-tuning task; it is robust to the presence of small training sets and label noise; and adds no overhead to the prediction time. We find that Training-with-Confusion improves the state-of-the-art on all…

A Tangible Evolution: System Architecture and Participatory Design Studies of the Designers’ Outpost

Scott Klemmer, Mark Newman, Ryan Farrell +2 more

Human-Computer Interaction · UC Berkeley · 2000 · cited 5× · OpenAlex ↗ · more like this ·

In our previous studies into web design, we found that pens, paper, walls, and tables were often used for explaining, developing, and communicating ideas during the early phases of design. One practice was to produce information architectures using paper on walls. This practice inspired us to work towards The Designer's Outpost, a tangible user interface that combines the affordances of paper and large physical workspaces with the advantages of electronic media to support information design. In this paper, we describe a series of participatory design studies that explored the combination of physical and electronic media in depth. These three studies employed both low- and high-fidelity prototypes. The design teams in our studies encouraged us to support free ink electronic annotations to sitemap pages, versioning of design artifacts, fluid transitions to other tools, and opportunities…

Tracking Down Under: Following the Satin Bowerbird

Aniruddha Kembhavi, Ryan Farrell, Yuancheng Luo +3 more

Computer Vision and Pattern Recognition · 2008 · cited 4× · DOI ↗ · more like this ·

Socio biologists collect huge volumes of video to study animal behavior (our collaborators work with 30,000 hours of video). The scale of these datasets demands the development of automated video analysis tools. Detecting and tracking animals is a critical first step in this process. However, off-the-shelf methods prove incapable of handling videos characterized by poor quality, drastic illumination changes, non-stationary scenery and foreground objects that become motionless for long stretches of time. We improve on existing approaches by taking advantage of specific aspects of this problem: by using information from the entire video we are able to find animals that become motionless for long intervals of time; we make robust decisions based on regional features; for different parts of the image, we tailor the selection of model features, choosing the features most helpful in…

Intelligent Image Collection: Building the Optimal Dataset

Matthew Gwilliam, Ryan Farrell

Computer Vision and Pattern Recognition · March 2020 · cited 3× · DOI ↗ · more like this ·

Key recognition tasks such as fine-grained visual categorization (FGVC) have benefited from increasing attention among computer vision researchers. The development and evaluation of new approaches relies heavily on benchmark datasets; such datasets are generally built primarily with categories that have images readily available, omitting categories with insufficient data. This paper takes a step back and rethinks dataset construction, focusing on intelligent image collection driven by: (i) the inclusion of all desired categories, and, (ii) the recognition performance on those categories. Based on a small, author-provided initial dataset, the proposed system recommends which categories the authors should prioritize collecting additional images for, with the intent of optimizing overall categorization accuracy. We show that mock datasets built using this method outperform datasets built…

Determination of Intensity Discrimination and Detection Capabilities by Absolute Identification Procedures

Ryan M. Farrell, Craig Pynn, Louis D. Braida

Computer Networks and Communications · The Journal of the Acoustical Society of America · July 1968 · cited 3× · DOI ↗ · more like this ·

Results of classical absolute-identification experiments on signal intensity indicate that when the number of stimuli is large, subjects make more identification errors than would be expected on the basis of the pairwise discriminability of the stimuli. These experiments, however, all employed interstimulus intensity increments that are large relative to discrimination ability. Recent work suggests that identification performance should be directly related to discrimination performance when the increments are small. Experiments using small increments have been conducted with tone pulses (discrimination) and with tone pulses masked by noise (detection). The results not only confirm the expected relationship (operating characteristics and sensitivity measures obtained with identification are the same as those obtained with conventional procedures), but indicate that the identification…

Pairwise Confusion for Fine-Grained Visual Classification

Abhimanyu Dubey, Otkrist Gupta, Pei Guo +3 more

cs.CV · May 2017 · cited 2× · arXiv ↗ · more like this ·

Fine-Grained Visual Classification (FGVC) datasets contain small sample sizes, along with significant intra-class variation and inter-class similarity. While prior work has addressed intra-class variation using localization and segmentation techniques, inter-class similarity may also affect feature learning and reduce classification performance. In this work, we address this problem using a novel optimization procedure for the end-to-end neural network training on FGVC tasks. Our procedure, called Pairwise Confusion (PC) reduces overfitting by intentionally {introducing confusion} in the activations. With PC regularization, we obtain state-of-the-art performance on six of the most widely-used FGVC datasets and demonstrate improved localization ability. {PC} is easy to implement, does not need excessive hyperparameter tuning during training, and does not add significant overhead during…

Elusive Images: Beyond Coarse Analysis for Fine-Grained Recognition

Connor Anderson, Matt Gwilliam, Evelyn Gaskin +1 more

Computer Vision and Pattern Recognition · 3 Jan 2024 · cited 1× · DOI ↗ · more like this ·

While the community has seen many advances in recent years to address the challenging problem of Fine-grained Visual Categorization (FGVC), progress seems to be slowing—new state-of-the-art methods often distinguish themselves by improving top-1 accuracy by mere tenths of a percent. However, across all of the now-standard FGVC datasets, there remain sizeable portions of the test data that none of the current state-of-the-art (SOTA) models can successfully predict. This paper provides a framework for identifying and studying the errors that current methods make across diverse fine-grained datasets. Three models of difficulty—Prediction Overlap, Prediction Rank and Pair-wise Class Confusion—are employed to highlight the most challenging sets of images and classes. Extensive experiments apply a range of standard and SOTA methods, evaluating them on multiple FGVC domains and datasets.…

The Islamic State’s West Africa Province

Jason Warner, Ryan O’farrell, Héni Nsaibia +1 more

Sociology and Political Science · Oxford University Press eBooks · March 2022 · cited 1× · DOI ↗ · more like this ·

Based in northeastern Nigeria, the Islamic State's most powerful province would emerge in March 2015, when the group formerly colloquially known as “Boko Haram” would pledge allegiance to the Islamic State, thus becoming the Islamic State’s West Africa Province (ISWAP). While Boko Haram had had, prior to its pledge to the Islamic State, interactions with Al-Qaeda’s affiliate in western Africa, Al-Qaeda in the Islamic Maghreb (AQIM), it never became a formal Al-Qaeda branch: thus, while IS’s global emergence did not extract Boko Haram away from Al-Qaeda, the newly "democratized" global jihadist landscape did give Boko Haram’s leader, Abubakar Shekau, another preferable option for affiliation. Given the mutual affinity between IS Central and Boko Haram, the former accepted the latter's pledge of <italic>bayah</italic> in March 2015: the process of “affiliate utility validation” was swift.…

The Islamic State’s Central Africa Province—Drc

Jason Warner, Ryan O’farrell, Héni Nsaibia +1 more

Anthropology · Oxford University Press eBooks · March 2022 · cited 1× · DOI ↗ · more like this ·

The Islamic State’s Central Africa Province (ISCAP) is the Islamic State’s newest province--both in Africa and globally--and the only province declared after the fall of Baghuz in March 2019. ISCAP is composed of two “wings”: one in eastern Democratic Republic of Congo (DRC) and the other in northern Mozambique. ISCAP’s Congolese wing, (ISCAP - Congo) originated from the re-branding of a longstanding, Ugandan-led Islamist insurgency known as the Allied Democratic Forces (ADF). By granting the ADF “affiliate utility validation” and elevating to a wing of its new Central African Province in April 2019, the Islamic State was able to ostensibly expand into new territories, helping it to retain its preeminence within the global jihadist sphere despite the reputational losses it might otherwise have suffered following its defeats in the core territories which made it so infamous. The…

Have Fun Storming the Castle(s)!

Connor Anderson, Adam Teuscher, Elizabeth Anderson +3 more

Computer Vision and Pattern Recognition · 2021 · cited 1× · DOI ↗ · more like this ·

In recent years, large-scale datasets, each typically tailored to a particular problem, have become a critical factor towards fueling rapid progress in the field of computer vision. This paper describes a valuable new dataset that should accelerate research efforts on problems such as fine-grained classification, instance recognition and retrieval, and geolocalization. The dataset, comprised of more than 2400 individual castles, palaces and fortresses from more than 90 countries, contains more than 770K images in total. This paper details the dataset's construction process, the characteristics including annotations such as location (geotagged latlong and country label), construction date, Google Maps link and estimated per-class and per-image difficulty. An experimental section provides baseline experiments for important vision tasks including classification, instance retrieval and…

Information assurance modeling using the Department of Defense architecture framework

Ryan O'Farrell, Sriharsha Banavara, D. L. Folds +1 more

Computer Networks and Communications · 11 Apr 2010 · cited 1× · DOI ↗ · more like this ·

Information assurance modeling is an important part of the overall security of an information technology system. In this paper we discuss the implementation of an Information Assurance Model using Department of Defense Architecture Framework so as to have traceability between systems and requirements. An assessment of the potential threats to a development effort is presented. Residual Risk was calculated using quantitative analysis. We describe a procedure to analyze the operational risk of an IT system using the DoDAF architecture and relate this operational risk to battlefield risk. We then present the verification and validation of the model. DIACAP scorecard and CNSSI 1253 control set are used as mediums to relate implementation and model.

A Tangible Evolution: System Architecture and Participatory Design

Scott Klemmer, Ryan Farrell, Mark Newman +2 more

Human-Computer Interaction · 20 Aug 2002 · cited 1× · OpenAlex ↗ · more like this ·

In our previous studies into web design, we found that pens, paper, walls, and tables were the primary tools used for explaining, developing, and communicating ideas during the early phases of design. One common practice was to produce information architectures using paper on walls. This practice inspired us to work towards The Designer''s Outpost, a tangible user interface that combines the affordances of paper and large physical workspaces with the advantages of electronic media to support information design. In this paper, we describe a series of participatory design studies that explored the combination of physical and electronic media in depth. These three studies employed both low- and high-fidelity prototypes. The design teams in our studies encouraged us to support free ink electronic annotations to sitemap pages, versioning of design artifacts, fluid transitions to other tools…

Semantic Capability Acquisition and Specialization During Vision-Language Model Fine-Tuning

Suguru Onda, Matthew Bailey, Ryan Farrell

cs.CV · 5 Oct 2026 · cited 0× · arXiv ↗ · more like this ·

Fine-tuning vision-language models (VLMs) is typically evaluated at a single downstream checkpoint, obscuring whether a semantic capability was never acquired or emerged earlier and later declined during specialization. We ask how semantic capabilities are acquired, when they peak, how well they transfer, and what remains at deployment. We study these dynamics as a semantic capability trajectory, tracking identity- and attribute-based capabilities over training. We formulate a trajectory-based framework that separates capability acquisition, capability-specific optima, and later specialization, and introduce Structured Semantic Routing (SSR) to study how the representation of supervision shapes what is acquired. Across six pretrained backbones spanning DFN, MetaCLIP, and OpenAI CLIP, we show that fine-tuning can acquire semantic capability beyond the pretrained state, including gains…

LMPNet for Weakly-supervised Keypoint Discovery

Pei Guo, Ryan Farrell

cs.CV · 3 Jul 2025 · cited 0× · arXiv ↗ · more like this ·

In this work, we explore the task of semantic object keypoint discovery weakly-supervised by only category labels. This is achieved by transforming discriminatively-trained intermediate layer filters into keypoint detectors. We begin by identifying three preferred characteristics of keypoint detectors: (i) spatially sparse activations, (ii) consistency and (iii) diversity. Instead of relying on hand-crafted loss terms, a novel computationally-efficient leaky max pooling (LMP) layer is proposed to explicitly encourage final conv-layer filters to learn "non-repeatable local patterns" that are well aligned with object keypoints. Informed by visualizations, a simple yet effective selection strategy is proposed to ensure consistent filter activations and attention mask-out is then applied to force the network to distribute its attention to the whole object instead of just the most…