Author

Kenneth Chaney

12 papers · 2015–2025

2015peak 2 in 20182025

Frequent co-authors

Papers

showing 1–12 of 12 RSS

A parallel manipulator for mobile manipulating UAVs

Todd W. Danko, Kenneth Chaney, Paul Oh

Computer Vision and Pattern Recognition · May 2015 · cited 72× · DOI ↗ · more like this ·

Manipulating objects using arms mounted to unmanned aerial vehicles (UAVs) is attractive because UAVs may access many locations that are otherwise inaccessible to traditional mobile manipulation platforms such as ground vehicles. Most previous efforts seeking to coordinate the combined manipulator-UAV system have focused on using a manipulator to extend the UAV's reach and assume that both the UAV and manipulator can reliably reach commanded goal poses. This work accepts the reality that state of the art UAV positioning precision is not of a high enough quality to reliably perform simple tasks such as grasping objects. A 6 degree of freedom parallel manipulator is used to robustly maintain precise end-effector positions despite host UAV perturbations. A description of a unique parallel manipulator that allows for very little moving mass, and is easily stowed below a quadrotor UAV is…

Live Demonstration: Unsupervised Event-Based Learning of Optical Flow, Depth and Egomotion

Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney +1 more

Computer Vision and Pattern Recognition · June 2019 · cited 33× · DOI ↗ · more like this ·

We propose a demo of our work, Unsupervised Event-based Learning of Optical Flow, Depth and Egomotion, which will also appear at CVPR 2019. Our demo consists of a CNN which takes as input events from a DAVIS-346b event camera, represented as a discretized event volume, and predicts optical flow for each pixel in the image. Due to the generalization abilities of our network, we are able to predict accurate optical flow for a very wide range of scenes, including for very fast motions and challenging lighting.

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney +1 more

cs.CV · February 2018 · cited 5× · arXiv ↗ · more like this ·

Event-based cameras have shown great promise in a variety of situations where frame based cameras suffer, such as high speed motions and high dynamic range scenes. However, developing algorithms for event measurements requires a new class of hand crafted algorithms. Deep learning has shown great success in providing model free solutions to many problems in the vision community, but existing networks have been developed with frame based images in mind, and there does not exist the wealth of labeled data for events as there does for images for supervised training. To these points, we present EV-FlowNet, a novel self-supervised deep learning pipeline for optical flow estimation for event based cameras. In particular, we introduce an image based representation of a given event stream, which is fed into a self-supervised neural network as the sole input. The corresponding grayscale images…

Event-based Continuous Color Video Decompression from Single Frames

Ziyun Wang, Friedhelm Hamann, Kenneth Chaney +3 more

cs.CV · December 2023 · cited 2× · arXiv ↗ · more like this ·

We present ContinuityCam, a novel approach to generate a continuous video from a single static RGB image and an event camera stream. Conventional cameras struggle with high-speed motion capture due to bandwidth and dynamic range limitations. Event cameras are ideal sensors to solve this problem because they encode compressed change information at high temporal resolution. In this work, we tackle the problem of event-based continuous color video decompression, pairing single static color frames and event data to reconstruct temporally continuous videos. Our approach combines continuous long-range motion modeling with a neural synthesis model, enabling frame prediction at arbitrary times within the events. Our method only requires an initial image, thus increasing the robustness to sudden motions, light changes, minimizing the prediction latency, and decreasing bandwidth usage. We also…

EvMAPPER: High Altitude Orthomapping with Event Cameras

Fernando Cladera, Kenneth Chaney, M. Ani Hsieh +2 more

cs.RO · 26 Sep 2024 · cited 1× · arXiv ↗ · more like this ·

Traditionally, unmanned aerial vehicles (UAVs) rely on CMOS-based cameras to collect images about the world below. One of the most successful applications of UAVs is to generate orthomosaics or orthomaps, in which a series of images are integrated together to develop a larger map. However, the use of CMOS-based cameras with global or rolling shutters mean that orthomaps are vulnerable to challenging light conditions, motion blur, and high-speed motion of independently moving objects under the camera. Event cameras are less sensitive to these issues, as their pixels are able to trigger asynchronously on brightness changes. This work introduces the first orthomosaic approach using event cameras. In contrast to existing methods relying only on CMOS cameras, our approach enables map generation even in challenging light conditions, including direct sunlight and after sunset.

Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation

Friedhelm Hamann, Ziyun Wang, Ioannis Asmanis +3 more

cs.CV · 15 Jul 2024 · cited 0× · arXiv ↗ · more like this ·

Current optical flow and point-tracking methods rely heavily on synthetic datasets. Event cameras are novel vision sensors with advantages in challenging visual conditions, but state-of-the-art frame-based methods cannot be easily adapted to event data due to the limitations of current event simulators. We introduce a novel self-supervised loss combining the Contrast Maximization framework with a non-linear motion prior in the form of pixel-level trajectories and propose an efficient solution to solve the high-dimensional assignment problem between non-linear trajectories and events. Their effectiveness is demonstrated in two scenarios: In dense continuous-time motion estimation, our method improves the zero-shot performance of a synthetically trained model on the real-world dataset EVIMO2 by 29%. In optical flow estimation, our method elevates a simple UNet to achieve state-of-the-art…

A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models

Jason Hughes, Marcel Hussing, Edward Zhang +24 more

cs.RO · 9 Dec 2025 · arXiv ↗ · more like this ·

This report presents a heterogeneous robotic system designed for remote primary triage in mass-casualty incidents (MCIs). The system employs a coordinated air-ground team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) to locate victims, assess their injuries, and prioritize medical assistance without risking the lives of first responders. The UAV identify and provide overhead views of casualties, while UGVs equipped with specialized sensors measure vital signs and detect and localize physical injuries. Unlike previous work that focused on exploration or limited medical evaluation, this system addresses the complete triage process: victim localization, vital sign measurement, injury severity classification, mental status assessment, and data consolidation for first responders. Developed as part of the DARPA Triage Challenge, this approach demonstrates how…

EvAC3D: From Event-based Apparent Contours to 3D Models via Continuous Visual Hulls

Ziyun Wang, Kenneth Chaney, Kostas Daniilidis

cs.CV · 11 Apr 2023 · arXiv ↗ · more like this ·

3D reconstruction from multiple views is a successful computer vision field with multiple deployments in applications. State of the art is based on traditional RGB frames that enable optimization of photo-consistency cross views. In this paper, we study the problem of 3D reconstruction from event-cameras, motivated by the advantages of event-based cameras in terms of low power and latency as well as by the biological evidence that eyes in nature capture the same data and still perceive well 3D shape. The foundation of our hypothesis that 3D reconstruction is feasible using events lies in the information contained in the occluding contours and in the continuous scene acquisition with events. We propose Apparent Contour Events (ACE), a novel event-based representation that defines the geometry of the apparent contour of an object. We represent ACE by a spatially and temporally continuous…

Semantic keypoint-based pose estimation from single RGB frames

Karl Schmeckpeper, Philip R. Osteen, Yufu Wang +6 more

cs.CV · 12 Apr 2022 · arXiv ↗ · more like this ·

This paper presents an approach to estimating the continuous 6-DoF pose of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior investigators, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training-image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Additionally, we accompany our main pipeline with a technique for semi-automatic data generation from unlabeled videos. This procedure allows us to train the learnable components of our method with minimal manual intervention in the labeling process. Empirically, we show that our approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios even against a…

Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks

Chankyu Lee, Adarsh Kumar Kosta, Alex Zihao Zhu +3 more

cs.NE · March 2020 · arXiv ↗ · more like this ·

Event-based cameras display great potential for a variety of tasks such as high-speed motion detection and navigation in low-light environments where conventional frame-based cameras suffer critically. This is attributed to their high temporal resolution, high dynamic range, and low-power consumption. However, conventional computer vision methods as well as deep Analog Neural Networks (ANNs) are not suited to work well with the asynchronous and discrete nature of event camera outputs. Spiking Neural Networks (SNNs) serve as ideal paradigms to handle event camera outputs, but deep SNNs suffer in terms of performance due to the spike vanishing phenomenon. To overcome these issues, we present Spike-FlowNet, a deep hybrid neural network architecture integrating SNNs and ANNs for efficiently estimating optical flow from sparse event camera outputs without sacrificing the performance. The…

Minibatch Processing in Spiking Neural Networks

Daniel J. Saunders, Cooper Sigrist, Kenneth Chaney +2 more

cs.NE · 5 Sep 2019 · arXiv ↗ · more like this ·

Spiking neural networks (SNNs) are a promising candidate for biologically-inspired and energy efficient computation. However, their simulation is notoriously time consuming, and may be seen as a bottleneck in developing competitive training methods with potential deployment on neuromorphic hardware platforms. To address this issue, we provide an implementation of mini-batch processing applied to clock-based SNN simulation, leading to drastically increased data throughput. To our knowledge, this is the first general-purpose implementation of mini-batch processing in a spiking neural networks simulator, which works with arbitrary neuron and synapse models. We demonstrate nearly constant-time scaling with batch size on a simulation setup (up to GPU memory limits), and showcase the effectiveness of large batch sizes in two SNN application domains, resulting in $\approx$880X and $\approx$24X…

Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney +1 more

cs.CV · 19 Dec 2018 · arXiv ↗ · more like this ·

In this work, we propose a novel framework for unsupervised learning for event cameras that learns motion information from only the event stream. In particular, we propose an input representation of the events in the form of a discretized volume that maintains the temporal distribution of the events, which we pass through a neural network to predict the motion of the events. This motion is used to attempt to remove any motion blur in the event image. We then propose a loss function applied to the motion compensated event image that measures the motion blur in this image. We train two networks with this framework, one to predict optical flow, and one to predict egomotion and depths, and evaluate these networks on the Multi Vehicle Stereo Event Camera dataset, along with qualitative results from a variety of different scenes.