arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Bryan Reimer

31 papers · 2007–2024

2007: 12008: 02009: 22010: 02011: 02012: 02013: 12014: 12015: 52016: 62017: 42018: 12019: 42020: 32021: 12022: 02023: 12024: 1
2007peak 6 in 20162024

Frequent co-authors

Papers

showing 1–25 of 31 RSS

Older Adult Perceptions of Smart Home Technologies: Implications for Research, Policy & Market Innovations in Healthcare

Joseph F. Coughlin, Lisa D’Ambrosio, Bryan Reimer +1 more

Demography · Conference proceedings · · cited 168× · DOI ↗ · more like this ·

Advances in information communications technology and related computational power are providing a wide array of systems and related services that form the basis of smart home technologies to support the health, safety and independence of older adults. While these technologies offer significant benefits to older people and their families, they are also transforming older adults into lead adopters of a new 24/7 lifestyle of being monitored, managed, and, at times, motivated, to maintain their health and wellness. To better understand older adult perceptions of smart home technologies and to inform future research a workshop and focus group was conducted with 30 leaders in aging advocacy and aging services from 10 northeastern states. Participants expressed support of technological advance along with a variety of concerns that included usability, reliability, trust, privacy, stigma…

Driver Gaze Region Estimation without Use of Eye Movement

Lex Fridman, Philipp Langhans, Joonbum Lee +1 more

Human-Computer Interaction · IEEE Intelligent Systems · · cited 139× · DOI ↗ · more like this ·

Automated estimation of the allocation of a driver's visual attention could be a critical component of future advanced driver assistance systems. In theory, vision-based tracking of the eye can provide a good estimate of gaze location. But in practice, eye tracking from video is challenging because of sunglasses, eyeglass reflections, lighting conditions, occlusions, motion blur, and other factors. Estimation of head pose, on the other hand, is robust to many of these effects but can't provide as fine-grained of a resolution in localizing the gaze. For the purpose of keeping the driver safe, it's sufficient to partition gaze into regions. In this effort, a proposed system extracts facial features and classifies their spatial configuration into six regions in real time. The proposed method achieves an average accuracy of 91.4 percent at an average decision rate of 11 Hz on a dataset of…

Driver Gaze Region Estimation Without Using Eye Movement

Lex Fridman, Philipp Langhans, Joonbum Lee +1 more

cs.CV · · cited 60× · arXiv ↗ · more like this ·

Automated estimation of the allocation of a driver's visual attention may be a critical component of future Advanced Driver Assistance Systems. In theory, vision-based tracking of the eye can provide a good estimate of gaze location. In practice, eye tracking from video is challenging because of sunglasses, eyeglass reflections, lighting conditions, occlusions, motion blur, and other factors. Estimation of head pose, on the other hand, is robust to many of these effects, but cannot provide as fine-grained of a resolution in localizing the gaze. However, for the purpose of keeping the driver safe, it is sufficient to partition gaze into regions. In this effort, we propose a system that extracts facial features and classifies their spatial configuration into six regions in real-time. Our proposed method achieves an average accuracy of 91.4% at an average decision rate of 11 Hz on a…

Value of Temporal Dynamics Information in Driving Scene Segmentation

Li Ding, Jack Terwilliger, Rini Sherony +2 more

cs.CV · · cited 25× · arXiv ↗ · more like this ·

Semantic scene segmentation has primarily been addressed by forming representations of single images both with supervised and unsupervised methods. The problem of semantic segmentation in dynamic scenes has begun to recently receive attention with video object segmentation approaches. What is not known is how much extra information the temporal dynamics of the visual scene carries that is complimentary to the information available in the individual frames of the video. There is evidence that the human visual system can effectively perceive the scene from temporal dynamics information of the scene's changing visual characteristics without relying on the visual characteristics of individual snapshots themselves. Our work takes steps to explore whether machine perception can exhibit similar properties by combining appearance-based representations and temporal dynamics representations in a…

City browser

Alexander Gruenstein, Jarrod Orszulak, Sean Liu +7 more

Artificial Intelligence · · cited 18× · DOI ↗ · more like this ·

This paper introduces City Browser, a prototype multimodal, conversational, spoken language interface for automotive navigational aid and information access. A study designed to evaluate the impact of age and gender on device interaction errors, perceptions and experiences with the system along with physiological indices of workload is outlined. Preliminary results, plans for further analysis and a larger scale user evaluation are presented.

SideEye: A Generative Neural Network Based Simulator of Human Peripheral Vision

Lex Fridman, Benedikt Jenik, Shaiyan Keshvari +3 more

cs.NE · · cited 14× · arXiv ↗ · more like this ·

Foveal vision makes up less than 1% of the visual field. The other 99% is peripheral vision. Precisely what human beings see in the periphery is both obvious and mysterious in that we see it with our own eyes but can't visualize what we see, except in controlled lab experiments. Degradation of information in the periphery is far more complex than what might be mimicked with a radial blur. Rather, behaviorally-validated models hypothesize that peripheral vision measures a large number of local texture statistics in pooling regions that overlap and grow with eccentricity. In this work, we develop a new method for peripheral vision simulation by training a generative neural network on a behaviorally-validated full-field synthesis model. By achieving a 21,000 fold reduction in running time, our approach is the first to combine realism and speed of peripheral vision simulation to a degree…

CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild

Li Ding, Jack Terwilliger, Aishni Parab +4 more

cs.HC · · cited 13× · arXiv ↗ · more like this ·

Non-intrusive, real-time analysis of the dynamics of the eye region allows us to monitor humans' visual attention allocation and estimate their mental state during the performance of real-world tasks, which can potentially benefit a wide range of human-computer interaction (HCI) applications. While commercial eye-tracking devices have been frequently employed, the difficulty of customizing these devices places unnecessary constraints on the exploration of more efficient, end-to-end models of eye dynamics. In this work, we propose CLERA, a unified model for Cognitive Load and Eye Region Analysis, which achieves precise keypoint detection and spatiotemporal tracking in a joint-learning framework. Our method demonstrates significant efficiency and outperforms prior work on tasks including cognitive load estimation, eye landmark detection, and blink estimation. We also introduce a…

Glanceable, legible typography over complex backgrounds

Ben D. Sawyer, Benjamin Wolfe, Jonathan Dobres +3 more

Human-Computer Interaction · Ergonomics · · cited 13× · DOI ↗ · more like this ·

Modern digital interfaces display typeface in ways new to the 500 year old art of typography, driving a shift in reading from primarily long-form to increasingly short-form. In safety-critical settings, such at-a-glance reading competes with the need to understand the environment. To keep both type and the environment legible, a variety of ‘middle layer’ approaches are employed. But what is the best approach to presenting type over complex backgrounds so as to preserve legibility? This work tests and ranks middle layers in three studies. In the first study, Gaussian blur and semi-transparent ‘scrim’ middle layer techniques best maximise legibility. In the second, an optimal combination of the two is identified. In the third, letter-localised middle layers are tested, with results favouring drop-shadows. These results, discussed in mixed reality (MR) including overlays, virtual reality…

Investigating Drivers' Head and Glance Correspondence

Joonbum Lee, Mauricio Muñoz, Lex Fridman +3 more

cs.OH · · cited 13× · arXiv ↗ · more like this ·

The relationship between a driver's glance pattern and corresponding head rotation is highly complex due to its nonlinear dependence on the individual, task, and driving context. This study explores the ability of head pose to serve as an estimator for driver gaze by connecting head rotation data with manually coded gaze region data using both a statistical analysis approach and a predictive (i.e., machine learning) approach. For the latter, classification accuracy increased as visual angles between two glance locations increased. In other words, the greater the shift in gaze, the higher the accuracy of classification. This is an intuitive but important concept that we make explicit through our analysis. The highest accuracy achieved was 83% using the method of Hidden Markov Models (HMM) for the binary gaze classification problem of (1) the forward roadway versus (2) the center stack.…

Adult children's perceptions of intelligent home systems in the care of elderly parents

Joseph F. Coughlin, Jasmin Lau, Lisa D’Ambrosio +1 more

Demography · · cited 13× · DOI ↗ · more like this ·

Smart home technologies and services are widely researched and are being commercialized for use in the homes of older adults throughout the world. However, widespread adoption by older adults of even the most affordable and simple systems has been mixed. While older adults may be the "users" of such technologies, we seek to understand the perceptions held by key "influencers" in the purchase and use decisions -- the adult child. We present an exploratory study based upon a convenience sample of adult children most likely to embrace intelligent home systems and services. Survey respondents are highly educated, technology savvy, middle to high-income adult children ages 35 and older. This group represents possible "lead adopters" who might advance the commercialization of these potentially useful technologies. Findings suggest that trust in the accuracy of the system, privacy concerns and…

MIT-AVT Clustered Driving Scene Dataset: Evaluating Perception Systems in Real-World Naturalistic Driving Scenarios

Li Ding, Michael Glazer, Meng Wang +3 more

Computer Vision and Pattern Recognition · · cited 10× · DOI ↗ · more like this ·

Solving the driving scene perception problem for driver-assistance systems and autonomous vehicles requires accurate and robust performance in both regularly-occurring driving scenarios (termed “common cases”) and rare outlier scenarios (termed “edge cases”). We propose an automated method for clustering common cases and detecting edge cases based on the visual characteristics of the external scene using deep learning. We apply this approach to develop a large-scale real-world video driving scene dataset of edge cases and common cases. This dataset consists of 1,156,592 10-second video clips, including 450 clusters of common cases, and 5,601 edge cases. We assign human-interpretable metadata labels (e.g., weather, lighting conditions) to the clusters through manual annotation. We further propose two automated methods for large-scale evaluation of scene segmentation models on…

Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions

Lex Fridman, Li Ding, Benedikt Jenik +1 more

cs.AI · · cited 9× · arXiv ↗ · more like this ·

We consider the paradigm of a black box AI system that makes life-critical decisions. We propose an "arguing machines" framework that pairs the primary AI system with a secondary one that is independently trained to perform the same task. We show that disagreement between the two systems, without any knowledge of underlying system design or operation, is sufficient to arbitrarily improve the accuracy of the overall decision pipeline given human supervision over disagreements. We demonstrate this system in two applications: (1) an illustrative example of image classification and (2) on large-scale real-world semi-autonomous driving data. For the first application, we apply this framework to image classification achieving a reduction from 8.0% to 2.8% top-5 error on ImageNet. For the second application, we apply this framework to Tesla Autopilot and demonstrate the ability to predict…

Driver Glance Classification In-the-wild: Towards Generalization Across Domains and Subjects

Sandipan Banerjee, Ajjen Joshi, Jay Turcot +2 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

Distracted drivers are dangerous drivers. Equipping advanced driver assistance systems (ADAS) with the ability to detect driver distraction can help prevent accidents and improve driver safety. In order to detect driver distraction, an ADAS must be able to monitor their visual attention. We propose a model that takes as input a patch of the driver's face along with a crop of the eye-region and classifies their glance into 6 coarse regions-of-interest (ROIs) in the vehicle. We demonstrate that an hourglass network, trained with an additional reconstruction loss, allows the model to learn stronger contextual feature representations than a traditional encoder-only classification module. To make the system robust to subject-specific variations in appearance and behavior, we design a personalized hourglass model tuned with an auxiliary input representing the driver's baseline glance…

A Driving Simulation Study Examining Destination Entry with iOS 5 Google Maps and a Garmin Portable GPS System

Celena Dopart, Anders Häggman, Cameron Thornberry +3 more

Transportation · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · · cited 7× · DOI ↗ · more like this ·

A simulation study compared 23 young adult drivers’ task completion time, mean glance time, number of glances, and percentage of long glances while performing a navigation entry task with a Garmin portable GPS system and a mobile navigation application (iOS 5 Google Maps) on an iPod Touch. We compared participants’ performance on these tasks using the National Highway Traffic Safety Administration (NHTSA) eye-glance acceptance criteria. We found that, irrespective of the device used, no participant was able to complete the task within the recommended total time window of 12 seconds. When entering a destination into the iOS interface, only 73.9% of the drivers meet the NHTSA criteria for long duration glances. With the Garmin system, 91.3% of the participants meet this criterion. All participants were able to maintain adequate mean off road glance durations. Finally, we compared the…

Semi-Automated Annotation of Discrete States in Large Video Datasets

Lex Fridman, Bryan Reimer

cs.CV · · cited 4× · arXiv ↗ · more like this ·

We propose a framework for semi-automated annotation of video frames where the video is of an object that at any point in time can be labeled as being in one of a finite number of discrete states. A Hidden Markov Model (HMM) is used to model (1) the behavior of the underlying object and (2) the noisy observation of its state through an image processing algorithm. The key insight of this approach is that the annotation of frame-by-frame video can be reduced from a problem of labeling every single image to a problem of detecting a transition between states of the underlying objected being recording on video. The performance of the framework is evaluated on a driver gaze classification dataset composed of 16,000,000 images that were fully annotated over 6,000 hours of direct manual annotation labor. On this dataset, we achieve a 13x reduction in manual annotation for an average accuracy of…

Relationship between Drivers' Self-Reported Health and Technology Perceptions Across the Lifespan

Chaiwoo Lee, Bruce Mehler, Alea Mehler +2 more

Demography · · cited 4× · DOI ↗ · more like this ·

This study examined relationships between individual user characteristics and perceptions about technology -- experience, adoption, ability to learn, and trust. Based on responses from a diverse sample of 610 individuals with driving experience, it was found that perceived health and well-being were strongly associated with experience and perceptions around technology, including attitudes toward established and new vehicle technologies. A comparison of results from a correlation analysis showed the effects of perceived health and well-being to be stronger and more significant compared to demographic characteristics and medical conditions. The findings suggest a need for a better understanding of user characteristics, rather than relying on observable traits.

Additional Findings on the Multi-Modal Demands of “Voice-Command” Interfaces

Bruce Mehler, Bryan Reimer, Jonathan Dobres +2 more

Artificial Intelligence · SAE technical papers on CD-ROM/SAE technical paper series · · cited 3× · DOI ↗ · more like this ·

<div class="section abstract"><div class="htmlview paragraph">This paper presents the results of a study of how people interacted with a production voice-command based interface while driving on public roadways. Tasks included phone contact calling, full address destination entry, and point-of-interest (POI) selection. Baseline driving and driving while engaging in multiple-levels of an auditory-vocal cognitive reference task and manual radio tuning were used as comparison points. Measures included self-reported workload, task performance, physiological arousal, glance behavior, and vehicle control for an analysis sample of 48 participants (gender balanced across ages 21-68). Task analysis and glance measures confirm earlier findings that voice-command interfaces do not always allow the driver to keep their hands on the wheel and eyes on the road, as some assume.…

Perceptual Evaluation of Driving Scene Segmentation

Li Ding, Rini Sherony, Bruce Mehler +1 more

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

Human visual perception forms different levels of abstractions expressing the essential semantic components in the scene at different scales. For real-world applications such as driving scene perception, abstractions of both coarse-level, such as the spatial presence of the lead vehicle, and the fine-level, such as the words on a traffic sign, serve as important signals for driver's decision making. However, the granularities of perception required for levels of abstractions are generally different. While current computer vision research makes significant progress in tasks of understanding the global scene (image classification), and dense scene (semantic segmentation), our work takes steps to explore the gap in between. In this paper, we propose Multi-class Probability Pyramid as a representation built on the top of pixel-level semantic scene labels. This representation forms…

A Framework for Robust Driver Gaze Classification

Lex Fridman, Joonbum Lee, Bryan Reimer +1 more

Human-Computer Interaction · SAE technical papers on CD-ROM/SAE technical paper series · · cited 2× · DOI ↗ · more like this ·

<div class="section abstract"><div class="htmlview paragraph">The challenge of developing a robust, real-time driver gaze classification system is that it has to handle difficult edge cases that arise in real-world driving conditions: extreme lighting variations, eyeglass reflections, sunglasses and other occlusions. We propose a single-camera end-toend framework for classifying driver gaze into a discrete set of regions. This framework includes data collection, semi-automated annotation, offline classifier training, and an online real-time image processing pipeline that classifies the gaze region of the driver. We evaluate an implementation of each component on various subsets of a large onroad dataset. The key insight of our work is that robust driver gaze classification in real-world conditions is best approached by leveraging the power of supervised learning to…

Owl and Lizard: Patterns of Head Pose and Eye Pose in Driver Gaze Classification

Lex Fridman, Joonbum Lee, Bryan Reimer +1 more

cs.CV · · cited 2× · arXiv ↗ · more like this ·

Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (1) vehicle interfaces and (2) the design of future Advanced Driver Assistance Systems. In this paper, we estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can we classify driver gaze using head and eye pose versus just using head pose? Second, are there individual-specific gaze strategies that strongly correlate with how much gaze classification improves with the addition of eye pose information? We answer these questions by evaluating data drawn from an on-road study of 40 drivers. The main insight of the paper is conveyed through the analogy of an "owl" and "lizard" which describes the degree to which the eyes and the head move…

Automated Synchronization of Driving Data Using Vibration and Steering Events

Lex Fridman, Daniel E Brown, William Angell +3 more

cs.RO · · cited 0× · arXiv ↗ · more like this ·

We propose a method for automated synchronization of vehicle sensors useful for the study of multi-modal driver behavior and for the design of advanced driver assistance systems. Multi-sensor decision fusion relies on synchronized data streams in (1) the offline supervised learning context and (2) the online prediction context. In practice, such data streams are often out of sync due to the absence of a real-time clock, use of multiple recording devices, or improper thread scheduling and data buffer management. Cross-correlation of accelerometer, telemetry, audio, and dense optical flow from three video sensors is used to achieve an average synchronization error of 13 milliseconds. The insight underlying the effectiveness of the proposed approach is that the described sensors capture overlapping aspects of vehicle vibrations and vehicle steering allowing the cross-correlation function…

Manpower

Joseph F. Coughlin, Jasmin Lau, Bryan Reimer

Demography · · cited 0× · OpenAlex ↗ · more like this ·

Smart home technologies and services are widely researched and being commercialized for use in the homes of older adults throughout the world. However, widespread adoption by older adults of even the most affordable and simple systems has been mixed. While older adults may be the “users ” of such technologies, we seek to understand the perceptions held by key “influencers ” in the buy and use decisions – the adult child. We present an exploratory study based upon a convenience sample of adult children most likely to embrace intelligent home systems and services. Survey respondents are highly educated, technology savvy, middle to high-income adult children ages 35 and older. This group represents possible “lead adopters ” that might advance the commercialization of these potentially useful technologies. Findings suggest that trust in the accuracy of the system, privacy concerns and…

The Context of Crash Occurrence: A Complexity-Infused Approach Integrating Semantic, Contextual, and Kinematic Features

Meng Wang, Zach Noonan, Pnina Gershon +3 more

cs.CV · · arXiv ↗ · more like this ·

Understanding the context of crash occurrence in complex driving environments is essential for improving traffic safety and advancing automated driving. Previous studies have used statistical models and deep learning to predict crashes based on semantic, contextual, or vehicle kinematic features, but none have examined the combined influence of these factors. In this study, we term the integration of these features ``roadway complexity''. This paper introduces a two-stage framework that integrates roadway complexity features for crash prediction. In the first stage, an encoder extracts hidden contextual information from these features, generating complexity-infused features. The second stage uses both original and complexity-infused features to predict crash likelihood, achieving an accuracy of 87.98\% with original features alone and 90.15\% with the added complexity-infused features.…

Eye Contact Between Pedestrians and Drivers

Dina AlAdawy, Michael Glazer, Jack Terwilliger +5 more

cs.HC · · arXiv ↗ · more like this ·

When asked, a majority of people believe that, as pedestrians, they make eye contact with the driver of an approaching vehicle when making their crossing decisions. This work presents evidence that this widely held belief is false. We do so by showing that, in majority of cases where conflict is possible, pedestrians begin crossing long before they are able to see the driver through the windshield. In other words, we are able to circumvent the very difficult question of whether pedestrians choose to make eye contact with drivers, by showing that whether they think they do or not, they can't. Specifically, we show that over 90\% of people in representative lighting conditions cannot determine the gaze of the driver at 15m and see the driver at all at 30m. This means that, for example, that given the common city speed limit of 25mph, more than 99% of pedestrians would have begun crossing…

Dynamics of Pedestrian Crossing Decisions Based on Vehicle Trajectories in Large-Scale Simulated and Real-World Data

Jack Terwilliger, Michael Glazer, Henri Schmidt +5 more

cs.HC · · arXiv ↗ · more like this ·

Humans, as both pedestrians and drivers, generally skillfully navigate traffic intersections. Despite the uncertainty, danger, and the non-verbal nature of communication commonly found in these interactions, there are surprisingly few collisions considering the total number of interactions. As the role of automation technology in vehicles grows, it becomes increasingly critical to understand the relationship between pedestrian and driver behavior: how pedestrians perceive the actions of a vehicle/driver and how pedestrians make crossing decisions. The relationship between time-to-arrival (TTA) and pedestrian gap acceptance (i.e., whether a pedestrian chooses to cross under a given window of time to cross) has been extensively investigated. However, the dynamic nature of vehicle trajectories in the context of non-verbal communication has not been systematically explored. Our work…