arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Jan Neumann

36 papers · 1985–2025

1985: 11986: 01987: 21988: 01989: 01990: 01991: 01992: 11993: 01994: 11995: 01996: 01997: 01998: 01999: 02000: 02001: 22002: 42003: 32004: 02005: 12006: 02007: 42008: 02009: 02010: 02011: 22012: 02013: 02014: 12015: 22016: 32017: 12018: 22019: 12020: 12021: 02022: 02023: 12024: 22025: 1
1985peak 4 in 20022025

Frequent co-authors

Papers

showing 1–25 of 36 RSS

Learning Temporal Regularity in Video Sequences

Mahmudul Hasan, Jonghyun Choi, Jan Neumann +2 more

cs.CV · · cited 1,410× · arXiv ↗ · more like this ·

Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene. We approach this problem by learning a generative model for regular motion patterns, termed as regularity, using multiple sources with very limited supervision. Specifically, we propose two methods that are built upon the autoencoders for their ability to work with little to no supervision. We first leverage the conventional handcrafted spatio-temporal local features and learn a fully connected autoencoder on them. Second, we build a fully convolutional feed-forward autoencoder to learn both the local features and the classifiers as an end-to-end learning framework. Our model can capture the regularities from multiple datasets. We evaluate our methods in both qualitative and quantitative ways - showing the learned regularity…

ActionFlowNet: Learning Motion Representation for Action Recognition

Joe Yue-Hei Ng, Jonghyun Choi, Jan Neumann +1 more

cs.CV · · cited 128× · arXiv ↗ · more like this ·

Even with the recent advances in convolutional neural networks (CNN) in various visual recognition tasks, the state-of-the-art action recognition system still relies on hand crafted motion feature such as optical flow to achieve the best performance. We propose a multitask learning model ActionFlowNet to train a single stream network directly from raw pixels to jointly estimate optical flow while recognizing actions with convolutional neural networks, capturing both appearance and motion in a single model. We additionally provide insights to how the quality of the learned optical flow affects the action recognition. Our model significantly improves action recognition accuracy by a large margin 31% compared to state-of-the-art CNN-based action recognition models trained without external large scale data and additional optical flow input. Without pretraining on large external labeled…

Bilattice-based Logical Reasoning for Human Detection

Vinay Shet, Jan Neumann, Visvanathan Ramesh +1 more

Computer Vision and Pattern Recognition · · cited 106× · DOI ↗ · more like this ·

The capacity to robustly detect humans in video is a critical component of automated visual surveillance systems. This paper describes a bilattice based logical reasoning approach that exploits contextual information and knowledge about interactions between humans, and augments it with the output of different low level detectors for human detection. Detections from low level parts-based detectors are treated as logical facts and used to reason explicitly about the presence or absence of humans in the scene. Positive and negative information from different sources, as well as uncertainties from detections and logical rules, are integrated within the bilattice framework. This approach also generates proofs or justifications for each hypothesis it proposes. These justifications (or lack thereof) are further employed by the system to explain and validate, or reject potential hypotheses.…

Microphone Arrays as Generalized Cameras for Integrated Audio Visual Processing

Adam O’Donovan, Ramani Duraiswami, Jan Neumann

Signal Processing · · cited 51× · DOI ↗ · more like this ·

Combinations of microphones and cameras allow the joint audio visual sensing of a scene. Such arrangements of sensors are common in biological organisms and in applications such as meeting recording and surveillance where both modalities are necessary to provide scene understanding. Microphone arrays provide geometrical information on the source location, and allow the sound sources in the scene to be separated and the noise suppressed, while cameras allow the scene geometry and the location and motion of people and other objects to be estimated. In most previous work the fusion of the audio-visual information occurs at a relatively late stage. In contrast, we take the viewpoint that both cameras and microphone arrays are geometry sensors, and treat the microphone arrays as generalized cameras. We employ computer-vision inspired algorithms to treat the combined system of arrays and…

Polydioptric camera design and 3D motion estimation

Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos

Computer Vision and Pattern Recognition · · cited 45× · DOI ↗ · more like this ·

Most cameras used in computer vision applications are still based on the pinhole principle inspired by our own eyes. It has been found though that this is not necessarily the optimal image formation principle for processing visual information using a machine. We describe how to find the optimal camera for 3D motion estimation by analyzing the structure of the space formed by the light rays passing through a volume of space. Every camera corresponds to a sampling pattern in light ray space, thus the question of camera design can be rephrased as finding the optimal sampling pattern with regard to a given task. This framework suggests that large field-of-view multi-perspective (polydioptric) cameras are the optimal image sensors for 3D motion estimation. We conclude by proposing design principles for polydioptric cameras and describe an algorithm for such a camera that estimates its 3D…

The Topological Information Content of a Map / An Attempt At A Rehabilitation Of Information Theory In Cartography

Jan Neumann

Geography, Planning and Development · Cartographica The International Journal for Geographic Information and Geovisualization · · cited 39× · DOI ↗ · more like this ·

This paper recalls promising attempts at the application of information theory in cartography and their denial by means of the concept of information increment generated in the course of map interpretation which is out of keeping with the principles of information theory. Critical analysis of the idea of information increment is presented, and applied information theory and topology are identified as a scientific environment facilitating explanation of this increment as an inherent attribute of a map. The paper introduces the topological information content of a map as an effective instrument for quantification of the increment mentioned and demonstrates its application in an experimental river basin map. Operation of this tool confirms that the concept of information increment is compatible with the principles of applied information theory which finds hardier usefulness in cartography.

Eyes from eyes: new cameras for structure from motion

Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos

Computer Vision and Pattern Recognition · · cited 28× · DOI ↗ · more like this ·

We investigate the relationship between camera design and the problem of recovering the motion and structure of a scene from video data. The visual information that could possibly be obtained is described by the plenoptic function. A camera can be viewed as a device that captures a subset of this function, that is, it measures some of the light rays in some part of the space. The information contained in the subset determines how difficult it is to solve subsequent interpretation processes. By examining the differential structure of the time varying plenoptic function we relate different known and new camera models to the spatiotemporal structure of the observed scene. This allows us to define a hierarchy of camera designs, where the order is determined by the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the…

Compound eye sensor for 3D ego motion estimation

Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos +1 more

Computer Vision and Pattern Recognition · · cited 25× · DOI ↗ · more like this ·

We describe a compound eye vision sensor for 3D ego motion computation. Inspired by eyes of insects, we show that the compound eye sampling geometry is optimal for 3D camera motion estimation. This optimality allows us to estimate the 3D camera motion in a scene-independent and robust manner by utilizing linear equations. The mathematical model of the new sensor can be implemented in analog networks resulting in a compact computational sensor for instantaneous 3D ego motion measurements in full six degrees of freedom.

Multi-camera networks: eyes from eyes

Cornelia Fermüller, Yiannis Aloimonos, Patrick Baker +3 more

Computer Vision and Pattern Recognition · · cited 22× · DOI ↗ · more like this ·

Autonomous or semi-autonomous intelligent systems, in order to function appropriately, need to create models of their environment, i.e., models of space time. These are descriptions of objects and scenes and descriptions of changes of space over time, that is, events and actions. Despite the large amount of research on this problem, as a community we are still far from developing robust descriptions of a system's spatiotemporal environment using video input (image sequences). Undoubtedly, some progress has been made regarding the understanding of estimating the structure of visual space, but it has not led to solutions to specific applications. There is, however, an alternative approach which is in line with today's "zeitgeist." The vision of artificial systems can be enhanced by providing them with new eyes. If conventional video cameras are put together in various configurations, new…

A hierarchy of cameras for 3D photography

Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos

Computer Vision and Pattern Recognition · · cited 8× · DOI ↗ · more like this ·

We investigate the relationship between camera design and 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. To compute the 3D structure of a scene accurately from a moving vision sensor we need to be able to estimate the motion of the sensor from the recorded image information, a problem that has been well-studied By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for…

OPEN EDUCATIONAL RESOURCES IN GERMANY STATE OF DEVELOPMENT AND SOME INITIAL LESSONS LEARNED

Jan Neumann, Dominic Orr, Jöran Muuß-Merholz

Computer Science Applications · · cited 6× · OpenAlex ↗ · more like this ·

This article is based on a comprehensive report published by UNESCO IITE in September 2018 (Orr, Neumann, Muus-Merholz, 2017). It summarizes the current state of OER in Germany and its development within the last five years. After a short introduction to the German educational system and its wider policy environment (see below 2), a short quantitative overview of the German OER landscape (see below 3) as well as a exemplary review of several OER initiatives will be provided (see below 4). Following, special emphasis will be given to an introduction to the current OERinfo-funding line (see below 5).

Animated Heads: From 3D Motion Fields to Action Descriptions

Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos

Computer Vision and Pattern Recognition · · cited 6× · DOI ↗ · more like this ·

We demonstrate a method to compute three-dimensional (3D) motion fields on a face. Twelve synchronized and calibrated cameras are positioned around a talking person, and observe its head in motion. We represent the head as a deformable mesh, which is fitted in a global optimization step to silhouette-contour and multi-camera stereo data derived from all images. The non-rigid displacement of the mesh from frame to frame, the 3D motion field, is determined from the spatiotemporal derivatives in all the images. We integrate these cues over time, thus producing an animated representation of the talking head. Our ability to estimate 3D motion fields points to a new framework for the study of action. The 3D motion fields can serve as an intermediate representation, which can be analyzed using geometrical and statistical tools for the purpose of extracting representations of generic human…

3rd Workshop on Recommendation Systems for Television and Online Video (RecSysTV 2016)

Jan Neumann, John P. Hannon, Claudio Riefolo +1 more

Information Systems · · cited 4× · DOI ↗ · more like this ·

For many households the television is the central entertainment hub in their home, and the average TV viewer spends about half of their leisure time in front of a TV. At any given moment, a costumer has hundreds to thousands of entertainment choices available, which makes some sort of automatic, personalized recommendations desirable to help consumers deal with the often overwhelming number of choices they face. The 3rd Workshop on Recommendation Systems for Television and Online Video aims to offer a place to present and discuss the latest academic and industrial research on recommendation systems for this challenging and exciting application domain.

Conversational content discovery via comcast X1 voice interface

Shahin Sefati, Parsa Saadatpanah, Hassan Sayyadi +1 more

Artificial Intelligence · · cited 2× · DOI ↗ · more like this ·

The global market for intelligent voice-enabled devices is expanding at a fast pace. Comcast, one of the largest cable provides in the US with about 30 million users, has recently reinvented the way that customers can discover and access content on an entertainment platform by introducing a voice remote control for its Xfinity X1 entertainment platform. Spoken language input allows the customer to express what they are interested in on their terms, which has made it significantly more convenient for the users to find their favorite TV channel or movie compared to the traditional limits of a screen menu navigated with the keys of a TV remote.

Language models for semantic extraction and filtering in video action recognition

Evelyne Tzoukermann, Jan Neumann, Jana Košecká +6 more

Artificial Intelligence · · cited 2× · OpenAlex ↗ · more like this ·

The paper addresses the following issues: (a) how to represent semantic information from natural language so that a vision model can utilize it? (b) how to extract the salient textual information relevant to vision? For a given domain, we present a new model of semantic extraction that takes into account word relatedness as well as word disambiguation in order to apply to a vision model. We automatically process the text transcripts and perform syntactic analysis to extract dependency relations. We then perform semantic extraction on the output to filter semantic entities related to actions. The resulting data are used to populate a matrix of co-occurrences utilized by the vision processing modules. Results show that explicitly modeling the co-occurrence of actions and tools significantly improved performance.

Talking heads: introducing the tool of 3D motion fields in the study of action

Jan Neumann, Yiannis Aloimonos

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

We demonstrate a method to complete three-dimensional (3D) motion fields on a face to serve as an intermediate representation for the study of actions. Twelve synchronized and calibrated cameras are positioned all around a talking person and observe its head in motion. We represent the head as a deformable mesh, which is fitted in a global optimization step to silhouette-contour and multi-camera stereo data derived from all images. The non-rigid displacement of the mesh from frame to frame, the 3D motion field, is determined from the normal flow information in all the images. We integrate these cues over time, thus producing a spatio-temporal representation of the talking head. Our ability to estimate 3D motion fields points to a new framework for the study of action. Using multicamera configurations we can estimate a sequence of evolving 3D motion fields representing specific actions.…

THE TOPOLOGICAL INFORMATION CONTENT OF THE MAP—A MEANS FOR RESOLVING CERTAIN PROBLEMS IN THEORETICAL CARTOGRAPHY

Jan Neumann

Geography, Planning and Development · Mapping Sciences & Remote Sensing · · cited 2× · DOI ↗ · more like this ·

The author attempts to reinterpret the “information increment” hypothesis—i.e., that information derived from studying the spatial relationships of map symbols represents an unmanipulated increment to knowledge above and beyond the strictly content-related information conveyed by the symbols themselves—in such a way as to reconcile differences between the school of “communicative” and “geographic” cartography. The basis for such a rapprochement, in his view, lies in recognizing the inseparability of information in its topologic and metric forms. Examples are provided in respect to the mapping of terrain and the hydrologic and settlement networks. Translated by Elliott B. Urdang, Providence, RI 02906 from: Izvestiya AN SSSR, seriya geograficheskaya, 1991, No. 2, pp. 122–131.

An abstract differential inequality and eigenvalues of variational inequalities

Jan Neumann

Computational Theory and Mathematics · Czech Digital Mathematics Library (Institute of Mathematics CAS) · · cited 2× · OpenAlex ↗ · more like this ·

A detailed analysis of properties of certain initial value problem for an abstract ordinary differential inequality is performed.The result obtained is used to give a new proof of a Miersemann s theorem on eigenvalues of variational inequalities in Hilbert spaces (see [11).While the proof presented by E. Miersemann in [13 issues from certain Krasnoselskii sideas (see 53), our access draws from a method proposed by I.V. Skrypnik (see [63 or 7]).

Semi-supervised Label Generation for 3D Multi-modal MRI Bone Tumor Segmentation

Anna Curto, Benjamin Schloßmacher, Christina Valle +5 more

Computer Vision and Pattern Recognition · Journal of Imaging Informatics in Medicine · · cited 1× · DOI ↗ · more like this ·

Medical image segmentation is challenging due to the need for expert annotations and the variability of these manually created labels. Previous methods tackling label variability focus on 2D segmentation and single modalities, but reliable 3D multi-modal approaches are necessary for clinical applications such as in oncology. In this paper, we propose a framework for generating reliable and unbiased labels with minimal radiologist input for supervised 3D segmentation, reducing radiologists' efforts and variability in manual labeling. Our framework generates AI-assisted labels through a two-step process involving 3D multi-modal unsupervised segmentation based on feature clustering and semi-supervised refinement. These labels are then compared against traditional expert-generated labels in a downstream task consisting of 3D multi-modal bone tumor segmentation. Two 3D-Unet models are…

Simulating Humans at Scale to Evaluate Voice Interfaces for TVs: the Round-Trip System at Comcast

Breck Baldwin, Lauren Reese, LIming Zhang +9 more

Artificial Intelligence · · cited 1× · DOI ↗ · more like this ·

Evaluating large-scale customer-facing voice interfaces involves a variety of challenges, such as data privacy, fairness or unintended bias, and the cost of human labor. Comcast's Xfinity Voice Remote is one such voice interface aimed at users looking to discover content on their TVs. The artificial intelligence (AI) behind the voice remote currently powers multiple voice interfaces, serving tens of millions of requests every day, from users across the globe.In this talk, we introduce a novel Round-Trip system we have built to evaluate the AI serving these voice interfaces in a semi-automated manner, providing a robust and cheap alternative to traditional quality assurance methods. We discuss five specific challenges we have encountered in Round-Trip and describe our solutions in detail.

Recognizing manipulation actions in arts and crafts shows using domain-specific visual and textual cues

Benjamin Sapp, Rizwan Chaudhry, Xiaodong Yu +6 more

Computer Vision and Pattern Recognition · · cited 1× · DOI ↗ · more like this ·

We present an approach for automatic annotation of commercial videos from an arts-and-crafts domain with the aid of textual descriptions. The main focus is on recognizing both manipulation actions (e.g. cut, draw, glue) and the tools that are used to perform these actions (e.g. markers, brushes, glue bottle). We demonstrate how multiple visual cues such as motion descriptors, object presence, and hand poses can be combined with the help of contextual priors that are automatically extracted from associated transcripts or online instructions. Using these diverse features and linguistic information we propose several increasingly complex computational models for recognizing elementary manipulation actions and composite activities, as well as their temporal order. The approach is evaluated on a novel dataset of comprised of 27 episodes of PBS Sprout TV, each containing on average 8…

Eyes from Eyes: Analysis of Camera Design Using Plenoptic Video Geometry

Jan Neumann, Cornelia Fermüller, Yiannis Aloimonos

Computer Vision and Pattern Recognition · · cited 1× · OpenAlex ↗ · more like this ·

We investigate the relationship between camera design and the problem of recovering the motion and structure of a scene from video data. The visual information that could possibly be obtained is described by the plenoptic function. A camera can be viewed as a device that captures a subset of this function, that is, it measures some of the light rays in some part of the space. The information contained in the subset determines how difficult it is to solve subsequent interpretation processes. By examining the differential structure of the time varying plenoptic function we relate different known and new camera models to the spatio-temporal structure of the observed scene. This allows us to define a hierarchy of camera designs, where the order is determined by the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is…

Die Annahmequote bei unterschiedlichen Gestaltungsmodellen des betrieblichen Eingliederungsmanagements (BEM)

Performa Nord, Zentrum für Gesunde Arbeit, Arbeitsmedizinischer Dienst, Bremen, Jan Neumann, V. Scherenberg

Sociology and Political Science · ASU Arbeitsmedizin Sozialmedizin Umweltmedizin · · cited 0× · DOI ↗ · more like this ·

The acceptance rate for different design models of occupational integration management (OIM) Since the introduction of occupational integration management (OIM) in 2004, various ways of implementing OIM have been developed. A systematic literature analysis was used to search for and compare OIM acceptance rates for different design models. However, only a few publications could be identified, and only some of these gave concrete figures. The majority of the results related to the public sector and showed rates of between 12 % to 58 %. To that extent there is a clear need for research to enable companies to select the optimum model for themselves and to be able to quickly reintegrate OIM beneficiaries into the company. Keywords: operational integration management (OIM) – Acceptance rate – OIM-design – systematic literature analysis

The Open Education policy registry: An open benchlearning tool

Leo Havemann, Javiera Atenas, Jan Neumann

Computer Science Applications · Open Research Online (The Open University) · · cited 0× · DOI ↗ · more like this ·

Open Education does not occur in a vacuum; ergo, policies aimed at fostering sustainable growth of Open Educational Practices (OEP) must acknowledge that such practices sit within a wider landscape of socioeconomic and educational ecosystems aiming to widespread efforts to democratise access to research and knowledge; however, the development of national and institutional-level OER & OEP policies appears to lag behind, therefore, the main objective of the OE policy registry is to become a hub for benchlearning, supporting institutions and communities of practice to develop OE policies that promote ethical developments of OE (Atenas et al, 2019). Much discussion of the need for open education policy has occurred at the supranational level. International OE declarations (e.g. Cape Town Declaration, 2007; Paris Declaration, 2012; UNESCO-Recommendation, 2019) have called for an OE ethos…

Fostering policy making in Open Education: The OER policy registry

Leo Havemann, Jan Neumann, Javiera Atenas

Computer Science Applications · Open Research Online (The Open University) · · cited 0× · OpenAlex ↗ · more like this ·

The OER World Map aims at be the platform that facilitates access to materials and and knowledge to promote open educational practices, acting as a source of information on current projects, practices and people for everyone interested in Open Education (OE). One of its newest developments is the OER Policy Registry, which aims not only to record current policies in OE but to become the knowledge hub to support the wider OE community to develop OE policies, helping them to bench-learn from other policies, to contact organisations that have developed policies and to openly learn from others. This session will explain how you can engage with the Policy Registry, and why you should.