arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Tim Hamann

13 papers · 2020–2026

2020: 12021: 22022: 32023: 12024: 22025: 12026: 3
2020peak 3 in 20222026

Frequent co-authors

Papers

showing 1–13 of 13 RSS

The OnHW Dataset

Felix Ott, Mohamad Wehbi, Tim Hamann +3 more

Computer Vision and Pattern Recognition · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · · cited 36× · DOI ↗ · more like this ·

This paper presents a handwriting recognition (HWR) system that deals with online character recognition in real-time. Our sensor-enhanced ballpoint pen delivers sensor data streams from triaxial acceleration, gyroscope, magnetometer and force signals at 100 Hz. As most existing datasets do not meet the requirements of online handwriting recognition and as they have been collected using specific equipment under constrained conditions, we propose a novel online handwriting dataset acquired from 119 writers consisting of 31,275 uppercase and lowercase English alphabet character recordings (52 classes) as part of the UbiComp 2020 Time Series Classification Challenge. Our novel OnHW-chars dataset allows for the evaluations of uppercase, lowercase and combined classification tasks, on both writer-dependent (WD) and writer-independent (WI) classes and we show that properly tuned machine…

Benchmarking Online Sequence-to-Sequence and Character-based Handwriting Recognition from IMU-Enhanced Pens

Felix Ott, David Rügamer, Lucas Heublein +4 more

cs.LG · · cited 29× · arXiv ↗ · more like this ·

Purpose. Handwriting is one of the most frequently occurring patterns in everyday life and with it come challenging applications such as handwriting recognition (HWR), writer identification, and signature verification. In contrast to offline HWR that only uses spatial information (i.e., images), online HWR (OnHWR) uses richer spatio-temporal information (i.e., trajectory data or inertial data). While there exist many offline HWR datasets, there is only little data available for the development of OnHWR methods on paper as it requires hardware-integrated pens. Methods. This paper presents data and benchmark models for real-time sequence-to-sequence (seq2seq) learning and single character-based recognition. Our data is recorded by a sensor-enhanced ballpoint pen, yielding sensor data streams from triaxial accelerometers, a gyroscope, a magnetometer and a force sensor at 100 Hz. We propose…

Surface-Free Multi-Stroke Trajectory Reconstruction and Word Recognition Using an IMU-Enhanced Digital Pen

Mohamad Wehbi, Daniel Luge, Tim Hamann +4 more

Human-Computer Interaction · Sensors · · cited 17× · DOI ↗ · more like this ·

Efficient handwriting trajectory reconstruction (TR) requires specific writing surfaces for detecting movements of digital pens. Although several motion-based solutions have been developed to remove the necessity of writing surfaces, most of them are based on classical sensor fusion methods limited, by sensor error accumulation over time, to tracing only single strokes. In this work, we present an approach to map the movements of an IMU-enhanced digital pen to relative displacement data. Training data is collected by means of a tablet. We propose several pre-processing and data-preparation methods to synchronize data between the pen and the tablet, which are of different sampling rates, and train a convolutional neural network (CNN) to reconstruct multiple strokes without the need of writing segmentation or post-processing correction of the predicted trajectory. The proposed system…

Digitizing Handwriting with a Sensor Pen: A Writer-Independent Recognizer

Mohamad Wehbi, Tim Hamann, Jens Barth +1 more

cs.LG · · cited 10× · arXiv ↗ · more like this ·

Online handwriting recognition has been studied for a long time with only few practicable results when writing on normal paper. Previous approaches using sensor-based devices encountered problems that limited the usage of the developed systems in real-world applications. This paper presents a writer-independent system that recognizes characters written on plain paper with the use of a sensor-equipped pen. This system is applicable in real-world applications and requires no user-specific training for recognition. The pen provides linear acceleration, angular velocity, magnetic field, and force applied by the user, and acts as a digitizer that transforms the analogue signals of the sensors into timeseries data while writing on regular paper. The dataset we collected with this pen consists of Latin lower-case and upper-case alphabets. We present the results of a convolutional neural…

ATLAS: An Approximate Time-Series LSTM Accelerator for Low-Power IoT Applications

Fabian Kreß, Alexey Serdyuk, Micha Hiegle +8 more

Computer Vision and Pattern Recognition · · cited 5× · DOI ↗ · more like this ·

Enabling the use of Deep Neural Networks (DNNs) for time-series-based applications on low-power devices such as wearables opens up a wide range of new features and services. However, inference requires an enormous amount of operations to be performed by the computing platform. In addition, Long Short-Term Memory (LSTM)-based networks require memory to store the internal cell state for future calculations. In this paper, we therefore propose a hardware/software co-design based low-power LSTM hardware accelerator architecture for Internet of Things (IoT) applications called ATLAS. The design is based on approximate computing techniques to reduce the power consumption and inference latency by achieving high accuracy. Exemplary, we investigate the impact of applying our proposed architecture to a DNN for handwriting recognition. Thereby, we can show that the accuracy decreases only slightly…

Hardware-aware Workload Distribution for AI-based Online Handwriting Recognition in a Sensor Pen

Fabian KreB, Alexey Serdyuk, Tim Hotfilter +4 more

Computer Vision and Pattern Recognition · 2022 11th Mediterranean Conference on Embedded Computing (MECO) · · cited 5× · DOI ↗ · more like this ·

Time series-based applications such as recognition of handwriting benefit from using Deep Neural Networks (DNNs) in terms of accuracy and efficiency. Due to strict power and memory limitations of embedded platforms in the Internet-of-Things (IoT), the inference of such DNNs is usually performed on more powerful and less constrained devices. However, inference on mobile devices such as smartphones or tablets leads to high system requirements. In this paper, we present our approach for distributing computational workload between sensor pen and a mobile device for handwriting recognition. This not only results in lower system requirements for mobile devices, but also opens up the possibility of storing compressed sensor data on the pen when remote devices are not available or connected. Therefore, we first quantize weights of the DNN layers to significantly reduce the memory footprint and…

KIHT: Kaligo-Based Intelligent Handwriting Teacher

Tanja Harbaum, Alexey Serdyuk, Fabian Kreß +8 more

Human-Computer Interaction · HAL (Le Centre pour la Communication Scientifique Directe) · · cited 3× · DOI ↗ · more like this ·

Kaligo-based Intelligent Handwriting Teacher (KIHT) is a bi-nationally funded research project. The aim of this joint project is to develop an intelligent learning device for automated handwriting, composed of existing components, which can be made available to as many students as possible. With KIHT, we specifically address the challenging task of using inertial sensors to retrace the trajectory of a pen without relying on external reference systems. The nearly unlimited freedom to let the pen glide over the paper has not yet provided a satisfactory solution to this challenge in the state-of-the-art methods, even with sophisticated algorithms and AI approaches. The final phase of the project is now being launched and together with partners from industry and academia, we are taking a holistic approach by considering the entire chain of components, from the pen to the embedded processing…

Write on Paper and Get the Online Digital Trace:\newline A New Era for Handwriting

Florent Imbert, Yann Soullard, Eric Anquetil +5 more

cs.LG · · cited 2× · arXiv ↗ · more like this ·

Capturing the digital trace of handwriting usually requires a specific stylus and a compatible substrate, be it a capacitive touchscreen, an ElectroMagnetic Resonance (EMR) tablet as used in Wacom systems or special paper. While writing on regular paper offers rich haptics, no latency and is well known for improving information retention, no low-cost and widely accepted, effective solution exists to digitize such a pen trace. The challenge is to accurately track the pen's trajectory without an external reference system while allowing unrestricted freedom of pen movement across a surface. We propose an innovative solution that combines a digital pen, advanced artificial intelligence algorithms, and adaptive AI techniques to reconstruct the digital trace of handwriting. Our approach integrates hardware development, focusing on a sensor-equipped pen, with software innovations to optimize…

Improving Online Handwriting Trajectory Reconstruction Based on Temporal Convolutional Networks

Alexey Serdyuk, Fabian Kreß, Iuliia Topko +4 more

Human-Computer Interaction · · cited 0× · DOI ↗ · more like this ·

Online handwriting trajectory acquisition enables a broad spectrum of educational applications by providing relevant and individualized feedback during the learning process. Acquiring handwriting with Inertial Measurement Units (IMUs) is a natural choice due to the compact form factor of this kind of sensor, allowing to integrate them into small size microcontroller-based devices like ball point pens, without compromising their ergonomics. Existing approaches have successfully utilized Temporal Convolutional Networks (TCNs) for trajectory reconstruction, achieving notable results. However, it is still a challenging task to achieve precise handwriting reconstruction due to unevenly sampled ground truth data in publicly available datasets and alignment errors of both the training and ground truth data. In order to address these issues, we introduce a new preprocessing pipeline that…

Tokenization vs. Augmentation: A Systematic Study of Writer Variance in IMU-Based Online Handwriting Recognition

Jindong Li, Dario Zanca, Vincent Christlein +4 more

cs.CV · · arXiv ↗ · more like this ·

Inertial measurement unit-based online handwriting recognition enables the recognition of input signals collected across different writing surfaces but remains challenged by uneven character distributions and inter-writer variability. In this work, we systematically investigate two strategies to address these issues: subword tokenization and concatenation-based data augmentation. Our experiments on the OnHW-Words500 dataset reveal a clear dichotomy between handling inter-writer and intra-writer variance. On the writer-independent split, structural abstraction via Bigram tokenization significantly improves generalization to unseen writing styles, reducing the word error rate (WER) from 15.40% to 12.99%. In contrast, on the writer-dependent split, tokenization degrades performance due to vocabulary distribution shifts between the training and validation sets. Instead, our proposed…

Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead

Jindong Li, Dario Zanca, Vincent Christlein +4 more

cs.CV · · arXiv ↗ · more like this ·

Online handwriting recognition using inertial measurement units opens up handwriting on paper as input for digital devices. Doing it on edge hardware improves privacy and lowers latency, but entails memory constraints. To address this, we propose Error-enhanced Contrastive Handwriting Recognition (ECHWR), a training framework designed to improve feature representation and recognition accuracy without increasing inference costs. ECHWR utilizes a temporary auxiliary branch that aligns sensor signals with semantic text embeddings during the training phase. This alignment is maintained through a dual contrastive objective: an in-batch contrastive loss for general modality alignment and a novel error-based contrastive loss that distinguishes between correct signals and synthetic hard negatives. The auxiliary branch is discarded after training, which allows the deployed model to keep its…

Robust and Efficient Writer-Independent IMU-Based Handwriting Recognition

Jindong Li, Tim Hamann, Jens Barth +3 more

cs.LG · · arXiv ↗ · more like this ·

Handwriting recognition (HWR) using inertial measurement unit (IMU) data remains challenging due to variations in writing styles and the limited availability of datasets. Previous approaches often struggle with handwriting from unseen writers, making writer-independent (WI) recognition a crucial yet difficult problem. This paper presents a model designed to improve WI HWR on IMU data, using a CNN encoder and BiLSTM-based decoder. Our approach demonstrates strong robustness to unseen handwriting styles, outperforming existing methods on the WI splits of both the public OnHW dataset and our word-based dataset, achieving character error rates (CERs) of 7.37% and 9.44%, and word error rates (WERs) of 15.12% and 32.17%, respectively. Robustness evaluation shows that our model maintains superior performance across different age groups, with knowledge learned from one group generalizing better…

Towards an IMU-based Pen Online Handwriting Recognizer

Mohamad Wehbi, Tim Hamann, Jens Barth +3 more

cs.LG · · arXiv ↗ · more like this ·

Most online handwriting recognition systems require the use of specific writing surfaces to extract positional data. In this paper we present a online handwriting recognition system for word recognition which is based on inertial measurement units (IMUs) for digitizing text written on paper. This is obtained by means of a sensor-equipped pen that provides acceleration, angular velocity, and magnetic forces streamed via Bluetooth. Our model combines convolutional and bidirectional LSTM networks, and is trained with the Connectionist Temporal Classification loss that allows the interpretation of raw sensor data into words without the need of sequence segmentation. We use a dataset of words collected using multiple sensor-enhanced pens and evaluate our model on distinct test sets of seen and unseen words achieving a character error rate of 17.97% and 17.08%, respectively, without the use…