arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Axel Berg

15 papers · 2015–2026

2015: 12016: 02017: 12018: 22019: 12020: 22021: 12022: 22023: 02024: 32025: 12026: 1
2015peak 3 in 20242026

Frequent co-authors

Papers

showing 1–15 of 15 RSS

Keyword Transformer: A Self-Attention Model for Keyword Spotting

Axel Berg, Mark O'Connor, Miguel Tairum Cruz

eess.AS · · cited 190× · arXiv ↗ · more like this ·

The Transformer architecture has been successful across many domains, including natural language processing, computer vision and speech recognition. In keyword spotting, self-attention has primarily been used on top of convolutional or recurrent encoders. We investigate a range of ways to adapt the Transformer architecture to keyword spotting and introduce the Keyword Transformer (KWT), a fully self-attentional architecture that exceeds state-of-the-art performance across multiple tasks without any pre-training or additional data. Surprisingly, this simple architecture outperforms more complex models that mix convolutional, recurrent and attentive layers. KWT can be used as a drop-in replacement for these models, setting two new benchmark records on the Google Speech Commands dataset with 98.6% and 97.7% accuracy on the 12 and 35-command tasks respectively.

Deep Ordinal Regression with Label Diversity

Axel Berg, Magnus Oskarsson, Mark O'Connor

cs.LG · · cited 61× · arXiv ↗ · more like this ·

Regression via classification (RvC) is a common method used for regression problems in deep learning, where the target variable belongs to a set of continuous values. By discretizing the target into a set of non-overlapping classes, it has been shown that training a classifier can improve neural network accuracy compared to using a standard regression approach. However, it is not clear how the set of discrete classes should be chosen and how it affects the overall solution. In this work, we propose that using several discrete data representations simultaneously can improve neural network learning compared to a single representation. Our approach is end-to-end differentiable and can be added as a simple extension to conventional learning methods, such as deep neural networks. We test our method on three challenging tasks and show that our method reduces the prediction error compared to a…

Extending GCC-PHAT using Shift Equivariant Neural Networks

Axel Berg, Mark O'Connor, Kalle Åström +1 more

eess.AS · · cited 19× · arXiv ↗ · more like this ·

Speaker localization using microphone arrays depends on accurate time delay estimation techniques. For decades, methods based on the generalized cross correlation with phase transform (GCC-PHAT) have been widely adopted for this purpose. Recently, the GCC-PHAT has also been used to provide input features to neural networks in order to remove the effects of noise and reverberation, but at the cost of losing theoretical guarantees in noise-free conditions. We propose a novel approach to extending the GCC-PHAT, where the received signals are filtered using a shift equivariant neural network that preserves the timing information contained in the signals. By extensive experiments we show that our model consistently reduces the error of the GCC-PHAT in adverse environments, with guarantees of exact time delay recovery in ideal conditions.

The Impact of International Student Mobility Programs on Brazilian Students’ Perceptions of Entrepreneurialism

Luísa Veras de Sandes-Guimarães, Artur Tavares Vilas Boas Ribeiro, Justin Axel-Berg +2 more

Political Science and International Relations · Journal of Studies in International Education · · cited 16× · DOI ↗ · more like this ·

This article assesses whether or not students with international exchange experience return to their home countries with qualitatively different perspectives on the way universities can play a role in fostering innovation and entrepreneurship ecosystems. It includes survey data collected from 516 Brazilian undergraduates asked to rank characteristics of an entrepreneurial university on a scale from 1 to 5. The findings of this study suggest that those with international experience tended to emphasize connections between university and nonacademic partners, compared with nontraveling counterparts, who emphasized the importance of infrastructure in the university. Therefore, use of such programs may have an impact upon the structure of ST&I (Science Technology and Innovation) systems.

wav2pos: Sound Source Localization using Masked Autoencoders

Axel Berg, Jens Gulin, Mark O'Connor +3 more

eess.AS · · cited 5× · arXiv ↗ · more like this ·

We present a novel approach to the 3D sound source localization task for distributed ad-hoc microphone arrays by formulating it as a set-to-set regression problem. By training a multi-modal masked autoencoder model that operates on audio recordings and microphone coordinates, we show that such a formulation allows for accurate localization of the sound source, by reconstructing coordinates masked in the input. Our approach is flexible in the sense that a single model can be used with an arbitrary number of microphones, even when a subset of audio recordings and microphone coordinates are missing. We test our method on simulated and real-world recordings of music and speech in indoor environments, and demonstrate competitive performance compared to both classical and other learning based localization methods.

Points to Patches: Enabling the Use of Self-Attention for 3D Shape Recognition

Axel Berg, Magnus Oskarsson, Mark O'Connor

cs.CV · · cited 2× · arXiv ↗ · more like this ·

While the Transformer architecture has become ubiquitous in the machine learning field, its adaptation to 3D shape recognition is non-trivial. Due to its quadratic computational complexity, the self-attention operator quickly becomes inefficient as the set of input points grows larger. Furthermore, we find that the attention mechanism struggles to find useful connections between individual points on a global scale. In order to alleviate these problems, we propose a two-stage Point Transformer-in-Transformer (Point-TnT) approach which combines local and global attention mechanisms, enabling both individual points and patches of points to attend to each other effectively. Experiments on shape classification show that such an approach provides more useful features for downstream tasks than the baseline Transformer, while also being more computationally efficient. In addition, we also…

Learning Multi-Target TDOA Features for Sound Event Localization and Detection

Axel Berg, Johanna Engman, Jens Gulin +2 more

eess.AS · · cited 1× · arXiv ↗ · more like this ·

Sound event localization and detection (SELD) systems using audio recordings from a microphone array rely on spatial cues for determining the location of sound events. As a consequence, the localization performance of such systems is to a large extent determined by the quality of the audio features that are used as inputs to the system. We propose a new feature, based on neural generalized cross-correlations with phase-transform (NGCC-PHAT), that learns audio representations suitable for localization. Using permutation invariant training for the time-difference of arrival (TDOA) estimation problem enables NGCC-PHAT to learn TDOA features for multiple overlapping sound events. These features can be used as a drop-in replacement for GCC-PHAT inputs to a SELD-network. We test our method on the STARSS23 dataset and demonstrate improved localization performance compared to using standard…

Deployment of strategic performance indicators in Latin American public universities

J.H. Axel-Berg, Jacques Marcovitch

Political Science and International Relations · Leiden Repository (Leiden University) · · cited 1× · OpenAlex ↗ · more like this ·

Jamil Salmi wrote in The Challenge of Establishing World Class Universities (2009) that the University of São Paulo possessed two of the necessary three conditions to become a world class university; a concentration of talent and financial resources, the third leg of his tripod, favourable governance, was to be improved. By extension, the governance model of Latin American Flagship Universities needs institutional agility to implement a strategic vision and form the outward-looking, global view required to be considered among of the world’s great universities. This paper details a research project funded by the São Paulo State Research Council (FAPESP) to analyse the abilities of the three São Paulo state universities to produce consistent, comparable and reliable performance indicators and build capabilities to implement them. The project first carried out an analysis of the existing…

Brazilian higher education performance in response to emerging societal demands

Justin Axel Berg, Jacques Marcovitch, Giovanna de Moura Rocha Lima

Political Science and International Relations · Dialnet (Universidad de la Rioja) · · cited 0× · OpenAlex ↗ · more like this ·

The present article posits that a novel concept of academic excellence is gradually superseding the notion of 'world-class universities'. This new paradigm is that of an engaged university. This emerging paradigm prioritises two main characteristics: first, a focus on generating strong research and teaching capabilities based on natural strengths and demands of the region, and second, the ability to transfer knowledge effectively into guiding public policy focus. This conception has arisen in response to the priorities and demands placed on institutions in contemporary society. The prevailing world-class paradigm for universities is being accorded a lower level of priority by leading higher education systems and institutions than was the case in the past. This paradigm is characterised by a relatively limited set of quantitative indicators, primarily focusing on publication in…

Breaking out of the TTO; turning measurements into indicators

Justin Axel-Berg, Samuel Carvalho de Aragão, Diogo Neves +2 more

Political Science and International Relations · Leiden Repository (Leiden University) · · cited 0× · OpenAlex ↗ · more like this ·

Collecting the right data to represent research impact is only half of the battle for policymakers and researchers. Only when it has gained broad acceptance among stakeholders, and is placed into the right part of a university’s organisational structure can institutional data become indicators, drivers of institutional change and measures of performance. This paper is details a four stage process for constructing and implementing new indicators for entrepreneurial activity in universities. The first is the consolidation of the concept, or vision of the university. The second is the formation of indicators following academic research, here exemplified by the Global Entrepreneurial University Metrics (GEUM) initiative. The third is building public awareness and acceptance of indicators, in this case by way of a university ranking. The final stage is building the institutional structures…

Competing on the world stage: the Universidade de Sa?o Paulo and global universities rankings

Justin Hugo Axel-Berg

Political Science and International Relations · Americanae (AECID Library) · · cited 0× · OpenAlex ↗ · more like this ·

As higher education becomes ever more universalised, and an increasingly fundamental driver in development of both human and technological capital, the need to assess and compare these capabilities has become an important feature of 21st century academia. Public debate over universities has crystallised around universities rankings. This dissertation explains the rise of the three main global university rankings and their relevance to universities on the periphery, in this case the Universidade de São Paulo. By looking first at their theoretical context and then at their methodological limitations, the dissertation shows that these rankings are inadequate measures of performance for modern research universities, and do not provide realistic or desirable goals for universities outside of the Anglo-American system. However, they do point to a variety of weaknesses within USP\\'s system of…

HeatKV: Head-tuned KV-cache Compression for Visual Autoregressive Modeling

Jonathan Cederlund, Axel Berg, William Isaksson +3 more

cs.CV · · arXiv ↗ · more like this ·

Visual Autoregressive (VAR) models have recently demonstrated impressive image generation quality while maintaining low latency. However, they suffer from severe KV-cache memory constraints, often requiring gigabytes of memory per generated image. We introduce HeatKV, a novel compression method that adapts cache allocation in each head based on its attention to previously generated scales. Using a small offline calibration set, the attention heads are ranked according to their attention scores over prior scales. Based on this ranking, we construct a static pruning schedule tailored to a given memory budget. Applied to the Infinity-2B model, HeatKV achieves $2 \times$ higher compression ratio in memory allocation for KV cache compared to existing methods, while maintaining similar or better image fidelity, prompt alignment and human perception score. Our method achieves a new…

Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point

Bokun Wang, Axel Berg, Durmus Alp Emre Acar +1 more

cs.LG · · arXiv ↗ · more like this ·

Recent work has shown that 8-bit floating point (FP8) can be used for efficiently training neural networks with reduced computational cost compared to training in FP32/FP16. In this work, we investigate the use of FP8 training in a federated learning context. This approach brings not only the usual benefits of FP8 which are desirable for on-device training at the edge, but also reduces client-server communication costs due to significant weight compression. We present a novel method for combining FP8 client training while maintaining a global FP32 server model and provide convergence analysis. Experiments with various machine learning models and datasets show that our method consistently yields communication reductions of at least 2.9x across a variety of tasks and models compared to an FP32 baseline to achieve the same trained model accuracy.

Deep-learning enhancement of large scale numerical simulations

Caspar van Leeuwen, Damian Podareanu, Valeriu Codreanu +11 more

cs.CE · · arXiv ↗ · more like this ·

Traditional simulations on High-Performance Computing (HPC) systems typically involve modeling very large domains and/or very complex equations. HPC systems allow running large models, but limits in performance increase that have become more prominent in the last 5-10 years will likely be experienced. Therefore new approaches are needed to increase application performance. Deep learning appears to be a promising way to achieve this. Recently deep learning has been employed to enhance solving problems that traditionally are solved with large-scale numerical simulations using HPC. This type of application, deep learning for high-performance computing, is the theme of this whitepaper. Our goal is to provide concrete guidelines to scientists and others that would like to explore opportunities for applying deep learning approaches in their own large-scale numerical simulations. These…

Improving the Performance of OTDOA based Positioning in NB-IoT Systems

Sha Hu, Axel Berg, Xuhong Li +1 more

cs.IT · · arXiv ↗ · more like this ·

In this paper, we consider positioning with observed-time-difference-of-arrival (OTDOA) for a device deployed in long-term-evolution (LTE) based narrow-band Internet-of-things (NB-IoT) systems. We propose an iterative expectation-maximization based successive interference cancellation (EM-SIC) algorithm to jointly consider estimations of residual frequency-offset (FO), fading-channel taps and time-of-arrival (ToA) of the first arrival-path for each of the detected cells. In order to design a low complexity ToA detector and also due to the limits of low-cost analog circuits, we assume an NB-IoT device working at a low-sampling rate such as 1.92 MHz or lower. The proposed EM-SIC algorithm comprises two stages to detect ToA, based on which OTDOA can be calculated. In a first stage, after running the EM-SIC block a predefined number of iterations, a coarse ToA is estimated for each of the…