arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Craig Atkinson

13 papers · 2008–2026

2008: 12009: 02010: 02011: 02012: 02013: 02014: 02015: 02016: 02017: 12018: 22019: 32020: 12021: 22022: 12023: 02024: 02025: 02026: 2
2008peak 3 in 20192026

Frequent co-authors

Papers

showing 1–13 of 13 RSS

Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks

Craig Atkinson, Brendan McCane, Lech Szymanski +1 more

cs.LG · · cited 79× · arXiv ↗ · more like this ·

In general, neural networks are not currently capable of learning tasks in a sequential fashion. When a novel, unrelated task is learnt by a neural network, it substantially forgets how to solve previously learnt tasks. One of the original solutions to this problem is pseudo-rehearsal, which involves learning the new task while rehearsing generated items representative of the previous task/s. This is very effective for simple tasks. However, pseudo-rehearsal has not yet been successfully applied to very complex tasks because in these tasks it is difficult to generate representative items. We accomplish pseudo-rehearsal by using a Generative Adversarial Network to generate items so that our deep network can learn to sequentially classify the CIFAR-10, SVHN and MNIST datasets. After training on all tasks, our network loses only 1.67% absolute accuracy on CIFAR-10 and gains 0.24% absolute…

Switched linear projections for neural network interpretability

Lech Szymanski, Brendan McCane, Craig Atkinson

cs.LG · · cited 2× · arXiv ↗ · more like this ·

We introduce switched linear projections for expressing the activity of a neuron in a deep neural network in terms of a single linear projection in the input space. The method works by isolating the active subnetwork, a series of linear transformations, that determine the entire computation of the network for a given input instance. With these projections we can decompose activity in any hidden layer into patterns detected in a given input instance. We also propose that in ReLU networks it is instructive and meaningful to examine patterns that deactivate the neurons in a hidden layer, something that is implicitly ignored by the existing interpretability methods tracking solely the active aspect of the network's computation.

Augmenting MSME Participation in Trade with Policy Digitalisation Efforts: Chile’s Contribution to ‘An Internet of Rules’

Craig Atkinson

Political Science and International Relations · · cited 1× · OpenAlex ↗ · more like this ·

The Internet’s potential to enable micro, small, and medium-sized enterprises (MSMEs) to conduct international commercial transactions has yet to be fully realised. Small enterprises, especially MSMEs based in developing countries, continue to face substantial obstacles to their participation in trade. While ‘International Economic Law’ has evolved amidst rapid globalisation and technological change, developing countries’ MSMEs are frequently unable to achieve the benefits associated with trade liberalisation. As ‘new generation’ agreements introduce further legal complexity in both the number and scope of trade rules, this article investigates Chile’s application of information and communications technology (ICT) to improve the accessibility and functionality of commercial policies. With an emphasis on computational approaches to trade policy design and delivery, the Undersecretary of…

Conceptual capacity and effective complexity of neural networks

Lech Szymanski, Brendan McCane, Craig Atkinson

cs.LG · · cited 1× · arXiv ↗ · more like this ·

We propose a complexity measure of a neural network mapping function based on the diversity of the set of tangent spaces from different inputs. Treating each tangent space as a linear PAC concept we use an entropy-based measure of the bundle of concepts in order to estimate the conceptual capacity of the network. The theoretical maximal capacity of a ReLU network is equivalent to the number of its neurons. In practice however, due to correlations between neuron activities within the network, the actual capacity can be remarkably small, even for very big networks. Empirical evaluations show that this new measure is correlated with the complexity of the mapping function and thus the generalisation capabilities of the corresponding network. It captures the effective, as oppose to the theoretical, complexity of the network function. We also showcase some uses of the proposed measure for…

Implementing the African Continental Free Trade Area

Craig Atkinson, Joseph Potvin

Artificial Intelligence · · cited 0× · DOI ↗ · more like this ·

This chapter suggests a rationale for a simple, scalable, and fast computational approach through the ‘Data With Direction Specification’ (DWDS) to supplement the implementation of the African Continental Free Trade Area (AfCFTA). The specification provides a way for digitally executable versions of rules to be published on the Internet in a platform-agnostic open standard format, across all types of rule-makers and rule-takers, together with the means to allow efficient discovery and transmission of information about rules that are ‘in effect’, ‘applicable’ to any category of transaction, and to be ‘invoked’ by a particular transaction. Relations of obligation, permission, and encouragement can be expressed and understood in the natural language, including vernacular languages, of each stakeholder and equivalently by their heterogeneous computational systems. The resulting ‘Internet of…

MIME: Mutual Information Minimisation Exploration

Haitao Xu, Brendan McCane, Lech Szymanski +1 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

We show that reinforcement learning agents that learn by surprise (surprisal) get stuck at abrupt environmental transition boundaries because these transitions are difficult to learn. We propose a counter-intuitive solution that we call Mutual Information Minimising Exploration (MIME) where an agent learns a latent representation of the environment without trying to predict the future states. We show that our agent performs significantly better over sharp transition boundaries while matching the performance of surprisal driven agents elsewhere. In particular, we show state-of-the-art performance on difficult learning games such as Gravitar, Montezuma's Revenge and Doom.

GRIm-RePR: Prioritising Generating Important Features for Pseudo-Rehearsal

Craig Atkinson, Brendan McCane, Lech Szymanski +1 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Pseudo-rehearsal allows neural networks to learn a sequence of tasks without forgetting how to perform in earlier tasks. Preventing forgetting is achieved by introducing a generative network which can produce data from previously seen tasks so that it can be rehearsed along side learning the new task. This has been found to be effective in both supervised and reinforcement learning. Our current work aims to further prevent forgetting by encouraging the generator to accurately generate features important for task retention. More specifically, the generator is improved by introducing a second discriminator into the Generative Adversarial Network which learns to classify between real and fake items from the intermediate activation patterns that they produce when fed through a continual learning agent. Using Atari 2600 games, we experimentally find that improving the generator can…

From Facilitation 2.0 to trade policy 3.0: Opportunities to expand and extend the rules of global trade

Craig Atkinson

Political Science and International Relations · Bern Open Repository and Information System (University of Bern) · · cited 0× · DOI ↗ · more like this ·

Opinion piece by Craig Atkinson in ICTSD blog. Continued expansion of the multilateral agenda through the World Trade Organization’s Trade Facilitation Agreement and beyond, or Facilitation 2.0, represents a coherent basis for extending trade policy’s design and delivery via digital technologies. A more comprehensive trade facilitation agenda will help to create opportunities for the use of “technology as regulation.” The realisation of Facilitation 2.0 may act as a catalyst for an evolution in trade policy’s functionality.

Pseudo-Rehearsal: Achieving Deep Reinforcement Learning without Catastrophic Forgetting

Craig Atkinson, Brendan McCane, Lech Szymanski +1 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Neural networks can achieve excellent results in a wide variety of applications. However, when they attempt to sequentially learn, they tend to learn the new task while catastrophically forgetting previous ones. We propose a model that overcomes catastrophic forgetting in sequential reinforcement learning by combining ideas from continual learning in both the image classification domain and the reinforcement learning domain. This model features a dual memory system which separates continual learning from reinforcement learning and a pseudo-rehearsal system that "recalls" items representative of previous tasks via a deep generative network. Our model sequentially learns Atari 2600 games without demonstrating catastrophic forgetting and continues to perform above human level on all three games. This result is achieved without: demanding additional storage requirements as the number of…

Increasing the accuracy of convolutional neural networks with progressive reinitialisation

Craig Atkinson, Brendan McCane, Lech Szymanski

Computer Vision and Pattern Recognition · · cited 0× · DOI ↗ · more like this ·

This article introduces a training technique called progressive reinitialisation. This technique involves training a Convolutional Neural Network layer by layer. This is achieved by training the whole network and progressively freezing lower layers' weights until the whole network is frozen. When a layer is frozen, all weights in higher layers (unfrozen layers) are reinitialised before training continues. This training procedure is shown to boost the final network's performance by about 2% on CIFAR-10 and 1% on SVHN in the absence of image augmentation.

Cafe Royal Publishing House 2005 - 2012

Craig Atkinson

Sociology and Political Science · CLOK (University of Central Lancashire) · · cited 0× · OpenAlex ↗ · more like this ·

In 2005 I founded Café Royal Books; a publishing house specialising in artists editions. I collaborate with artists working internationally, allowing them a space with few restraints within which to test new work in book form. I publish approximately 15 titles each year. My work is collected by Tate Gallery, the Museum of Modern Art New York, and the Victoria and Albert Museum among others.

Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning

Craig Atkinson

cs.LG · · arXiv ↗ · more like this ·

We identify intervention bias as a previously unquantified failure mode of zero-shot large-language-model (LLM) educational advisory agents: without task-specific training, they recommend action when a hindsight-optimal oracle policy mandates inaction. In a six-arm ablation on the Open University Learning Analytics Dataset (N=800 students, four temporal cutoffs), at day 56 -- when the oracle designates 70.1% of students as needing no intervention -- zero-shot GPT-4o recommends action for 73%, a 43 percentage-point false-positive rate. Commercial RAG and SQL-augmented retrieval are comparably miscalibrated; at 10,000 students this implies about 4,300 unnecessary advisor contacts per cycle. Supervised policy learning eliminates this bias: a trajectory-conditioned ONNX Decision Transformer (DT) and a snapshot XGBoost classifier, trained on the same oracle-labelled trajectories under strict…

Tethered Reasoning: Decoupling Entropy from Hallucination in Quantized LLMs via Manifold Steering

Craig Atkinson

cs.LG · · arXiv ↗ · more like this ·

Quantized language models face a fundamental dilemma: low sampling temperatures yield repetitive, mode-collapsed outputs, while high temperatures (T > 2.0) cause trajectory divergence and semantic incoherence. We present HELIX, a geometric framework that decouples output entropy from hallucination by tethering hidden-state trajectories to a pre-computed truthfulness manifold. HELIX computes a Unified Truth Score (UTS) combining token-level semantic entropy with Mahalanobis distance from the manifold. When UTS indicates trajectory divergence, graduated steering vectors redirect activations toward structurally coherent regions while affecting only 0.2-2.5% of tokens. On 4-bit quantized Granite 4.0 H Small (32B/9B active, hybrid Mamba-Transformer): GSM8K maintains 88.84% accuracy at T = 3.0 (2.81pp degradation from T = 0.5); MMLU maintains 72.49% across 14,042 questions (1.24pp…