Author

Adam White

148 papers · 1847–2026

1967peak 13 in 20102026

Frequent co-authors

Papers

showing 1–25 of 148 RSS

Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction

Richard S. Sutton, Joseph Modayil, Michael Delp +4 more

Artificial Intelligence · Adaptive Agents and Multi-Agents Systems · 2 May 2011 · cited 305× · DOI ↗ · more like this ·

Maintaining accurate world knowledge in a complex and changing environment is a perennial problem for robots and other artificial intelligence systems. Our architecture for addressing this problem, called Horde, consists of a large number of independent reinforcement learning sub-agents, or demons. Each demon is responsible for answering a single predictive or goal-oriented question about the world, thereby contributing in a factored, modular way to the system’s overall knowledge. The questions are in the form of a value function, but each demon has its own policy, reward function, termination function, and terminal-reward function unrelated to those of the base problem. Learning proceeds in parallel by all demons simultaneously so as to extract the maximal training information from whatever actions are taken by the system as a whole. Gradient-based temporal-difference learning methods…

The new political economy of private security

Adam White

Political Science and International Relations · Theoretical Criminology · 7 Jul 2011 · cited 125× · DOI ↗ · more like this ·

This article maps out a new research agenda for interpreting the trajectory and dynamics of domestic private security provision in advanced democratic countries: the ‘new political economy of private security’. It proceeds on the basis that the key challenge in this field is to construct an agenda which takes account of how both the economic context (shifts in supply and demand) and the political context (state-centric conceptions of legitimacy) of domestic security simultaneously serve to shape the conduct of contemporary private security providers. In attempting to meet this challenge, the article not only builds upon the important theoretical research already undertaken in the form of the nodal governance and anchored pluralism models, but also sheds new light on the nature of domestic private security today.

Measurable Counterfactual Local Explanations for Any Classifier

Adam White, Artur d'Avila Garcez

cs.AI · August 2019 · cited 115× · arXiv ↗ · more like this ·

We propose a novel method for explaining the predictions of any classifier. In our approach, local explanations are expected to explain both the outcome of a prediction and how that prediction would change if 'things had been different'. Furthermore, we argue that satisfactory explanations cannot be dissociated from a notion and measure of fidelity, as advocated in the early days of neural networks' knowledge extraction. We introduce a definition of fidelity to the underlying classifier for local explanation models which is based on distances to a target decision boundary. A system called CLEAR: Counterfactual Local Explanations via Regression, is introduced and evaluated. CLEAR generates w-counterfactual explanations that state minimum changes necessary to flip a prediction's classification. CLEAR then builds local regression models, using the w-counterfactuals to measure and improve…

The digital matatu project: Using cell phones to create an open source data for Nairobi's semi-formal bus system

Sarah Williams, Adam White, Peter Waiganjo +2 more

Transportation · Journal of Transport Geography · 30 Oct 2015 · cited 106× · DOI ↗ · more like this ·

In many of the world's growing cities, semi-formal buses form the basis of public transit systems. However, little open and standardized data exist on these systems. The Digital Matatus project in Nairobi, Kenya set out to test whether the geo-locative capabilities of mobile technology could be used to collect data on a semi-formal transit system and whether that data could be translated into the General Transit Feed Specification (GTFS) data standard for wider use. The results of this work show that mobile technologies, particularly mobile phones, which are increasingly prevalent in developing countries, can indeed be used effectively to collect and deliver data in a modified GTFS format for semi-formal transit. Perhaps more importantly, through our work in Nairobi, we were able to identify the benefits and technical needs for developing data on semi-formal transit. Overall, the work…

RL-Glue: Language-Independent Software for Reinforcement-Learning Experiments

B. K. Tanner, Adam White

Artificial Intelligence · Journal of Machine Learning Research · December 2009 · cited 96× · OpenAlex ↗ · more like this ·

RL-Glue is a standard, language-independent software package for reinforcement-learning experiments. The standardization provided by RL-Glue facilitates code sharing and collaboration. Code sharing reduces the need to re-engineer tasks and experimental apparatus, both common barriers to comparatively evaluating new ideas in the context of the literature. Our software features a minimalist interface and works with several languages and computing platforms. RL-Glue compatibility can be extended to any programming language that supports network socket communication. RL-Glue has been used to teach classes, to run international competitions, and is currently used by several other open-source software and hardware projects.

Privileging the Bromance: A Critical Appraisal of Romantic and Bromantic Relationships

Stefan Robinson, Adam White, Eric Anderson

Gender Studies · Men and Masculinities · 12 Oct 2017 · cited 68× · DOI ↗ · more like this ·

In this research, utilizing data from thirty semistructured interviews, we examine how heterosexual undergraduate men compare their experiences of bromances to that of their romantic relationships (romances). We find that the increasingly intimate, emotive, and trusting nature of bromances offers young men a new social space for emotional disclosure, outside of traditional heterosexual relationships. Participants state that the lack of boundaries and judgment in a bromance is expressed as emotionally rivalling the benefits of a heterosexual romance. Our participants mostly determined that a bromance offered them elevated emotional stability, enhanced emotional disclosure, social fulfilment, and better conflict resolution, compared to the emotional lives they shared with girlfriends. Thus, this research provides an empirically grounded conceptual framework for understanding men’s view of…

Post-crisis Policing and Public–Private Partnerships

Adam White

Political Science and International Relations · The British Journal of Criminology · 16 Sep 2014 · cited 58× · DOI ↗ · more like this ·

The post-financial crisis ‘politics of austerity’ have prompted many police forces to explore a range of radical new budget-reducing policies, including outsourcing key service areas to the private sector on an unprecedented scale. This article analyses the largest outsourcing contract to date: the £229 million Lincolnshire Police–G4S strategic partnership. It addresses the question: to what extent has the outsourcing process engendered a shift from the logic of the public good to the logic of the market in the delivery of those frontline operations covered in the contract? In arguing that these operations are characterized more by a complex blurring of logics than a straightforward unidirectional shift, the article contributes towards both explanatory and normative dimensions of the ‘transformation of policing’ debate.

The Transformation of Policing: From Ratios to Rationalities

Adam White, Martin Gill

Political Science and International Relations · The British Journal of Criminology · 27 Aug 2012 · cited 58× · DOI ↗ · more like this ·

A prominent feature of academic and popular writing about policing is the ‘transformation’ thesis: the contention that, as the ratio of private security to police actors increases in a policing system, the orientation of the system shifts from the public good to the market. The purpose of this article is to critique this thesis. Instead of analysing transformation using the ratio heuristic, it focuses on the everyday rationalities guiding policing actors. Applying this perspective to the British case, it argues that, rather than witnessing a marked shift towards the market, we are in fact seeing a complex blurring of identities, with both private security and police actors drawing upon a mix of public good and market rationalities to inform their actions.

The politics of police ‘privatization’: A multiple streams approach

Adam White

Political Science and International Relations · Criminology & Criminal Justice · 8 Sep 2014 · cited 53× · DOI ↗ · more like this ·

In October 2010, the UK Coalition government announced a 20 per cent reduction in the police budget over four years as part of its post-financial crisis Comprehensive Spending Review. In response to this new ‘politics of austerity’ many police forces began to consider outsourcing key services areas to the private sector on an unprecedented scale so as to make the required savings. However, this policy initiative was soon cut short. By March 2012 a prominent media debate on police ‘privatization’ was starting to heat up; in July 2012 the failure of G4S to meet the conditions of its Olympics security contract generated extensive negative publicity and triggered a parliamentary enquiry; and in the run-up to the November 2012 Police and Crime Commissioner elections many candidates could be found campaigning on an ‘anti-privatization’ platform – all of which had the effect of removing police…

Organizing Experience: A Deeper Look at Replay Mechanisms for Sample-based Planning in Continuous State Domains

Yangchen Pan, Muhammad Zaheer, Adam White +2 more

cs.AI · 12 Jun 2018 · cited 52× · arXiv ↗ · more like this ·

Model-based strategies for control are critical to obtain sample efficient learning. Dyna is a planning paradigm that naturally interleaves learning and planning, by simulating one-step experience to update the action-value function. This elegant planning strategy has been mostly explored in the tabular setting. The aim of this paper is to revisit sample-based planning, in stochastic and continuous domains with learned models. We first highlight the flexibility afforded by a model over Experience Replay (ER). Replay-based methods can be seen as stochastic planning methods that repeatedly sample from a buffer of recent agent-environment interactions and perform updates to improve data efficiency. We show that a model, as opposed to a replay buffer, is particularly useful for specifying which states to sample from during planning, such as predecessor states that propagate information in…

A Greedy Approach to Adapting the Trace Parameter for Temporal Difference Learning

Martha White, Adam White

cs.AI · July 2016 · cited 51× · arXiv ↗ · more like this ·

One of the main obstacles to broad application of reinforcement learning methods is the parameter sensitivity of our core learning algorithms. In many large-scale applications, online computation and function approximation represent key strategies in scaling up reinforcement learning algorithms. In this setting, we have effective and reasonably well understood algorithms for adapting the learning-rate parameter, online during learning. Such meta-learning approaches can improve robustness of learning and enable specialization to current task, improving learning speed. For temporal-difference learning algorithms which we study here, there is yet another parameter, $λ$, that similarly impacts learning speed and stability in practice. Unfortunately, unlike the learning-rate parameter, $λ$ parametrizes the objective function that temporal-difference methods optimize. Different choices of $λ$…

Does Early Sex Typing Result from Children's Attempts to Match Their Behavior to Sex Role Stereotypes?

David G. Perry, Adam White, Louise C. Perry

Education · Child Development · December 1984 · cited 48× · DOI ↗ · more like this ·

PERRY, DAVID G.; WHITE, ADAM J.; and PERRY, LOUISE C. Does Early Sex Typing Result from Children's Attempts to Match Their Behavior to Sex Role Stereotypes? CHILD DEVELOPMENT, 1984, 55, 2114-2121. Most cognitive theories of sex typing-including cognitive social learning theory and cognitive-developmental theory-suggest that sex typing results, in part, from a 2-step process: children are thought to acquire stereotypes of sex-appropriate behavior, primarily through observational learning; they are then thought to adopt as their own preferences the behaviors they have encoded as appropriate for their own sex. To determine if very early sex typing occurs in this sequence, 2-, 3-, 4-, and 5-year-olds were tested both for knowledge of sex role stereotypes and for preferences for sex-typed activities. Contrary to the hypothesized sequence, boys' acquisition of sex role stereotypes lagged…

DEVELOPING A PREDICTIVE APPROACH TO KNOWLEDGE

Adam White

Computer Science Applications · University of Alberta Library · 2015 · cited 47× · DOI ↗ · more like this ·

Understanding how an artificial agent may represent, acquire, update, and use large amounts of knowledge has long been an important research challenge in artificial intelligence. The quantity of knowledge, or knowing a lot, may be nicely thought of as making and updat- ing many predictions about many different courses of action. This predictive approach to knowledge ensures the knowledge is grounded in and learned from low-level data generated by an autonomous agent interacting with the world. Because predictive knowledge can be maintained without human intervention, its acquisition can potentially scale with available data and computing resources. The idea that knowledge might be expressed as prediction has been explored by Cunningham (1972), Becker (1973), Drescher (1990), Sutton and Tanner (2005), Rafols (2006), and Sutton (2009, 2012). Other uses of predictions include representing…

The Reinforcement Learning Competitions

Shimon Whiteson, B. K. Tanner, Adam White

Artificial Intelligence · AI Magazine · June 2010 · cited 45× · DOI ↗ · more like this ·

This article reports on the reinforcement learning competitions, which have been held annually since 2006. In these events, researchers from around the world developed reinforcement learning agents to compete in domains of various complexity and difficulty. We focus on the 2008 competition, which employed fundamentally redesigned evaluation frameworks that aimed systematically to encourage the submission of robust learning methods. We describe the unique challenges of empirical evaluation in reinforcement learning and briefly review the history of the previous competitions and the evaluation frameworks they employed. We describe the novel frameworks developed for the 2008 competition as well as the software infrastructure on which they rely. Furthermore, we describe the six competition domains, present selected competition results, and discuss the implications of these results. Finally…

Investigating practical linear temporal difference learning

Adam White, Martha White

cs.LG · February 2016 · cited 44× · arXiv ↗ · more like this ·

Off-policy reinforcement learning has many applications including: learning from demonstration, learning multiple goal seeking policies in parallel, and representing predictive knowledge. Recently there has been an proliferation of new policy-evaluation algorithms that fill a longstanding algorithmic void in reinforcement learning: combining robustness to off-policy sampling, function approximation, linear complexity, and temporal difference (TD) updates. This paper contains two main contributions. First, we derive two new hybrid TD policy-evaluation algorithms, which fill a gap in this collection of algorithms. Second, we perform an empirical comparison to elicit which of these new linear TD methods should be preferred in different situations, and make concrete suggestions about practical use.

Investigating the Properties of Neural Network Representations in Reinforcement Learning

Han Wang, Erfan Miahi, Martha White +5 more

cs.LG · March 2022 · cited 43× · arXiv ↗ · more like this ·

In this paper we investigate the properties of representations learned by deep reinforcement learning systems. Much of the early work on representations for reinforcement learning focused on designing fixed-basis architectures to achieve properties thought to be desirable, such as orthogonality and sparsity. In contrast, the idea behind deep reinforcement learning methods is that the agent designer should not encode representational properties, but rather that the data stream should determine the properties of the representation -- good representations emerge under appropriate training schemes. In this paper we bring these two perspectives together, empirically investigating the properties of representations that support transfer in reinforcement learning. We introduce and measure six representational properties over more than 25 thousand agent-task settings. We consider Deep Q-learning…

Twelve not so angry men: Inclusive masculinities in Australian contact sports

Ashnil Murray, Adam White

Gender Studies · International Review for the Sociology of Sport · 15 Oct 2015 · cited 38× · DOI ↗ · more like this ·

Sport’s utility in the development of a conservative orthodox ideal of masculinity based upon homophobia, aggression and emotional restrictiveness is well evidenced in critical masculinities scholarship. However, contemporary research is reflecting a more nuanced understanding of male behaviour in many Western contexts, with men performing softer and more inclusive versions of masculinities. Through exploring the experiences of twelve Australian contact sport athletes, this research establishes findings to support the growing body of inclusive masculinities research. Results show that these men value a softer representation of masculinity based upon pro-gay sentiments and being emotionally open, while often being critical of aspects of orthodox masculinities which male team sport previously promoted.

Reward-Respecting Subtasks for Model-Based Reinforcement Learning

Richard S. Sutton, Marlos C. Machado, G. Zacharias Holland +4 more

cs.LG · February 2022 · cited 33× · arXiv ↗ · more like this ·

To achieve the ambitious goals of artificial intelligence, reinforcement learning must include planning with a model of the world that is abstract in state and time. Deep learning has made progress with state abstraction, but temporal abstraction has rarely been used, despite extensively developed theory based on the options framework. One reason for this is that the space of possible options is immense, and the methods previously proposed for option discovery do not take into account how the option models will be used in planning. Options are typically discovered by posing subsidiary tasks, such as reaching a bottleneck state or maximizing the cumulative sum of a sensory signal other than reward. Each subtask is solved to produce an option, and then a model of the option is learned and made available to the planning process. In most previous work, the subtasks ignore the reward on the…

Masculinities, Media and the Rugby Mind: An Analysis of Stakeholder Views on the Relationship Between Rugby Union, the Media, Masculine-Influenced Views on Injury, and Concussion

Keith Parry, Adam White, Jamie Cleland +4 more

Gender Studies · Communication & Sport · 30 Jun 2021 · cited 31× · DOI ↗ · more like this ·

Rugby union, alongside other collision and contact sports, faces ever mounting pressure from increased recognition of concussive injuries and the risks they present to athletes, both in the short-term and long-term. Here, the media is a central component of increasing pressure for cultural change. This research analysed data from 524 self-selected survey respondents to examine rugby union fans’ and stakeholders’ perceptions of media portrayal of concussion and how it might influence their own perceptions. We found evidence of a complex and heterogenous relationship between perceptions of masculinity, views and attitudes toward mass media, and degree of involvement in rugby union. Specifically, partisans of the sport generally saw mass media as hostile, with coverage biased against rugby, allowing them to manufacture doubt regarding risk information, as well as maintaining involvement in…

General Value Function Networks

Matthew Schlegel, Andrew Jacobsen, Zaheer Abbas +3 more

cs.LG · July 2018 · cited 31× · arXiv ↗ · more like this ·

State construction is important for learning in partially observable environments. A general purpose strategy for state construction is to learn the state update using a Recurrent Neural Network (RNN), which updates the internal state using the current internal state and the most recent observation. This internal state provides a summary of the observed sequence, to facilitate accurate predictions and decision-making. At the same time, specifying and training RNNs is notoriously tricky, particularly as the common strategy to approximate gradients back in time, called truncated Back-prop Through Time (BPTT), can be sensitive to the truncation window. Further, domain-expertise--which can usually help constrain the function class and so improve trainability--can be difficult to incorporate into complex recurrent units used within RNNs. In this work, we explore how to use multi-step…

Interval Estimation for Reinforcement-Learning Algorithms in Continuous-State Domains

Martha White, Adam White

Artificial Intelligence · Neural Information Processing Systems · 6 Dec 2010 · cited 25× · OpenAlex ↗ · more like this ·

The reinforcement learning community has explored many approaches to obtaining value estimates and models to guide decision making; these approaches, however, do not usually provide a measure of confidence in the estimate. Accurate estimates of an agent’s confidence are useful for many applications, such as biasing exploration and automatically adjusting parameters to reduce dependence on parameter-tuning. Computing confidence intervals on reinforcement learning value estimates, however, is challenging because data generated by the agentenvironment interaction rarely satisfies traditional assumptions. Samples of valueestimates are dependent, likely non-normally distributed and often limited, particularly in early learning when confidence estimates are pivotal. In this work, we investigate how to compute robust confidences for value estimates in continuous Markov decision processes. We…

Adapting Behavior via Intrinsic Reward: A Survey and Empirical Study

Cam Linke, Nadia M. Ady, Martha White +2 more

Artificial Intelligence · Journal of Artificial Intelligence Research · 14 Dec 2020 · cited 24× · DOI ↗ · more like this ·

Learning about many things can provide numerous benefits to a reinforcement learning system. For example, learning many auxiliary value functions, in addition to optimizing the environmental reward, appears to improve both exploration and representation learning. The question we tackle in this paper is how to sculpt the stream of experience—how to adapt the learning system’s behavior—to optimize the learning of a collection of value functions. A simple answer is to compute an intrinsic reward based on the statistics of each auxiliary learner, and use reinforcement learning to maximize that intrinsic reward. Unfortunately, implementing this simple idea has proven difficult, and thus has been the focus of decades of study. It remains unclear which of the many possible measures of learning would work well in a parallel learning setting where environmental reward is extremely sparse or…

Questioning Anglocentrism in plural policing studies: Private security regulation in Belgium and the United Kingdom

Pieter Leloup, Adam White

Political Science and International Relations · European Journal of Criminology · 12 May 2021 · cited 20× · DOI ↗ · more like this ·

Recent Continental European scholarship has identified a problematic Anglocentric bias running through the field of comparative plural policing studies. It has sought to counter this bias by emphasizing a series of divergent plural policing trajectories between the more market-friendly countries in the Anglosphere and the more state-centric countries in Continental Europe. While acknowledging the significance of this corrective, we argue that it tends to overemphasize the levels of divergence between these two regions. We substantiate this claim by examining the rise of the private security industry and its regulation by the state in the UK (representing the Anglosphere) and Belgium (representing Continental Europe). Interpreting historical and contemporary data through Sabatier and Weible’s advocacy coalition framework, which focuses on the cut and thrust of democratic politics, we…

Contrastive Counterfactual Visual Explanations With Overdetermination

Adam White, Kwun Ho Ngan, James Phelan +4 more

cs.CV · June 2021 · cited 17× · arXiv ↗ · more like this ·

A novel explainable AI method called CLEAR Image is introduced in this paper. CLEAR Image is based on the view that a satisfactory explanation should be contrastive, counterfactual and measurable. CLEAR Image explains an image's classification probability by contrasting the image with a corresponding image generated automatically via adversarial learning. This enables both salient segmentation and perturbations that faithfully determine each segment's importance. CLEAR Image was successfully applied to a medical imaging case study where it outperformed methods such as Grad-CAM and LIME by an average of 27% using a novel pointing game metric. CLEAR Image excels in identifying cases of "causal overdetermination" where there are multiple patches in an image, any one of which is sufficient by itself to cause the classification probability to be close to one.

Fragile alliances: culture, funding and sustainability in police–academic partnerships

Matthew Bacon, Joanna Shapland, Layla Skinns +1 more

Political Science and International Relations · Evidence & Policy · 27 Feb 2020 · cited 16× · DOI ↗ · more like this ·

Background Police–academic partnerships have developed significantly over the past decade or so, spurred on by the expansion of the evidence-based policing movement, the increasing value attached to impactful research in the academy, the ascendance of the professionalisation agenda in the police, and the growing necessity of cross-sectoral collaborations under conditions of post-financial crisis austerity. This trend has given rise to a burgeoning literature in the discipline of criminology which is concerned with charting the progress of these partnerships and setting out the ideal conditions for their future expansion. Aims and objectives we advance a sympathetic critique of this literature, adding a note of caution to its largely optimistic outlook. Methods we do this by combining a narrative review of the literature on police–academic partnerships with insights from elsewhere in the…