arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Summerville

33 papers · 2005–2023

2005: 12006: 02007: 02008: 02009: 02010: 02011: 02012: 02013: 02014: 02015: 32016: 82017: 102018: 32019: 32020: 32021: 12022: 02023: 1
2005peak 10 in 20172023

Frequent co-authors

Papers

showing 1–25 of 33 RSS

Procedural Content Generation via Machine Learning (PCGML)

Adam Summerville, Sam Snodgrass, Matthew Guzdial +5 more

cs.AI · · cited 474× · arXiv ↗ · more like this ·

This survey explores Procedural Content Generation via Machine Learning (PCGML), defined as the generation of game content using machine learning models trained on existing content. As the importance of PCG for game development increases, researchers explore new avenues for generating high-quality content with or without human involvement; this paper addresses the relatively new paradigm of using machine learning (in contrast with search-based, solver-based, and constructive methods). We focus on what is most often considered functional game content such as platformer levels, game maps, interactive fiction stories, and cards in collectible card games, as opposed to cosmetic content such as sprites and sound effects. In addition to using PCG for autonomous generation, co-creativity, mixed-initiative design, and compression, PCGML is suited for repair, critique, and content analysis…

Super Mario as a String: Platformer Level Generation Via LSTMs

Adam Summerville, Michael Mateas

cs.NE · · cited 164× · arXiv ↗ · more like this ·

The procedural generation of video game levels has existed for at least 30 years, but only recently have machine learning approaches been used to generate levels without specifying the rules for generation. A number of these have looked at platformer levels as a sequence of characters and performed generation using Markov chains. In this paper we examine the use of Long Short-Term Memory recurrent neural networks (LSTMs) for the purpose of generating levels trained from a corpus of Super Mario Brothers levels. We analyze a number of different data representations and how the generated levels fit into the space of human authored Super Mario Brothers levels.

Expanding Expressive Range: Evaluation Methodologies for Procedural Content Generation

Adam Summerville

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 57× · DOI ↗ · more like this ·

Procedural Content Generation (PCG) has been a part of video games for the majority of their existence and have been an area of active research over the past decade. How- ever, despite the interest in PCG there is no commonly ac- cepted methodology for assessing and analyzing a generator. Furthermore, the recent trend towards machine learned PCG techniques commonly state the goal of learning the design within the original content, but there has been little assess- ment of whether these techniques actually achieve this goal. This paper presents a number of techniques for the assess- ment and analysis of PCG systems, allowing practitioners and researchers better insight into the strengths and weaknesses of these systems, allowing for better comparison of systems, and reducing the reliance on ad-hoc, cherry-picking-prone tech- niques.

Toward Characters Who Observe, Tell, Misremember, and Lie

James Ryan, Adam Summerville, Michael Mateas +1 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 39× · DOI ↗ · more like this ·

Knowledge and its attendant phenomena are central to human storytelling and to the human experience more generally, but we find very few games that revolve around these concerns. This works to preclude a whole class of narrative experiences in games, and it also damages character believability. In this paper, we present an AI framework that supports gameplay with non-player characters who observe and form knowledge about the world, propagate knowledge to other characters, misremember and forget knowledge, and lie. We outline this framework through the lens of a gameplay experience that is intended to showcase it, called Talk of the Town, which we are currently developing. From a review of earlier projects, we find that our system has a novel combination of features found only independently across other systems, and that it is among the first to support character memory fallibility.

Understanding mario

Adam Summerville, Julian R. H. Mariño, Sam Snodgrass +2 more

Artificial Intelligence · · cited 37× · DOI ↗ · more like this ·

Evaluating the output of content generators is still one of the key open research challenges in Procedural Content Generation (PCG). This paper presents a collection of metrics for evaluating the quality of platform game levels, and analyzes how well these metrics are able to capture the human-perceived difficulty, visual aesthetics and enjoyment of these levels. We show empirically, in the context of Infinite Mario Bros (IMB), that some of the proposed metrics yield correlation values with human ratings that are near empirical upper bounds derived from a human inter-rater agreement study. We also show that a simple linear regression model using a subset of our metrics as input features is able to substantially outperform a previous approach that uses a neural network for predicting human-perceived difficulty, visual aesthetics, and enjoyment in IMB levels.

Draft-Analysis of the Ancients: Predicting Draft Picks in DotA 2 using Machine Learning

Adam Summerville, Michael Cook, Ben Steenhuisen

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 36× · DOI ↗ · more like this ·

Analysing strategic decision-making in eSports is an increasingly important problem -- for players, for teams, for commentators, for viewers and for broadcasters. Such analysis is extremely difficult, however, because of the comparatively small quantities of data, the ever-shifting state of competitive play, and the huge complexity of the game. In this paper we describe a system for predicting drafting decisions in DOTA 2, and evaluate both how the system performs compared to human experts, as well as the new kinds of analysis made possible by automation.

Exploring Level Blending across Platformers via Paths and Affordances

Anurag Sarkar, Adam Summerville, Sam Snodgrass +2 more

cs.LG · · cited 27× · arXiv ↗ · more like this ·

Techniques for procedural content generation via machine learning (PCGML) have been shown to be useful for generating novel game content. While used primarily for producing new content in the style of the game domain used for training, recent works have increasingly started to explore methods for discovering and generating content in novel domains via techniques such as level blending and domain transfer. In this paper, we build on these works and introduce a new PCGML approach for producing novel game content spanning multiple domains. We use a new affordance and path vocabulary to encode data from six different platformer games and train variational autoencoders on this data, enabling us to capture the latent level space spanning all the domains and generate new content with varying proportions of the different domains.

Learning Player Tailored Content From Observation: Platformer Level Generation from Video Traces using LSTMs

Adam Summerville, Matthew Guzdial, Michael Mateas +1 more

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 27× · DOI ↗ · more like this ·

A touted use of Procedural Content Generation is generating content tailored to specific players. Previous work has relied on human identification of player profile features which are then mapped to level generator features. We present a machine-learned technique to train generators on Super Mario Bros. videos, generating levels based on latent play styles learned from the video. We evaluate the generators in comparison to the original levels and a machine-learned generator trained using simulated players.

Gemini: Bidirectional Generation and Analysis of Games via ASP

Adam Summerville, Chris Martens, Ben Samuel +3 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 25× · DOI ↗ · more like this ·

Current approaches to game generation don't understand the games they generate. As a result, even the most sophisticated systems in this regard, e.g., Game-o-Matic, betray this problem — generating games with goals that are at odds with their mechanics. We describe Gemini, the first bidirectional game generation and analysis system. Gemini is able to take games as input, perform a proceduralist reading of them, and produce possible interpretations that the games might afford. By utilizing the declarative nature of Answer Set Programming (ASP), this analysis pathway opens up generation of games targeting specific interpretations and makes it possible to ensure the generated games are consistent with the desired reading. For Gemini, we developed a game specification language capable of expressing a larger domain of games than is possible with VGDL, the most widespread representation. We…

CHARDA: Causal Hybrid Automata Recovery via Dynamic Analysis

Adam Summerville, Joseph Osborn, Michael Mateas

cs.AI · · cited 22× · arXiv ↗ · more like this ·

We propose and evaluate a new technique for learning hybrid automata automatically by observing the runtime behavior of a dynamical system. Working from a sequence of continuous state values and predicates about the environment, CHARDA recovers the distinct dynamic modes, learns a model for each mode from a given set of templates, and postulates causal guard conditions which trigger transitions between modes. Our main contribution is the use of information-theoretic measures (1)~as a cost function for data segmentation and model selection to penalize over-fitting and (2)~to determine the likely causes of each transition. CHARDA is easily extended with different classes of model templates, fitting methods, or predicates. In our experiments on a complex videogame character, CHARDA successfully discovers a reasonable over-approximation of the character's true behaviors. Our results also…

Sampling Hyrule: Multi-Technique Probabilistic Level Generation for Action Role Playing Games

Adam Summerville, Michael Mateas

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 22× · DOI ↗ · more like this ·

Procedural Content Generation (PCG) using machine learning is a fast growing area of research. Action Role Playing Game (ARPG) levels represent an interesting challenge for PCG due to their multi-tiered structure and nonlinearity. Previous work has used Bayes Nets (BN) to learn properties of the topological structure of levels from The Legend of Zelda. In this paper we describe a method for sampling these learned distributions to generate valid, playable level topologies. We carry this deeper and learn a sampleable representation of the individual rooms using Principal Component Analysis. We combine the two techniques and present a multi-scale machine learned technique for procedurally generating ARPG levels from a corpus of levels from The Legend of Zelda.

Germinate: A Mixed-Initiative Casual Creator for Rhetorical Games

Max Kreminski, Melanie Dickinson, Joseph Osborn +3 more

Sociology and Political Science · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 21× · DOI ↗ · more like this ·

Digital games are hindered as an artform by significant technical barriers to entry, which exclude many would-be game developers from participating in this medium of expression. Casual creators for game design attempt to mitigate these barriers, but—like conventional game development tools—often require users to “work their way up” from low-level mechanics to high-level rhetorical or expressive goals, rather than allowing them to start with high-level rhetorical goals and “work their way down.” The constraint-based game generator Gemini is well-suited to the generation of games that meet high-level expressive goals, but is difficult for casual users to work with. We present Germinate, a mixed-initiative casual creator for rhetorical games that extends Gemini with a more approachable graphical user interface. A preliminary expert evaluation revealed that Germinate affords a playful…

Automatic Mapping of NES Games with Mappy

Joseph C. Osborn, Adam Summerville, Michael Mateas

cs.AI · · cited 21× · arXiv ↗ · more like this ·

Game maps are useful for human players, general-game-playing agents, and data-driven procedural content generation. These maps are generally made by hand-assembling manually-created screenshots of game levels. Besides being tedious and error-prone, this approach requires additional effort for each new game and level to be mapped. The results can still be hard for humans or computational systems to make use of, privileging visual appearance over semantic information. We describe a software system, Mappy, that produces a good approximation of a linked map of rooms given a Nintendo Entertainment System game program and a sequence of button inputs exploring its world. In addition to visual maps, Mappy outputs grids of tiles (and how they change over time), positions of non-tile objects, clusters of similar rooms that might in fact be the same room, and a set of links between these rooms. We…

MCMCTS PCG 4 SMB: Monte Carlo Tree Search to Guide Platformer Level Generation

Adam Summerville, Shweta Philip, Michael Mateas

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 21× · DOI ↗ · more like this ·

Markov chains are an enticing option for machine learned generation of platformer levels, but offer poor control for designers and are likely to produce unplayable levels. In this paper we present a method for guiding Markov chain generation using Monte Carlo Tree Search that we call Markov Chain Monte Carlo Tree Search (MCMCTS). We demonstrate an example use for this technique by creating levels trained on a corpus of levels from Super Mario Bros. We then present a player modeling study that was run with the hopes of using the data to better inform the generation of levels in future work.

Mystical Tutor: A Magic: The Gathering Design Assistant via Denoising Sequence-to-Sequence Learning

Adam Summerville, Michael Mateas

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 19× · DOI ↗ · more like this ·

Procedural Content Generation (PCG) has seen heavy focus on the generation of levels for video games, aesthetic content, and on rule creation, but has seen little use in other domains. Recently, the ready availability of Long Short Term Memory Recurrent Neural Networks (LSTM RNNs) has seen a rise in text based procedural generation, including card designs for Collectible Card Games (CCGs) like Hearthstone or Magic: The Gathering. In this work we present a mixed-initiative design tool, Mystical Tutor, that allows a user to type in a partial specification for a card and receive a full card design. This is achieved by using sequence-to-sequence learning as a denoising sequence autoencoder, allowing Mystical Tutor to learn how to translate from partial specifications to full.

Cooperation and Codenames: Understanding Natural Language Processing via Codenames

Andrew Kim, Maxim Ruzmaykin, Aaron H. Truong +1 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 17× · DOI ↗ · more like this ·

Codenames – a board game by Vlaada Chvátil – is a game that requires deep, multi-modal language understanding. One player, the codemaster, gives a clue to another set of players, the guessers, and the guessers must determine which of 25 possible words on the board correspond to the clue. The nature of the game requires understanding language in a multi-modal manner – e.g., the clue ‘cold’ could refer to temperature or disease. The recently proposed Codenames AI Competition seeks to advance natural language processing, by using Codenames as a testbed for multi-modal language understanding. In this work, we evaluate a number of different natural language processing techniques (ranging from neural approaches to classical knowledge-base methods) in the context of the Codenames AI framework, attempting to determine how different approaches perform. The agents are evaluated when working with…

Mechanics Automatically Recognized via Interactive Observation: Jumping

Adam Summerville, Joseph C. Osborn, Christoffer Holmgård +1 more

cs.AI · · cited 17× · arXiv ↗ · more like this ·

Jumping has been an important mechanic since its introduction in Donkey Kong. It has taken a variety of forms and shown up in numerous games, with each jump having a different feel. In this paper, we use a modified Nintendo Entertainment System (NES) emulator to semi-automatically run experiments on a large subset (30%) of NES platform games. We use these experiments to build models of jumps from different developers, series, and games across the history of the console. We then examine these models to gain insights into different forms of jumping and their associated feel.

From Mechanics to Meaning

Adam Summerville, Chris Martens, Sarah Harmon +4 more

Artificial Intelligence · IEEE Transactions on Games · · cited 15× · DOI ↗ · more like this ·

While generative approaches to game design offer great promise, systems can only reliably generate what they can “understand,” which is often represented in a limited, implicit form in hand-crafted evaluation functions or constructive rules. Proceduralist readings, a semiformal approach for interpreting the meaning of a game based on its underlying processes and interactions in conjunction with aesthetic and cultural cues, offer a novel, systematic approach to game understanding. We formalize proceduralist argumentation as a logic program that performs static reasoning over game specifications to derive higher level meanings, as part of Gemini, a bidirectional game analysis and generation system.

How to Tame Your Data: Data Augmentation for Dialog State Tracking

Adam Summerville, Jordan Hashemi, James Ryan +1 more

Artificial Intelligence · · cited 10× · DOI ↗ · more like this ·

Dialog State Tracking (DST) is a problem space in which the effective vocabulary is practically limitless. For example, the domain of possible movie titles or restaurant names is bound only by the limits of language. As such, DST systems often encounter out-ofvocabulary words at inference time that were never encountered during training. To combat this issue, we present a targeted data augmentation process, by which a practitioner observes the types of errors made on held-out evaluation data, and then modifies the training data with additional corpora to increase the vocabulary size at training time. Using this with a RoBERTa-based Transformer architecture, we achieve state-of-the-art results in comparison to systems that only mask trouble slots with special tokens. Additionally, we present a datarepresentation scheme for seamlessly retargeting DST architectures to new domains.

Studying the Effects of Training Data on Machine Learning-Based Procedural Content Generation

Sam Snodgrass, Adam Summerville, Santiago Ontañón

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 10× · DOI ↗ · more like this ·

The exploration of Procedural Content Generation via Machine Learning (PCGML) has been growing in recent years. However, while the number of PCGML techniques and methods for evaluating PCG techniques have been increasing, little work has been done in determining how the quality and quantity of the training data provided to these techniques effects the models or the output. Therefore, little is known about how much training data would actually be needed to deploy certain PCGML techniques in practice. In this paper we explore this question by studying the quality and diversity of the output of two well-known PCGML techniques (multi-dimensional Markov chains and Long Short-term Memory Recurrent Neural Networks) in generating Super Mario Bros. levels while varying the amount and quality of the training data.

Procedural Content Generation via Knowledge Transformation (PCG-KT)

Anurag Sarkar, Matthew Guzdial, Sam Snodgrass +3 more

cs.AI · · cited 8× · arXiv ↗ · more like this ·

We introduce the concept of Procedural Content Generation via Knowledge Transformation (PCG-KT), a new lens and framework for characterizing PCG methods and approaches in which content generation is enabled by the process of knowledge transformation -- transforming knowledge derived from one domain in order to apply it in another. Our work is motivated by a substantial number of recent PCG works that focus on generating novel content via repurposing derived knowledge. Such works have involved, for example, performing transfer learning on models trained on one game's content to adapt to another game's content, as well as recombining different generative distributions to blend the content of two or more games. Such approaches arose in part due to limitations in PCG via Machine Learning (PCGML) such as producing generative models for games lacking training data and generating content for…

Bad News

James Ryan, Adam Summerville, Ben Samuel

Sociology and Political Science · · cited 7× · DOI ↗ · more like this ·

Bad News is a game about death notification that combines deep simulation and live performance. After discovering a dead body, the player is tasked with tracking down a next of kin to inform him or her of the death. To do this, the player must discover the identities of both the deceased and next of kin, as well as the current location of the latter. Gameplay is underpinned by a rich simulation of a generic small town inhabited by several hundred non-player characters (NPCs) who build up subjective knowledge of one another as the simulation proceeds. The player interface is serviced by Wizard-of-Oz techniques, and the core gameplay interaction is embodied conversation with live-acted NPCs. Stationed out of sight from the player, a 'wizard' executes his or her commands (as they are spoken aloud) by live-coding modifications to the simulation. When the player encounters an NPC, an actor…

Automated Game Design Learning

Joseph C Osborn, Adam Summerville, Michael Mateas

cs.AI · · cited 6× · arXiv ↗ · more like this ·

While general game playing is an active field of research, the learning of game design has tended to be either a secondary goal of such research or it has been solely the domain of humans. We propose a field of research, Automated Game Design Learning (AGDL), with the direct purpose of learning game designs directly through interaction with games in the mode that most people experience games: via play. We detail existing work that touches the edges of this field, describe current successful projects in AGDL and the theoretical foundations that enable them, point to promising applications enabled by AGDL, and discuss next steps for this exciting area of study. The key moves of AGDL are to use game programs as the ultimate source of truth about their own design, and to make these design properties available to other systems and avenues of inquiry.

MappyLand: Fast, Accurate Mapping for Console Games

Joseph Osborn, Adam Summerville, Nathan Dailey +1 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 4× · DOI ↗ · more like this ·

We present MappyLand, a rewrite and enhancement of the earlier Mappy automatic game mapping system, which leverages instrumentation of a game console emulator to produce, from a sequence of game inputs, accurate annotations for action/adventure games including object detection and tracking, in-game camera movement, grid-based tile maps, and links between identified disparate spaces. The overhead of generating these annotations is on the order of one millisecond per observed frame. We also show a higher latency (but still online) algorithm for merging together previous observations of distinct game maps for the purposes of agent localization across a long period of time. Specifically, our system can determine a consistent graph of game rooms from a set of strings of game rooms, capturing behaviors like backtracking and synthesizing observations from multiple play sessions. Altogether…

Proceduralist Readings, Procedurally

Chris Martens, Adam Summerville, Michael Mateas +4 more

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · · cited 4× · DOI ↗ · more like this ·

While generative approaches to game design offer great promise, systems can only reliably generate what they can “understand,” often limited to what can be handencoded by system authors. Proceduralist readings, a way of deriving meaning for games based on their underlying processes and interactions in conjunction with aesthetic and cultural cues, offer a novel, systematic approach to game understanding. We formalize proceduralist argumentation as a logic program that performs static reasoning over game specifications to derive higher-level meanings (e.g., deriving dynamics from mechanics), opening the door to broader and more culturally-situated game generation.