arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Sales

44 papers · 2007–2026

2007: 12008: 02009: 02010: 02011: 02012: 02013: 12014: 32015: 32016: 32017: 22018: 32019: 32020: 12021: 32022: 12023: 62024: 72025: 52026: 2
2007peak 7 in 20242026

Frequent co-authors

Papers

showing 1–25 of 44 RSS

Accessing Socially Excluded People — Trust and the Gatekeeper in the Researcher-Participant Relationship

Nick Emmel, Kahryn Hughes, Joanne Greenhalgh +1 more

Sociology and Political Science · Sociological Research Online · · cited 207× · DOI ↗ · more like this ·

This paper describes methodological findings from research to recruit and research hard-to-reach socially excluded people. We review the ways in which researchers have used particular strategies to access hard-to-reach individuals and groups and note that little attention has been given to understanding the implications of the nature of the trust relationship between researcher and participant. Gatekeepers invariably play a role in accessing socially excluded people in research, yet discussion to date invariably focuses on the instrumental role gatekeepers play in facilitating researchers’ access. In this paper we explore the possibilities for analysing relationships in terms of trust and distrust between gatekeeper and socially excluded participant. Our analysis considers the different kinds of relationships that exist between gatekeepers and socially excluded people and, in…

Depression among Mexican-origin mothers: Exploring the immigrant paradox.

Esther J. Calzada, Adam Sales

Sociology and Political Science · Cultural Diversity & Ethnic Minority Psychology · · cited 31× · DOI ↗ · more like this ·

OBJECTIVES: Residential time in the United States appears to be a risk factor for mental health problems among Latinos, according to studies of the immigrant paradox, or a pattern of findings documenting better health for immigrants relative to U.S.-born Latinos. The present study used an acculturative stress model to examine the immigrant paradox with a sample of Mexican-origin mothers. METHOD: Mothers [N = 175; age = 33.52 (10.75)], who were foreign- or U.S.-born, were categorized as more acculturated or less acculturated based on cluster analyses using demographic and language proficiency variables. The association between acculturative status and depressive symptoms approximately 9 months later was tested with two models: with both cultural stressors (i.e., perceived discrimination, acculturative stress) and cultural assets (i.e., familismo, ethnic identity) as mediators, and with…

Precise Unbiased Estimation in Randomized Experiments using Auxiliary Observational Data

Johann A. Gagnon-Bartsch, Adam C. Sales, Edward Wu +4 more

stat.AP · · cited 24× · arXiv ↗ · more like this ·

Randomized controlled trials (RCTs) are increasingly prevalent in education research, and are often regarded as a gold standard of causal inference. Two main virtues of randomized experiments are that they (1) do not suffer from confounding, thereby allowing for an unbiased estimate of an intervention's causal impact, and (2) allow for design-based inference, meaning that the physical act of randomization largely justifies the statistical assumptions made. However, RCT sample sizes are often small, leading to low precision; in many cases RCT estimates may be too imprecise to guide policy or inform science. Observational studies, by contrast, have strengths and weaknesses complementary to those of RCTs. Observational studies typically offer much larger sample sizes, but may suffer confounding. In many contexts, experimental and observational data exist side by side, allowing the…

A systematic analysis of the science of sandboxing

Michael Maass, Adam Sales, Benjamin Chung +1 more

Signal Processing · PeerJ Computer Science · · cited 23× · DOI ↗ · more like this ·

Sandboxes are increasingly important building materials for secure software systems. In recognition of their potential to improve the security posture of many systems at various points in the development lifecycle, researchers have spent the last several decades developing, improving, and evaluating sandboxing techniques. What has been done in this space? Where are the barriers to advancement? What are the gaps in these efforts? We systematically analyze a decade of sandbox research from five top-tier security and systems conferences using qualitative content analysis, statistical clustering, and graph-based metrics to answer these questions and more. We find that the term “sandbox” currently has no widely accepted or acceptable definition. We use our broad scope to propose the first concise and comprehensive definition for “sandbox” that consistently encompasses research sandboxes. We…

Review

Adam Sales

Statistics and Probability · Journal of Educational and Behavioral Statistics · · cited 22× · DOI ↗ · more like this ·

Causal mediation analysis is the study of mechanisms—variables measured between a treatment and an outcome that partially explain their causal relationship. The past decade has seen an explosion of research in causal mediation analysis, resulting in both conceptual and methodological advancements. However, many of these methods have been out of reach for applied quantitative researchers, due to their complexity and the difficulty of implementing them in standard statistical software distributions. The mediation package in R provides a set of simple commands that execute some of the newer causal mediation methods. This article will summarize some of the recent advances in mediation analysis, critically review the mediation package, and demonstrate, by example, some of its capabilities.

Rebar: Reinforcing a Matching Estimator with Predictions from High-Dimensional Covariates

Adam C Sales, Ben B Hansen, Brian Rowan

stat.ME · · cited 22× · arXiv ↗ · more like this ·

In causal matching designs, some control subjects are often left unmatched, and some covariates are often left unmodeled. This article introduces "rebar," a method using high-dimensional modeling to incorporate these commonly discarded data without sacrificing the integrity of the matching design. After constructing a match, a researcher uses the unmatched control subjects--the remnant--to fit a machine learning model predicting control potential outcomes as a function of the full covariate matrix. The resulting predictions in the matched set are used to adjust the causal estimate to reduce confounding bias. We present theoretical results to justify the method's bias-reducing properties as well as a simulation study that demonstrates them. Additionally, we illustrate the method in an evaluation of a school-level comprehensive educational reform program in Arizona.

The role of mastery learning in an intelligent tutoring system: Principal stratification on a latent variable

Adam Sales, John F. Pane

Artificial Intelligence · The Annals of Applied Statistics · · cited 20× · DOI ↗ · more like this ·

Students in Algebra I classrooms typically learn at different rates and struggle at different points in the curriculum—a common challenge for math teachers. Cognitive Tutor Algebra I (CTA1), an educational computer program, addresses such student heterogeneity via what they term “mastery learning,” where students progress from one section of the curriculum to the next by demonstrating appropriate “mastery” at each stage. However, when students are unable to master a section’s skills even after trying many problems, they are automatically promoted to the next section anyway. Does promotion without mastery impair the program’s effectiveness? At least in certain domains, CTA1 was recently shown to improve student learning on average in a randomized effectiveness study. This paper uses student log data from that study in a continuous principal stratification model to estimate the…

Toward Personalizing Students' Education with Crowdsourced Tutoring

Ethan Prihar, Thanaporn Patikorn, Anthony F. Botelho +2 more

Computer Science Applications · · cited 19× · DOI ↗ · more like this ·

As more educators integrate their curricula with online learning, it is easier to crowdsource content from them. Crowdsourced tutoring has been proven to reliably increase students' next problem correctness. In this work, we confirmed the findings of a previous study in this area, with stronger confidence margins than previously, and revealed that only a portion of crowdsourced content creators had a reliable benefit to students. Furthermore, this work provides a method to rank content creators relative to each other, which was used to determine which content creators were most effective overall, and which content creators were most effective for specific groups of students. When exploring data from TeacherASSIST, a feature within the ASSISTments learning platform that crowdsources tutoring from teachers, we found that while overall this program provides a benefit to students, some…

Limitless Regression Discontinuity

Adam C. Sales, Ben B. Hansen

stat.AP · · cited 18× · arXiv ↗ · more like this ·

Conventionally, regression discontinuity analysis contrasts a univariate regression's limits as its independent variable, $R$, approaches a cut-point, $c$, from either side. Alternative methods target the average treatment effect in a small region around $c$, at the cost of an assumption that treatment assignment, $\mathcal{I}\left[R<c\right]$, is ignorable vis a vis potential outcomes. Instead, the method presented in this paper assumes Residual Ignorability, ignorability of treatment assignment vis a vis detrended potential outcomes. Detrending is effected not with ordinary least squares but with MM-estimation, following a distinct phase of sample decontamination. The method's inferences acknowledge uncertainty in both of these adjustments, despite its applicability whether $R$ is discrete or continuous; it is uniquely robust to leading validity threats facing regression discontinuity…

The Influence of Discrimination and Coping Style on Blood Pressure Among Black/African American Women in the InterGEN Study

Michelle L. Wright, Sungju Lim, Adam Sales +4 more

Sociology and Political Science · Health Equity · · cited 11× · DOI ↗ · more like this ·

Purpose: Although research has explored the effects of racism on mental health, few studies have investigated the effects of racism on physical health. In this study, we examined the influence of racial discrimination and race-related stress and coping on blood pressure within a cohort of Black/African American women. Methods: This was a secondary data analysis of 226 Black/African American women from the Intergenerational Impact of Genetic and Psychological Factors on Blood Pressure study. Experiences of racial discrimination and coping, measured by the Experiences of Discrimination scale and the Race-Related Events Scale, were analyzed in relation to systolic blood pressure (SBP) and diastolic blood pressure (DBP). Multiple linear regression was used to explore the interaction effect of coping and discrimination on blood pressure for both scales. Results: Age and elevated body mass…

Using polytomous latent class analysis to compare patterns of substance use and co‐occurring health‐risk behaviors between students in alternative and mainstream high schools

Karen E. Johnson, Adam Sales, Lynn Rew +2 more

Sociology and Political Science · Journal of Adolescence · · cited 11× · DOI ↗ · more like this ·

INTRODUCTION: Alternative high school (AHS) students, an understudied and underserved population, experience educational, social, and health disparities relative to students in mainstream high schools. Disparities in single types of substance use are particularly high, yet no known studies have compared patterns of substance use or relationships between these patterns and other health-risk behaviors between AHS and mainstream high schools. METHODS: Using data from the Texas Alternative School Health Survey (n = 515; mean age 17.1 years, 49% male, 59% Hispanic, 23% White, 15% Black) and the Texas Youth Risk Behavior Survey (n = 2,113; mean age 16 years, 47% male, 64% Hispanic, 22% White, 7% Black), we used latent class analyses to compare patterns of substance use in AHSs and mainstream high schools. We used latent class regression to examine relationships between patterns of substance…

A Comparison of Faculty and Bibliometric Valuation of Serials Subscriptions at an Academic Research Library

Steven Knowlton, Adam Sales, Kevin W. Merriman

Information Systems · Serials Review · · cited 9× · DOI ↗ · more like this ·

AbstractDespite their professional training and study in the development of research collections in academic settings, librarians often consult with or even defer to faculty in selecting materials. Faculty often use various methods of evaluation that tend to emphasize qualitative data or even anecdotal evidence. Bibliometric analysis offers emerging tools to quantify these decisions, reflecting fundamental principles of library science. This study compares faculty choices of serials subscription cancellations to the choices that would have been predicted using a bibliometric tool, the California Digital Library Weighted Value Algorithm (CDL-WVA). Faculty choices differed significantly from the decisions predicted by CDL-WVA. However, as the bibliometric score increased, so did the rate of match between faculty choice and decisions predicted by CDL-WVA. Implications of these findings for…

How Common are Common Wrong Answers? Crowdsourcing Remediation at Scale

Ashish Gurung, Sami Baral, M. Lee +7 more

Statistics and Probability · · cited 7× · DOI ↗ · more like this ·

Solving mathematical problems is cognitively complex, involving strategy formulation, solution development, and the application of learned concepts. However, gaps in students' knowledge or weakly grasped concepts can lead to errors. Teachers play a crucial role in predicting and addressing these difficulties, which directly influence learning outcomes. However, preemptively identifying misconceptions leading to errors can be challenging. This study leverages historical data to assist teachers in recognizing common errors and addressing gaps in knowledge through feedback. We present a longitudinal analysis of incorrect answers from the 2015-2020 academic years on two curricula, Illustrative Math and EngageNY, for grades 6, 7, and 8. We find consistent errors across 5 years despite varying student and teacher populations. Based on these Common Wrong Answers (CWAs), we designed a…

Testing epistemic democracy’s claims for majority rule

William J. Berger, Adam Sales

Political Science and International Relations · Politics Philosophy & Economics · · cited 6× · DOI ↗ · more like this ·

While epistemic democrats have claimed that majority rule recruits the wisdom of the crowd to identify correct answers to political problems, the conjecture remains abstract. This article illustrates how majority rule leverages the epistemic capacity of the electorate to practically enhance the instrumental value of elections. To do so, we identify a set of sufficient conditions that effect such a majority rule mechanism, even when the decision in question is multidimensional. We then look to the case of sociotropic economic voting in US presidential elections to provide empirical tractability for these conditions. We find that absent such an epistemic capacity a number of presidential elections might well have been decided differently. By generating clear conditions for the plausibility of claims made by epistemic democrats, and demonstrating their correspondence to empirical data…

Student Log-Data from a Randomized Evaluation of Educational Technology: A Causal Case Study

Adam C Sales, John F Pane

stat.AP · · cited 6× · arXiv ↗ · more like this ·

Randomized evaluations of educational technology produce log data as a bi-product: highly granular data student and teacher usage. These datasets could shed light on causal mechanisms, effect heterogeneity, or optimal use. However, there are methodological challenges: implementation is not randomized and is only defined for the treatment group, and log datasets have a complex structure. This paper discusses three approaches to help surmount these issues. One approach uses data from the treatment group to estimate the effect of usage on outcomes in an observational study. Another, causal mediation analysis, estimates the role of usage in driving the overall effect. Finally, principal stratification estimates overall effects for groups of students with the same "potential" usage. We analyze hint data from an evaluation of the Cognitive Tutor Algebra I curriculum using these three…

The Effect of Assistance on Gamers: Assessing The Impact of On-Demand Hints &amp; Feedback Availability on Learning for Students Who Game the System

Kirk Vanacore, Ashish Gurung, Adam Sales +1 more

Computer Science Applications · · cited 5× · DOI ↗ · more like this ·

Gaming the system, characterized by attempting to progress through a learning activity without engaging in essential learning behaviors, remains a persistent problem in computer-based learning platforms. This paper examines a simple intervention to mitigate the harmful effects of gaming the system by evaluating the impact of immediate feedback on students prone to gaming the system. Using a randomized controlled trial comparing two conditions - one with immediate hints and feedback and another with delayed access to such resources - this study employs a Fully Latent Principal Stratification model to determine whether students inclined to game the system would benefit more from the delayed hints and feedback. The results suggest differential effects on learning, indicating that students prone to gaming the system may benefit from restricted or delayed access to on-demand support.…

Mastery Learning in Practice: A (Mostly) Descriptive Analysis of Log Data from the Cognitive Tutor Algebra I Effectiveness Trial

Anita Israni, Adam C Sales, John F Pane

stat.AP · · cited 4× · arXiv ↗ · more like this ·

Mastery learning, the notion that students learn best if they move on from studying a topic only after having demonstrated mastery, sits at the foundation of the theory of intelligent tutoring. This paper is an exploration of how mastery learning plays out in practice, based on log data from a large randomized effectiveness trial of the Cognitive Tutor Algebra I (CTAI) curriculum. We find that students frequently progressed from CTAI sections they were working on without demonstrating mastery and worked units out of order. Moreover, these behaviors were substantially more common in the second year of the study, in which the CTAI effect was significantly larger. We explore the various ways students departed from the official CTAI curriculum, focusing on heterogeneity between years, states, schools, and students. The paper concludes with an observational study of the effect on post-test…

The Role of Mastery Learning in Intelligent Tutoring Systems: Principal Stratification on a Latent Variable

Adam C. Sales, John F. Pane

stat.AP · · cited 3× · arXiv ↗ · more like this ·

Students in Algebra I classrooms typically learn at different rates and struggle at different points in the curriculum---a common challenge for math teachers. Cognitive Tutor Algebra I (CTA1), educational computer program, addresses such student heterogeneity via what they term "mastery learning," where students progress from one section of the curriculum to the next by demonstrating appropriate "mastery" at each stage. However, when students are unable to master a section's skills even after trying many problems, they are automatically promoted to the next section anyway. Does promotion without mastery impair the program's effectiveness? At least in certain domains, CTA1 was recently shown to improve student learning on average in a randomized effectiveness study. This paper uses student log data from that study in a continuous principal stratification model to estimate the…

Investigating the Impact of Skill-Related Videos on Online Learning

Ethan Prihar, Aaron Haim, Tracy Shen +4 more

Computer Science Applications · · cited 2× · DOI ↗ · more like this ·

Many online learning platforms and MOOCs incorporate some amount of video-based content into their platform, but there are few randomized controlled experiments that evaluate the effectiveness of the different methods of video integration. Given the large amount of publicly available educational videos, an investigation into this content's impact on students could help lead to more effective and accessible video integration within learning platforms. In this work, a new feature was added into an existing online learning platform that allowed students to request skill-related videos while completing their online middle-school mathematics assignments. A total of 18,535 students participated in two large-scale randomized controlled experiments related to providing students with publicly available educational videos. The first experiment investigated the effect of providing students with…

Remote Monitoring of Implementation Fidelity Using Log-File Data from Multiple Online Learning Platforms

Kirk Vanacore, Erin Ottmar, Allison S. Liu +1 more

Computer Science Applications · PsyArXiv (OSF Preprints) · · cited 2× · DOI ↗ · more like this ·

Implementation monitoring is an essential component of efficacy studies in education research. Yet traditional methods of fidelity of implementation monitoring can be costly and imprecise. In this paper, we use log-file data from an efficacy study assessing the impact of gamified learning technologies on seventh grade students’ algebraic knowledge, to measure the fidelity of implementations remotely within and across each programs. The COVID-19 pandemic disrupted plans for in-person observations, making reliance on remote monitoring imperative. We conducted an exploratory data analysis and estimated multi-level and latent profile models to explore the use of student level log file data to evaluate the qualities of their implementation contexts. Three metrics emerged as best suited to evaluate the students’ dosage and engagement in their respective programs. The percentage of assignments…

Enforcement of Employment Tribunal Awards

Emily Rose, Morag McDermont, Nicole Busby +2 more

Political Science and International Relations · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · · cited 2× · OpenAlex ↗ · more like this ·

This report examines the experiences of Employment Tribunal (ET) claimants after an ET award was made in their favour. A workers’ success in the ET is by no means the end of the story when it comes to achieving justice for a wrongdoing against them in the workplace. Enforcing an award made by an ET can itself be highly problematic . 1 The data presented in this report were collected as part of a European Research Council funded project entitled Citizens Advice Bureaux and Employment Disputes . 2 The overall aim of this project was to understand workers’ experiences as they attempted to resolve problems faced at work, including identifying potential barriers to justice. Our particular focus was on workers who could not easily afford the services of a solicitor. As such, participants were recruited through Citizens Advice Bureaux (CABx) who are a key provider of employment advice to this…

Math Problem Supports in an Authentic Classroom Context: Unpacking the Effect of Immediate Feedback and Re-Attempts in Online Problem Sets on Middle School Students’ Math Achievement

Hannah Smith, Erin Ottmar, Vy Ngo +4 more

Computer Science Applications · Journal of Research on Educational Effectiveness · · cited 1× · DOI ↗ · more like this ·

Using data from a randomized controlled trial within middle school classrooms, we examined whether the availability of immediate feedback with problem re-attempt, compared to delayed feedback on practice problem sets led to higher learning on math posttest achievement or state standardized assessments. Next, we examined if the availability of immediate hints and feedback on practice problem sets led to higher performance as measured by the percentage of problems completed or accuracy on problems compared to delayed feedback. Third, we examined if performance during practice mediated the effects of immediate feedback and hints with reattempts on math achievement and learning. We found no differences between the two problem support conditions on state standardized achievement, and weak evidence for a small positive effect of immediate re-attempt feedback on posttest performance. However…

E-TRIALS: Empowering Data-Driven Decisions to Enhance Computer-Based Learning Platforms

Abubakir Siedahmed, Yanping Pei, Adam C Sales +5 more

stat.AP · · cited 1× · arXiv ↗ · more like this ·

Computer-based learning platforms (CBLPs) have become a common medium in schools, transforming how students learn and interact with educational content. However, researchers still lack adequate tools to address the diverse set of challenges that students face in these environments. In this paper, we introduce \textbf{Ed-Tech Research Infrastructure to Advance Learning Sciences (E-TRIALS)}, a free tool developed by ASSISTments to help researchers conduct randomized controlled trials in the realm of learning sciences. We describe its features, the types of experiments it supports, and how it can address critical research questions. We showcase E-TRIALS' capabilities through two real-world interventions. Finally, we evaluate the efficacy of interventions using three average treatment effect (ATE) estimators. Student's t-test, regression, and Leave-One-Out Potential outcomes (LOOP). The…

Combining observational and experimental data for causal inference considering data privacy

Charlotte Z. Mann, Adam C. Sales, Johann A. Gagnon-Bartsch

stat.ME · · cited 1× · arXiv ↗ · more like this ·

Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive information might be tolerable to organizations that house confidential data. In these cases, organizations can employ data privacy techniques, which decrease disclosure risk, potentially at the expense of data utility. In this paper, we explore disclosure limiting transformations of observational data, which can be combined with experimental data to estimate the sample and population average treatment effects. We consider leveraging observational data to improve generalizability of treatment effect estimates when a randomized experiment (RCT) is…

Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results

Adam C. Sales, Ethan B. Prihar, Johann A. Gagnon-Bartsch +1 more

stat.ME · · cited 1× · arXiv ↗ · more like this ·

Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in small samples. However, often experimental samples and/or treatment effects are small, A/B tests are underpowered, and effect estimates are overly imprecise. Recent methodological advances have shown that power and statistical precision can be substantially boosted by coupling design-based causal estimation to machine-learning models of rich log data from historical users who were not in the experiment. Estimates using these techniques remain unbiased and inference remains exact without any additional assumptions. This paper reviews those methods and applies them to a new dataset including over 250 randomized A/B comparisons conducted within ASSISTments…