arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Stuart Shaw

23 papers · 2002–2025

2002: 12003: 02004: 12005: 02006: 02007: 02008: 02009: 02010: 22011: 32012: 12013: 02014: 32015: 02016: 22017: 12018: 12019: 22020: 12021: 02022: 32023: 02024: 12025: 1
2002peak 3 in 20112025

Frequent co-authors

Papers

showing 1–23 of 23 RSS

E-portfolios in teaching, learning and assessment: tensions in theory and praxis

Emma Walland, Stuart Shaw

Education · Technology Pedagogy and Education · · cited 36× · DOI ↗ · more like this ·

E-portfolios are increasingly used in innovative ways, particularly in higher education, where they have the potential to transform teaching, learning and assessment. Given that students are learning in a hypertextual, digitalised and multimedia world, there is an ever-pressing need for assessment to be more authentic, engaging, and to develop transversal skills. This research analysed the challenges and opportunities of e-portfolios in secondary schools and universities. A review of the literature was undertaken in which studies conducted primarily across Europe and the US over the last 20 years were evaluated. Analysis identified several tensions relating to e-portfolio theory and praxis which were organised according to three main themes: the theory and research underpinning e-portfolios; the uses and purposes of e-portfolios; and the challenges and opportunities related to…

A purpose-led approach towards the development of competency frameworks

Simon Child, Stuart Shaw

Human Factors and Ergonomics · Journal of Further and Higher Education · · cited 26× · DOI ↗ · more like this ·

The concept of ‘competence’ has gained traction in recent years because of its power in informing broader educational objectives, such as readiness for transition to later educational stages. The responsibility for developing competencies in learners is increasingly laid onto educational institutions via accountability arrangements. As a result, competency frameworks are value-laden and motivated by interrelated purposes including construct definition, the development of assessment criteria, and workplace readiness. The present article introduces a methodological approach for constructing a competency framework that accounts for these underpinning motivations. The present article first introduces three binary distinctions that developers can utilise in making structural decisions on a competency framework: binary vs continuum; atomistic vs holistic, and context-specific vs…

Fair assessment viewed through the lenses of measurement theory

Isabel Nisbet, Stuart Shaw

Education · Assessment in Education Principles Policy and Practice · · cited 23× · DOI ↗ · more like this ·

Fairness in assessment is seen as increasingly important but there is a need for greater clarity in use of the term ‘fair’. Also, fairness is perceived through a range of ‘lenses’ reflecting different traditions of thought. The lens used determines how fairness is seen and described. This article distinguishes different uses of ‘fair’ which have relevance to assessment. It identifies some of the ‘lenses’ used to examine fairness in assessment and suggests a framework of questions which can be applied to each lens. It then applies this approach to the lens of theoretical writing about assessment measurement. It describes a broad consensus among measurement theorists in their account of fairness and critiques aspects of that view.

A framework for evidencing assessment validity in large-scale, high-stakes international examinations

Stuart Shaw, Victoria Crisp, N. C. Johnson

Education · Assessment in Education Principles Policy and Practice · · cited 17× · DOI ↗ · more like this ·

It is important for educational assessment bodies to demonstrate how they are seeking to meet the demands of validity. The approach to validity taken here assumes a ‘consequentialist’ view where the appropriacy of the inferences made on the basis of assessment results is seen as central. This paper describes the development of a systematic approach to the collection of evidence that can support claims about validity for general qualifications. An operational framework was developed drawing on Kane (2006 Kane, M.T. 2006. “Validation”. In Educational measurement, 4th ed., Edited by: Brennan, R.L. 17–64. Westport, CT: Praeger. [Google Scholar]). The framework involves a list of inferences to be justified as indicated by a number of linked validation questions. For each question various data would be gathered to provide ‘evidence for validity’ and to identify any ‘threats to validity’. The…

Fair high-stakes assessment in the long shadow of Covid-19

Isabel Nisbet, Stuart Shaw

Political Science and International Relations · Assessment in Education Principles Policy and Practice · · cited 11× · DOI ↗ · more like this ·

Fairness in assessment has become increasingly topical and controversial in recent years. Assessment theoreticians are writing more about fairness and assessment practitioners have developed processes and good practice to minimise unfairness. There is also increased scrutiny by students, parents and the wider public – not only of the fairness of assessments themselves and their outcomes, but of their use, notably for selection for college or university. This is in a context of continued awareness of inequalities in society and their impact on education and assessment. And on top of all these questions has been the impact – and the continuing long shadow – of Covid. Can there be fair assessment in such an unfair world?We consider three types of challenge to fair assessment:•Theoretical challenges•Challenges from thinking about social justice•Challenges from the way that statistics were…

What is computer-based testing washback, how can it be evaluated and how can this support practitioner research?

Martin Johnson, Stuart Shaw

Education · Journal of Further and Higher Education · · cited 10× · DOI ↗ · more like this ·

With the introduction of a new initiative in a teaching and learning environment there is an ethical responsibility to consider whether the impact of the introduction has met its intended goals, and whether it has harmed those who are influenced by it. Technology and infrastructure developments have encouraged a continued growth in the development and introduction of computer-based tests (CBTs) in educational environments. In the educational assessment literature, enquiry into the impact of testing (of all types) is known as ‘washback’. This is a reference to the way in which a test might have a range of influences on learners and teachers prior to the test-taking event. This article reviews the literature on CBT washback and outlines a framework for studying its effects as it is introduced into educational contexts. We then outline a research framework that we have developed (based on…

Assessment for Learning in International Contexts: exploring shared and divergent dimensions in teacher values and practices

Paul Warwick, Stuart Shaw, Martin Johnson

Education · The Curriculum Journal · · cited 10× · DOI ↗ · more like this ·

The Assessment for Learning in International Contexts (ALIC) project sought to extend knowledge around teachers’ understandings of Assessment for Learning (AfL). Using a modified version of a survey item devised by James and Pedder for use with teachers in England, evidence was gathered about the assessment practices that were highly valued by teachers across international contexts. The extent of congruence between these values and teachers’ reported classroom practices was explored and dimensions of teachers’ assessment practices were derived through factor analysis. While there was considerable congruence across the ALIC cohort of teachers and data sets derived from English teachers, particularly with respect to the items that have positive values–practice gaps, there were some interesting differences. Two components were derived from factor analysis, rather than the three derived by…

From Paper to Screen: some issues on the way

Nicholas Raikes, Jackie Greatorex, Stuart Shaw

Computer Science Applications · · cited 8× · OpenAlex ↗ · more like this ·

In the UK and elsewhere some examination agencies have programmes to move from an entirely paper-based examination system to a hybrid where paper scripts are scanned and digital images are distributed electronically for on-screen marking (scoring). This hybrid is seen as a way of realising some of the benefits of digital scripts in a context where paper is likely to remain important for many years to come. The University of Cambridge Local Examinations Syndicate (UCLES) first tried onscreen marking of scanned paper scripts in 1999. Results from this and subsequent trials were encouraging but inconclusive, and recently UCLES and its UK-trading subsidiary OCR launched a comprehensive programme of research, development and evaluation. In the present paper we share some of the issues that we believe will need to be investigated during the programme, and present results from UCLES’ first two…

UTILISING TECHNOLOGY IN THE ASSESSMENT OF COLLABORATION: A CRITIQUE OF PISA’S COLLABORATIVE PROBLEM SOLVING TASKS

Stuart Shaw, Simon Child

Computer Science Applications · EDULEARN proceedings · · cited 5× · DOI ↗ · more like this ·

When the outcome of individuals is affected by their own and others' actions. positive interdependence is when individuals believe that they can achieve their goals if other individuals achieve their goals as well. negative interdependence (or competition) is when individuals believe they can only achieve their goals if others fail. Implies a degree of synchronicity between group members, in that they are compelled to work together, and are thus motivated to do so

Should we be banking on it? Exploring potential issues in the use of ‘item’ banking with structured examination questions

Victoria Crisp, Stuart Shaw, Tom Bramley

Education · Assessment in Education Principles Policy and Practice · · cited 4× · DOI ↗ · more like this ·

Item banking involves tests being constructed by selecting from a bank of pre-written questions. There are various examples of multiple-choice tests where item banking is used, but few examples involving other question types. This research explored the use of banking with structured questions. Three question writers were asked to construct examination papers using a bank of structured physics questions and were interviewed. Factors considered when constructing examinations included the topic and skills to be assessed, the intended difficulty level of questions, and task type (e.g., definition, explanation, calculation, graph use). Participants’ strategies varied with one spending time viewing a range of questions, the second using information about the questions to narrow down which to view, and the third relying mostly on information about the questions. Several issues arose such as…

Applying methods to evaluate construct validity in the context of A level assessment

Victoria Crisp, Stuart Shaw

Education · Educational Studies · · cited 4× · DOI ↗ · more like this ·

Abstract Validity is a central principle of assessment relating to the appropriateness of the uses and interpretations of test results. Usually, one of the inferences that we wish to make is that the score reflects the extent of a student’s learning in a given domain. Thus, it is important to establish that the assessment tasks elicit performances that reflect the intended constructs. This research explored the use of three methods for evaluating whether there are threats to validity in relation to the constructs elicited in international A level geography examinations: (a) Rasch analysis; (b) analysis of processes expected and apparent when students answer questions; and (c) qualitative analysis of responses to items identified as potentially problematic. The results provided strong evidence to support validity with regard to the elicitation of constructs although one question part was…

Policy and practice in relation to external moderation of school-based assessment in 13 examination systems internationally

Damian Murchan, Stuart Shaw, Evgenia Likhovtseva

Education · Assessment in Education Principles Policy and Practice · · cited 1× · DOI ↗ · more like this ·

This study investigates how different jurisdictions implement external moderation of school-based assessment (SBA). Concerns about high-stakes examinations at upper secondary level prompt systems to incorporate SBA into their qualifications. While addressing issues of validity and student stress, SBA raises reliability concerns, potentially compromising trust in the qualifications. External moderation is frequently used to allay such concerns. This study identifies illustrations of moderation in secondary school exit examinations and investigates the local contexts in which they occur. A two-phase sequential design was used to explore the variables of interest across 13 jurisdictions using a review of publicly available documentation and interviews with examination officials in 9 examination organisations. Findings reveal how education systems use moderation to ensure consistency of…

Preparing for College Success: Exploring the Impact of the High School Cambridge Acceleration Program on U.S. University Students.

Stuart Shaw, Magda A. Werno

Education · College and university · · cited 1× · OpenAlex ↗ · more like this ·

Central to the growing debate about postsecondary education in the United States is the extent to which many students are underprepared for college-level study (Barnes et al. 2010; Barnes and Slate 2010; Camara 2013; Conley 2011; Kahlenberg 2010; Ravitch 2010; Rosenbaum, Stephan and Rosenbaum 2010; Symonds, Schwartz and Ferguson 2011; Zhao 2009, Zhao and Liu 2011).Attention in the United States has focused on the capability of high school acceleration strategies to produce students who are able to advance directly to college-level courses (Clinedinst, Hurley and Hawkins 2013; Judy, Ebmeyer and Schneider 2015; Killgore 2009; Rubin 2014). Acceleration programs are one of the main policy mechanisms for increasing college enrollment because they can have positive effects on cognitive strategies, content knowledge, and learning skills and techniques, as well as on the affective aspects of…

Assessing the Impact of the CAMBRIDGE INTERNATIONAL ACCELERATION PROGRAM on U.S. University Determinants of Success: A Multi-Level Modeling Approach

Stuart Shaw, J. Collins Warren, Tim Gill

Education · College and university · · cited 1× · OpenAlex ↗ · more like this ·

THIS ARTICLE FOCUSES ON THE RESEARCH BEING CONDUCTED BY CAMBRIDGE INTERNATIONAL EXAMINATIONS (CAMBRIDGE) TO ENSURE THAT ITS INTERNATIONAL ASSESSMENTS PREPARE STUDENTS AS WELL AS OTHER ACCELERATION PROGRAMS FOR CONTINUED STUDY IN U.S. COLLEGES AND UNIVERSITIES. THE STUDY, WHICH BUILDS ON PREVIOUS FRESHMAN G PA DATA MODELING WORK USING DATA SUPPLIED BY FLORIDA STATE UNIVERSITY, IS DESIGNED TO INVESTIGATE THE IMPACT OF THE CAMBRIDGE PROGRAM ON VARIOUS ASPECTS OF UNIVERSITY ENGAGEMENT.his study is part of an ongoing initiative to evaluate the predictive validity of the Cambridge acceleration program in the United States. Establishing predictive validity by relating high school performance to later academic performance has a long and established heritage in u.s. educational research (see, e.g., Burton and Ramist 2001; Cohn, Manion and Morrison 2007; Culpepper and Davenport 2009; Kobrin et…

Interpreting examiners' annotations on examination papers: a sociocultural analysis

Martin Johnson, Stuart Shaw

Education · Irish Educational Studies · · cited 1× · DOI ↗ · more like this ·

Abstract In Ireland and the UK it is accepted practice that agencies with formal responsibility for delivering school examinations allow examination candidates, and in many cases their teachers, to see their examination papers once they have been marked. Returned papers can carry various pieces of information; as well as the total score given for a performance, additional information is included in the form of the annotations left on the examination paper by the marking examiner. As far as we know there has been no research into how this information affects those who come into contact with it. The study uses teacher interview and survey data to explore whether a sociocultural approach to analysis can illuminate the factors that might influence their interpretation of those annotations. These analyses suggest that a key influence on the valid interpretation of an examiner's annotations…

A tricky task for teachers: assessing pre-university students' research reports

Stuart Shaw, Irenka Suto

Education · · cited 1× · OpenAlex ↗ · more like this ·

In England, some students preparing for university conduct independent research and write up lengthy reports which may contribute to nationally recognised qualifications. This provides an opportunity to investigate a specialist subject in depth, to cross boundaries with an inter-disciplinary enquiry, or to explore a novel non-school subject. However, multi-faceted challenges arise when assessment schemes are designed to reward generic research skills rather than subject knowledge. We investigated the feasibility of applying a single mark scheme, rewarding generic research skills, to research reports covering diverse topics. The study involved fifteen teachers with varied subject specialisms from a range of secondary schools across England. All teachers were currently or soon to be supporting 16-19 year olds to conduct independent research. Additionally, an experienced Chief Examiner…

Changes to Thinking about Education and Assessment

Isabel Nisbet, Stuart Shaw

Education · · cited 0× · DOI ↗ · more like this ·

The first part of this chapter considers changes to thinking about education, starting with the purposes of education and then reviewing theories of learning, with a 20th-century move from behaviourism to different forms of constructivism, followed by challenges to any constructivist consensus and a revival of support for “knowledge”. It then considers “competence” as an aim of education and a subject for assessment, and finally discusses “big data” and artificial intelligence. The second part considers thinking about assessment itself. It describes the impact of technology on conceptual ideas about assessment, and then outlines some approaches to assessment that reflect an increased recognition of the importance and relevance of the social context in which assessments take place. The authors support the view that it may be necessary to rethink the relationships between - and…

Research claims within the education industry: managing reflective practice

Sinéad Fitzsimons, Stuart Shaw

Education · London Review of Education · · cited 0× · DOI ↗ · more like this ·

The education industry is a far-reaching, innovative and rapidly evolving field of business. To ensure success and integrity in the education industry, organisations and companies strive to deliver high-quality products and services in an efficient and ethical manner. Education research plays an important part in the education industry by underpinning product and service developments, and through illustrating impact. Organisations and companies also share these research claims when marketing to potential customers and investors. However, there can sometimes exist a disjunction between those conducting research and those responsible for interpreting the research for the purpose of public dissemination. This article first investigates what constitutes an education research claim. The risks associated with such claims are then identified and a review process suggested so educational bodies…

Assessment for Learning in International Contexts: approaches and challenges in researching teacher values and practices

P. Warwick, Stuart Shaw, Martin Johnson

Education · · cited 0× · OpenAlex ↗ · more like this ·

This is an Accepted Manuscript of an article published by Taylor & Francis in The Curriculum Journal on the 6th of November 2014, available online: http://wwww.tandfonline.com/10.1080/09585176.2014.975732. This version will be under embargo until the 6th of May 2016.

Investigating the Impact of Cambridge International Assessments on U.S. Stakeholders: Student and Teacher Perceptions.

Stuart Shaw

Education · College and university · · cited 0× · OpenAlex ↗ · more like this ·

Investigating the Impact of Cambridge International Assessments on U.S. Stakeholders As part of the continuing program to study the impact of its international assessments, the University of Cambridge International Examinations ('Cambridge') has undertaken a series of studies investigating the impact on a range of US stakeholders. This paper reports on research designed to respond to a series of and impact questions seeking data on: the profiles of the candidates taking Cambridge assessments and teachers preparing students for the assessments; the of Cambridge assessments on courses preparing students; and the impact of Cambridge assessments on the students who have taken them. Data were collected on an opportunity sample of five case study high schools and two universities. The main findings of the study are presented according to the type of data collected: questionnaires, interviews…

How well can teachers assess pre-university students' research reports? Findings from an empirical study

Irenka Suto, Stuart Shaw

Education · · cited 0× · OpenAlex ↗ · more like this ·

General description: In the UK and internationally, many students preparing for university are given the challenge of conducting independent research and writing up a report of around 4000 or 5000 words. Such research activities provide students with opportunities to investigate a specialist area of study in greater depth, to cross boundaries with an inter-disciplinary enquiry, or to explore a novel non-school subject such as archaeology, cosmology or anthropology. We theorise that, as is the case in higher education (Brown et al. 1997), independent research encourages intellectual curiosity whilst enabling students to develop skills in practical and analytical research, higher order thinking, interpretation and time management. When applying to university, students can use their reports to demonstrate motivation for their intended course of study and to differentiate themselves from…