arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Michael Timms

17 papers · 2003–2018

2003: 12004: 02005: 12006: 02007: 12008: 02009: 02010: 32011: 12012: 42013: 02014: 12015: 22016: 12017: 02018: 2
2003peak 4 in 20122018

Frequent co-authors

Papers

showing 1–17 of 17 RSS

Science assessments for all: Integrating science simulations into balanced state science assessment systems

Edys Quellmalz, Michael Timms, Matt Silberglitt +1 more

Education · Journal of Research in Science Teaching · · cited 151× · DOI ↗ · more like this ·

Abstract This article reports on the collaboration of six states to study how simulation‐based science assessments can become transformative components of multi‐level, balanced state science assessment systems. The project studied the psychometric quality, feasibility, and utility of simulation‐based science assessments designed to serve formative purposes during a unit and to provide summative evidence of end‐of‐unit proficiencies. The frameworks of evidence‐centered assessment design and model‐based learning shaped the specifications for the assessments. The simulations provided the three most common forms of accommodations in state testing programs: audio recording of text, screen magnification, and support for extended time. The SimScientists program at WestEd developed simulation‐based, curriculum‐embedded, and unit benchmark assessments for two middle school topics, Ecosystems and…

The Evidence-Based Reasoning Framework: Assessing Scientific Reasoning

Nathaniel J. Brown, Erin Marie Furtak, Michael Timms +2 more

Education · Educational Assessment · · cited 111× · DOI ↗ · more like this ·

Recent science education reforms have emphasized the importance of students engaging with and reasoning from evidence to develop scientific explanations. A number of studies have created frameworks based on Toulmin's (1958/2003) argument pattern, whereas others have developed systems for assessing the quality of students' reasoning to support their scientific explanations. This article presents the centrepiece of this special issue, the Evidence-Based Reasoning Framework, which combines these two approaches to create an analytic tool intended as a foundation for assessing students' ability to reason from evidence in writing and classroom discussions. The article reviews previous frameworks developed to assess students' ability to reason scientifically and describes the elements of the Evidence-Based Reasoning Framework. It then provides an overview of the four articles in the special…

Challenges in STEM learning in Australian schools: Literature and policy review

Michael Timms, Kathryn Moyle, Paul R Weldon +1 more

Education · ACEReSearch (Australian Council for Educational Research) · · cited 84× · OpenAlex ↗ · more like this ·

This literature and policy review outlines the complex context related to STEM learning in Australian schools and focuses on student outcomes, the teacher workforce and the curriculum. This paper also sheds light on possible policy directions by examining lessons from other countries. STEM education is a broad enterprise that starts in early childhood education, continues through the years of schooling and extends into tertiary education supported by contributions from extracurricular and enrichment activities, science centres and museums. However, the focus in this document is on primary and secondary schooling. Australian STEM education seems caught in a whirlpool of problems that are contributing to one another. Student engagement and performance in STEM are declining, but we do not have the supply of qualified teachers we need to improve learning. The STEM curriculum is unbalanced…

Investigating the dynamics of formative assessment: relationships between teacher knowledge, assessment practice and learning

Joan L. Herman, Ellen Osmundson, Yunyun Dai +2 more

Education · Assessment in Education Principles Policy and Practice · · cited 55× · DOI ↗ · more like this ·

This exploratory study of elementary school science examines questions central to policy, practice and research on formative assessment: What is the quality of teachers’ content-pedagogical and assessment knowledge? What is the relationship between teacher knowledge and assessment practice? What is the relationship between teacher knowledge, assessment practice and student learning? Drawing on multiple measures, hierarchical linear modelling and path analysis, results suggest that despite weaknesses in teachers’ content-pedagogical and assessment knowledge, teachers’ formative assessment practices are positively related to student learning. Relationships between teachers’ knowledge and assessment practices are mixed. Findings underscore both the potential and challenge of bringing effective formative practice to fruition as well as the need for continued research.

A Framework for Analyzing Scientific Reasoning in Assessments

Nathaniel J. Brown, Sam O. Nagashima, Alice C. Fu +2 more

Education · Educational Assessment · · cited 55× · DOI ↗ · more like this ·

The Evidence-Based Reasoning Assessment System (EBRAS) brings together advances in modeling scientific reasoning and assessment design to guide the development of written assessment items that target, disentangle, and elicit evidence of the multiple proficiencies underlying scientific argumentation. In this study, the EBRAS was used to assess the conceptual sophistication, specificity, and validity of middle and high school students' reasoning on the topic of buoyancy. Results imply items designed using the EBRAS are more reliable in formative contexts and can better reveal misconceptions, rhetorical strategies, and specific logical errors than traditional items that focus on accuracy. The EBRAS is proposed as a more valid, reliable, and useful means for analyzing and assessing students' use of evidence in scientific reasoning.

The nature and impact of teachers’ formative assessment practices

Joan L. Herman, Ellen Osmundson, Carlos Ayala +2 more

Education · · cited 52× · OpenAlex ↗ · more like this ·

Theory and research suggest the critical role that formative assessment can play in student learning. The use of assessment in guiding instruction has long been advocated: Through the assessment of students’ needs and the monitoring of student progress, learning sequences can be appropriately designed, instruction adjusted during the course of learning, and programs refined to be more effective in promoting student learning goals. Moving toward more modern pedagogical conceptions, assessment moves from an information source on which to base action to part and parcel of the teaching and learning process. The following study provides food for thought about the research methods needed to study teachers’ assessment practices and the complexity of assessing their effects on student learning. On the one hand, our study suggests that effective formative assessment is a highly interactive…

Challenges in STEM learning in Australian schools

Michael Timms, Kathryn Moyle, Paul R Weldon +1 more

Education · ACEReSearch (Australian Council for Educational Research) · · cited 49× · OpenAlex ↗ · more like this ·

Australian STEM education seems caught in a whirlpool of problems that are contributing to one another. Student engagement and performance in STEM are declining, but we do not have the supply of qualified teachers we need to improve learning. The STEM curriculum is unbalanced and fragmented, leading to less interest among students. It is not possible to break out of the downward cycle from within the current system and it requires policy changes that address the issues raised in this report. This means developing well-considered, systemic and joined-up policies that address the following challenges: Improving student outcomes, building the STEM teacher workforce, and rethinking the STEM curriculum. This issue of Policy Insights is informed by a literature and policy review undertaken by the authors in 2017, as well as by the key messages of the ACER 2016 Research Conference, ‘Improving…

Towards a model of how learners process feedback: A deeper look at learning

Michael Timms, Sacha DeVelle, Dulce Lay

Artificial Intelligence · Australian Journal of Education · · cited 37× · DOI ↗ · more like this ·

It is well known that learners using intelligent learning environments make different use of the feedback provided by the intelligent learning environment and exhibit different patterns of behaviour. Traditional approaches to measuring such behaviour have focused on observational methods, think-aloud protocols, ratings and log data. More recently, the field of educational neuroscience has placed greater emphasis on real-time measures using eye tracking, electroencephalogram and functional magnetic resonance imaging. Our work fits into the latter vein. Drawing on a literature review from cognitive psychology, neuroscience and education, we describe our Learner Processing of Feedback in Intelligent Learning Environments model of how learners process feedback. We also present findings from a pilot study as a preliminary test of the model. Seventeen learners participated in an experiment…

ChemVLab+ : evaluating a virtual lab tutor for high school chemistry

Jodi L. Davenport, Anna N. Rafferty, Michael Timms +2 more

Education · ICLS · · cited 20× · OpenAlex ↗ · more like this ·

Conceptual understanding in chemistry requires students to connect quantitative calculations, microscopic chemical processes (e.g., atoms and molecules), and macroscopic outcomes (e.g., concentrations, color, temperature). The ChemCollective Virtual Lab's interactive simulation environment fosters the mapping between these representations by providing students with a fully functional online chemistry lab with realistic chemicals, glassware, and equipment. In the ChemVLab+ project (www.chemvlab.org), we enhance the virtual lab by developing intelligent tutoring for this open-ended simulation environment and creating embedded formative assessment activities. Students have opportunities to apply chemistry knowledge to meaningful contexts and receive immediate, individualized feedback as the system estimates their proficiency based on their actions. We report findings from a classroom field…

The promise of simulation-based science assessment: the Calipers project

Edys Quellmalz, Michael Timms, Barbara C. Buckley

Education · International Journal of Learning Technology · · cited 15× · DOI ↗ · more like this ·

The Calipers project developed and studied a new generation of simulation-based assessment systems. The project aimed to demonstrate the potential of technology- and simulation-based assessments to provide high-quality evidence of complex performances for science tests that address accountability or formative goals. End-of-unit, benchmark assessments for the topics of ecosystems and for forces and motion were developed to test national science standards at the middle school and secondary levels. Technical quality evidence documented the alignment of the assessments with national science standards, expert reviews of content and item quality, cognitive analyses of students thinking-aloud, and analyses of teacher and student data gathered from classroom pilot testing. The project broke new ground in harnessing the affordances of technology to transform what, how, when and where science…

Using Item Response Theory (IRT) to select hints in an ITS

Michael Timms

Artificial Intelligence · · cited 13× · OpenAlex ↗ · more like this ·

Many Intelligent Tutoring Systems (ITSs) are designed to tutor students by providing error feedback and hints as students solve a series of problems, where the tutoring system's feedback is based on a model of the student's ability in the subject domain. Few ITSs, however, model the difficulty of the problems the student is solving, and those that do rarely utilize an empirical approach. Item Response Theory, a methodology applied in educational measurement, offers a way to empirically model both a student's ability and the difficulty of the problem on a common scale. This paper first describes a method for using Item Response Theory (IRT) to model the difficulty of the problems (items), which allows the system to determine the level of hints that students need during problem solving. Second, the paper reports on how the method was used in the FOSS Self-assessment System, an ITS…

New measurement paradigms

Michael Timms, Douglas H. Clements, Janice D. Gobert +4 more

Artificial Intelligence · ACEReSearch (Australian Council for Educational Research) · · cited 11× · OpenAlex ↗ · more like this ·

This collection of New Measurement Paradigms papers represents a snapshot of the variety of measurement methods in use at the time of writing across several projects funded by the National Science Foundation (US) through its REESE and DR K–12 programs. All of the projects are developing and testing intelligent learning environments that seek to carefully measure and promote student learning, and the purpose of this collection of papers is to describe and illustrate the use of several measurement methods employed to achieve this. The papers are deliberately short because they are designed to introduce the methods in use and not to be a textbook chapter on each method. The New Measurement Paradigms collection is designed to serve as a reference point for researchers who are working in projects that are creating e-learning environments in which there is a need to make judgments about…

A Web-based System to Support Inquiry Learning - Towards Determining How Much Assistance Students Need

Bruce M. McLaren, Michael Timms, Doug Weihnacht +4 more

Education · · cited 5× · DOI ↗ · more like this ·

How much assistance should be provided to students as they learn with educational technology? Providing help allows students to proceed when struggling, yet can depress their motivation to learn independently. Assistance withholding encourages students to learn for themselves, yet can also lead to frustration. The web-based inquiry-learning program, Voyage to Galapagos (VTG), helps students “follow” the steps of Darwin through a simulation of the Galapagos Island. Students explore the islands, take pictures of animals, evaluate their characteristics and behavior, and use scientific methodology to “discover” evolution. A preliminary study with 48 middle school students examined three levels of assistance: (1) no support, (2) error flagging, text feedback on errors, and hints, and (3) pre-emptive hints with error flagging, error feedback, and hints. The results indicate that…

Assessment in interactive learning environments

Michael Timms, Jason M. Lodge

Education · ACEReSearch Repository (Australian Council for Educational Research) · · cited 0× · OpenAlex ↗ · more like this ·

There is an increasing interest in using digital technologies to create interactive learning environments (ILEs) that both teach and assess student skills that are hard or impossible to assess using ‘static’ items such as traditional, multiple-choice questions. These interactive learning environments try to do two things simultaneously: firstly, to monitor the learning of the student in real time, providing feedback to help the student progress through the learning task; and secondly, to use the information gathered during the learning to make judgements about where the student is in learning of the topic. Essentially, ILEs draw upon the same source of data — the interactions of the student with the learning materials and embedded assessment tasks — to perform these measurements. To make these kinds of decisions, ILEs collect and analyse many variables; the complexity of these data…

Research Article Science Assessments for All: Integrating Science Simulations Into Balanced State Science Assessment Systems

Edys Quellmalz, Michael Timms, Matt Silberglitt +1 more

Education · · cited 0× · OpenAlex ↗ · more like this ·

This article reports on the collaboration of six states to study how simulation-based sci- ence assessments can become transformative components of multi-level, balanced state science assess- ment systems. The project studied the psychometric quality, feasibility, and utility of simulation-based science assessments designed to serve formative purposes during a unit and to provide summative evi- dence of end-of-unit proficiencies. The frameworks of evidence-centered assessment design and model- based learning shaped the specifications for the assessments. The simulations provided the three most common forms of accommodations in state testing programs: audio recording of text, screen magnifica- tion, and support for extended time. The SimScientists program at WestEd developed simulation-based, curriculum-embedded, and unit benchmark assessments for two middle school topics, Ecosystems and…

Affordances of Simulation-based Science Assessments

Jodi L. Davenport, Edys Quellmalz, Michael Timms

Education · eScholarship (California Digital Library) · · cited 0× · OpenAlex ↗ · more like this ·

What are the affordances of simulation-based assessments for eliciting science inquiry skills?To test the affordances, WestEd's SimScientists program created 3 versions (static, active and interactive) of computer-based Ecosystems assessments.In the static version, students viewed still images on the screen.In the active version, students viewed animations, but did not conduct active investigations.In the interactive version, students designed and ran their own experiments.All versions assessed the same science inquiry skills (e.g., making observations, designing experiments, evaluating predictions etc.).Eight middle-school students completed each assessment version while thinking aloud.On items that required declarative knowledge, students performed similarly across versions.However, on items requiring more sophisticated reasoning (e.g., experimental design), students performed better…