A National Center for Validity of Educational Assessment and Measurement
Proposing VAL-Ed
Validity and Assessment in K12:
An Overhaul is Overdue
In the past 20-30 years, the U.S. K12 education system has seen significant innovation, thanks to more nuanced theories of learning and instructional models from the learning sciences, as well as technological developments and modes of R&D enabled by the data sciences. Unfortunately, educational assessment (both summative and formative) has failed to evolve alongside these innovations. We argue this lag is due to outdated views of assessment validity.*
Modernizing approaches to validity is an ideal access point to accelerate innovation (including addressing concerns of equity and bias in assessments) in both classroom and high-stakes assessment. More specifically, modernizing approaches to validity can increase confidence in:
1) alternative assessment approaches, which have been designed but have not been widely adopted
2) other forms of assessment where researchers and developers have been creating new approaches that are difficult to scale given traditional definitions and approaches to validity (e.g., accountability metrics)
The lack of innovation in assessment approaches has led to wide-scale dissatisfaction with the K12 education system across a range of stakeholders, including educators, students, and families.
*Validity is a fundamental aspect of developing and evaluating assessment designs, specifically referring to the degree to which evidence and theory support the interpretation of assessment outcomes in the context of their intended use. The concept of validity is deceptively simple: does an assessment accurately measure what is intended?
VAL-Ed Contributors
The below lists include individuals who have supported the ideation of developing and launching VAL-Ed.
Experts Consulted via Interview
(Fall 2022)
About VAL-Ed
In response to the challenges facing educational assessments, the Walton Family Foundation funded the Validity Landscape Analysis (VLA) in the fall of 2022. The VLA project was designed to explore how rethinking current validity ideas and methods could enable improvements in formative and summative assessment research, development, and implementation practices. The project undertook this exploration with two guiding questions in mind:
1) What approaches to validity need to be modernized and why?
2) What type of project or organization could be created to address validity issues in assessment and measurement, and via what activities?
To address these questions, VLA developed a preliminary landscape analysis drawing from interviews with field experts and stakeholders, as well as a scan of the literature. VLA also held a convening in Chicago before the annual 2023 AERA/NCME conference. The convening brought together a group of scholars, assessment and measurement practitioners, and funders to build off the landscape analysis and identify priorities for a potential R&D Center – the National Center for Validity of Educational Assessment and Measurement(VAL-Ed or the Center).
Convening participants were asked to identify a list of validity challenges that meet the following criteria:
should be addressed in educational assessment and measurement and
can be addressed via efforts in the next three-five years (i.e., where promising work exists, where activities and work products can be articulated and feasibly produced, etc.) and
are not being (sufficiently or collaboratively) addressed by other efforts
Using this criteria, and building on issues identified by educational assessment R&D stakeholders in prior work, the convening participants identified priorities for VAL-Ed, which have been consolidated into four research themes:
Validity of Assessment Systems
Validity Considerations of Generative AI in Assessment R&D
Validity of Measures of Educational Comparability
Validity Considerations of Assessment Content, Implementation, and Contextualization
What Would VAL-Ed Accomplish?
Within the four research themes, VAL-Ed would work to develop outputs within each of the key levers in assessment validity workflows, supporting the Cyclical R&D Workflow (pictured right).
New RFPs
Creation of new RFPs outside of traditional foci (coupled with funding) allow product development teams to develop necessary “out of the box” exemplars
New Empirical Evidence Base
Expansion of the empirical evidence base, which provides fodder for innovation as educational decision-makers need evidence to justify taking new approaches within assessment validity R&D
New Frameworks to Leverage Research
Response to calls of action for the development of new frameworks provides researchers with successful frameworks to leverage in the advancement of their own work
New Field Facing Guides
Informed field-facing documents guiding decision-making, technical working groups, funding opportunities, etc.
Cyclical R&D Workflow
What Are the Three VAL-Ed Workstrands?
VAL-Ed, as imagined by the work within VLA, would operate via three, interdependent workstrands (described below), designed to directly respond to the four research themes. VLA uncovered that siloed efforts constitute a bottleneck, field-wide, within educational assessment R&D. While good work is being accomplished, it is happening in isolation.
These workstrands have been designed to enable VAL-Ed members to “de-silo” the work and carry out critical research, providing the needed empirical evidence that innovative ideas, approaches, and methodologies are feasible and productive pathways towards changing the status quo within educational assessment. In other words, the three workstrands would support the Cyclical R&D Workflow (pictured above) that provides the field and its stakeholders with the necessary outputs to leverage innovative R&D work.
While the three VAL-Ed workstrands are presented in the order necessary to create stepping stones towards supporting the Cyclical R&D Workflow, these workstrands should be understood as interdependent and overlapping. For example, Strand One largely focuses on fostering community and networks within VAL-Ed, which will be an ongoing effort throughout the life of the Center.
Strand One:
Human/Social Capacity & Organization
Strand One will focus on building out networks and fostering new, multi-stakeholder partnerships – both of which are necessary to “de-silo” R&D work to support innovations in validity approaches, and thus assessment, in K12.
Strand Two:
Technical Tools and Resources
Strand Two will promote the development of new resources, i.e., mathematical models and conceptual frameworks, to support effective and innovative approaches to validity. These resources will be used to bolster the research/empirical base needed to carry out the Cyclical R&D Workflow, and will be made publicly available for the benefit of R&D initiatives outside of VAL-Ed.
Strand Three:
R&D Enablers
Strand Three will utilize the resources developed and sourced during Strand Two to create artifacts for the field to leverage in R&D work (see the Cyclical R&D Workflow), including: policy documents, field facing guidance, white papers, publications, RFPs, and proof of concept of new methods.
Get in touch
To ask questions, share any thoughts or comments, or join the mailing list for VAL-Ed, please fill out the contact form.