Areas
As
organizations matures their use of assessments they will focus on one or
more of the three Areas to move from a lower to a higher level of
maturity. This section explores each of these Areas in more depth, with
the aim of helping organizations identify and respond to the challenges
of successfully deploying assessments (either internally or externally)
consistently, predictably, cost effectively, efficiently, and reliably.
Within the three Areas there are some key issues to be concerned with:
Development (of Assessments)
- Assessments are developed to be of high quality.
- Assessments will move from being inefficient and random in the first
phase of the model to being efficient and predictable in the fourth.
Delivery (Management)
- Assessments delivered securely, consistently and reliably.
- Delivery will move from being insecure, inconsistent and unreliable
in the first phase to trustworthy and predictable in the fourth.
Presenting Results (Reporting)
- Assessment results delivered securely to the right person using informative presentation
- In the first phase, results will be difficult to obtain and interpret. In the last, they will be aligned with organization goals and fully actionable.
More on Development
Create and maintain assessments to be:
Secure, valid, reliable, and defensible are the attributes to help stakeholders to trust the results. Focus on these attributes promote the creation of defensible assessments resulting in sound decisions.
- Security is a vital component of effective assessments. It is necessary at all stages, from authoring and scheduling to delivery, reporting and analysis. Protecting assessment content, administering assessments appropriately, and storing results effectively maintains the integrity of the assessment program and helps maintain stakeholders’ confidence in it.
- Valid and reliable assessments measure what they are
intended to and consistently measure the same thing over time, during
multiple administrations. We can put our trust in such assessments,
which in turn give us consistent, trustable data. Reliable harvesting
and analysis of information leads to valid analysis based on
statistically valid principles.
This requires: Clearly established objectives. Does the organization want to enable learning, retention and/or performance? Is the objective to measure knowledge skills, abilities and/or attitudes? Or is related to compliance? Does the organization need to harvest data and represent results to demonstrate compliance?
Items that are on topic, cost-effective and appropriate: Careful thought must be given to questions, choices, scoring mechanisms, accommodations, and media such as graphics, sound and video.
- Defensibility is essential when it comes to granting degrees, hiring, promotion, demotion and firing, because the actions require that evidence be available to show that the assessment works in sympathy with the job role or course curriculum. In the area of accreditation and regulatory compliance this means that the data gathering and reporting provide evidence of compliance
Efficiency is an important aspect of Assessment development to ensure continued engagement of Subject Matter Experts and other Stakeholders.
- Subject Matter Experts, Producers, consumers and stakeholders within an organization and within the extended organization have jobs to do, or studies to complete, so the assessment process should serve stakeholder needs rather than distract them. Time, money and energy should not be wasted in the assessment development process. Work flow processes enable predictable production of high-quality deliverables. Accurate estimates of costs, deliverable and timescales help organizations gauge a project’s viability. Deliverables need to be on time, within budget and consistent with the organization’s requirements.
Effective development of assessments is required to
allow an organization to reliably analyze the expected benefits of
proposed projects and end up with the results they need by the time they
need them.
More On Delivery
Deliver the right stimulus with the right levels of security to harvest data appropriately and effectively. This part involves numerous people (mostly candidates, respondents, participants, etc); good communications to set expectations and predictable processes to enhance satisfaction are key to success.More on Presenting Results
Presenting Results to the right person, in the right context, with the right presentation and at the right time in a fast-paced world is essential. Excellent information delivered too late has little or no value. It is simply too late! And information that is not analyzed accurately or delivered appropriately is likewise wasted. Analyzing data appropriately and delivering it using the right presentation, using the right analysis, to the right person at the right time is achieved by:- Collecting the results and using statistically sound principles to calculate useful and meaningful statistics.
- Representing data in a way that empowers the audience and the organization, providing guidance based on sound and informed interpretation.
- Presenting results in a useful and actionable way. Too much, too little or poorly represented data hinders effective action.
- Conveying information in a way that suits each
person’s wants and needs:
- A candidate wants a certificate.
- A psychometrician wants data to analyze to determine if the items and assessments are valid and reliable.
- A program manager wants to know how the certification program is doing.
- A decision maker wants aggregated survey results with comparisons to other departments units.
- An executive wants to see the impact of activities to learning outcomes and/or performance.
- Effectively presenting the analysis of results:
- Analysis and communication of results needs to fit within the context of both the assessment and the consumer of the information.
- Enabling effective and useful comparisons (compare apples with apples!)
This website is maintained by