Assessment for CBE

Tips from Steve Klingler - WGU

(from http://nextgenlearning.org/blog/wgu-and-other-breakthrough-models-competency-means-rethinking-assessment?utm_source=Informz&utm_medium=Email+Marketing&utm_campaign=NextGen)

Klingler described key considerations for the measurement of competency:

1. It’s important to take a comprehensive research-based approach to design both of the competencies and their assessments.
   The subject matter experts who structure the competencies for WGU utilize criteria from a variety of sources including practical experience, job task analysis, published standards, and other research data. Different types of instruments are used, including performance tasks, objective exams, live observations, simulations, and portfolios. Instruments address each individual competency statement in a manner appropriate to the type of objective being assessed, with students receiving credit only after passing every assessment. Items are designed in a collaborative process that ensures they are truly accurate, objective and thorough, including a psychometric edit for alignment, clarity, sensitivity, bias, and fairness.
2. The development environment for assessments should be professional and well designed.
   All development at WGU takes place centrally, and workflow is carefully managed. It’s possible to look at the history of and prior versions of each item.
3. Objective assessments should be delivered in a standardized and secure way.
   For those assessments that involves taking objective tests, students use a secure, proctored online environment. The testing interface, which utilizes the Excelsoft SARAS assessment system, is customized to resemble other WGU systems, allowing students to focus on the questions rather than on the testing environment. An adaptive, tailored testing approach allows for the shortening of test times by curtailing the use of questions that are too hard or too easy for the  test-taker’s level. Computer scoring means that immediate results are available.
4. Online proctoring is the key to ensuring integrity and accurate results.
   WGU uses a combination of systems, Kryterion and ProctorU, to enable security of the testing envirionment. An advance readiness check allows students to ensure ahead of time that everything is working, reducing test anxiety. There are checks on identity that, for example, utilize biometrics, and on security via the lockdown of the browser, a wide-angle webcam, and audio and screen monitoring during the test.
5. Students should receive feedback in a manner appropriate to a program driven by competency.
   Rather than “failing,” students learn that they have not yet demonstrated competency. Results are presented in the form of a coaching report, in which test results are organized by course topic, which makes it feasible for the student and his/her advisor to modify his or her study plan appropriately for the next round.
6. Performance assessments require a distinct but equally thoughtful approach.
   WGU students also demonstrate competencies through culminating experiences in which they are presented with open-ended problems solvable in multiple ways, allowing them to apply their abilities in an approximate real-world context. Dedicated evaluators score these, using rubrics, and provide specific written feedback. WGU uses calibrations and monitors inter-rater reliability to ensure both accuracy and consistency.
7. Portfolios represent a third and especially comprehensive approach to competency-based assessment.
   At WGU, these may be new work or assembled from prior performance tasks. As with performance tasks, evaluators score them against a rubric. They can be shared with potential employers or others via a personal URL.