SHOW Me What You Can DO

Writing performance testing items for certification exams requires careful consideration

- BEVERLY VAN DE VELDE

There are several approaches to measuring the knowledge, skills, and abilities of a candidate who wants to prove their competency by achieving a certification credential. In recent years, many exam creators have shifted away from achieving these measurements by using traditional exam metrics such as multiple-choice questions.

One increasingly popular method of measuring knowledge, skills, abilities is through the use of “performance” items that require an exam candidate to perform a specific task or solve a specific problem. The purpose of this article is to address several of the considerations and goals of performance test item writing.

While I will cover many topics, I will also exclude many details, such as how to define competencies and job tasks, psychometrics, types of performance tests, test administration, policies, infrastructure, deployment considerations, scoring methods, scalability, archiving, security, localization, maintenance, and costs. Should you have questions regarding these details, I encourage you to seek out other resources.

Validity and reliability

Regardless of the structure of an exam, I recommend a job task analysis to identify competencies (i.e., knowledge, skills, and/or abilities) and to ensure that all items map back to the blueprint. It is the item writer’s responsibility to ensure that each item measures what is intended to be measured, as well as the frequency to which competency is measured via relevant tasks.

Reliability can be tricky in performance tests and is often a source of debate amongst psychometricians for various reasons. For example, sometimes a performance test requires fewer measurements than a traditional exam. Or, a performance test might have a mix of highly related tasks measured on the outcome of tasks broken out into several points of measurement.