Does it make sense to use written instruments to assess communication skills? Systematic review on the concurrent and predictive value of written assessment for performance

Abstract

Objectives - To evaluate possible associations between learners’ results in written and performance-based assessments of communication skills (CS), either in concurrent or predictive study designs.<p>

<p>Methods - Search included four databases for peer-reviewed studies containing both written and performance-based CS assessment. Eleven studies met the inclusion criteria.<p>

<p>Results - Included studies predominantly assessed undergraduate medical students. Studies reported mainly low to medium correlations between written and performance-based assessment results (Objective Structured Clinical Examinations or encounters with simulated patients), and gave correlation coefficients ranging from 0.13 to 0.53 (p < 0.05). Higher correlations were reported when specific CS, like motivational interviewing were assessed. Only a few studies gave sufficient reliability indicators of both assessment formats.<p>

<p>Conclusions - Written assessment scores seem to predict performance-based assessments to a limited extent but cannot replace them entirely. Reporting of assessment instruments’ psychometric properties is essential to improve the interpretation of future findings and could possibly affect their predictive validity for performance.<p>

<p>Practice implications - Within longitudinal CS assessment programs, triangulation of assessment including written assessment is recommended, taking into consideration possible limitations. Written assessments with feedback can help students and trainers to elaborate on procedural knowledge as a strong support for the acquisition and transfer of CS to different contexts.