Measuring and Developing Professionalism through Situational Judgment Tests and Behavior-Based Assessments: A Practical Guide and Workshop

While it is critical to assess knowledge and skills of medical professionals for board certification, it is recognized that professional manner can be equally important for successful practice. Psychometricians are accustomed to developing tests to measure knowledge, but crafting an assessment that objectively assesses professional attributes and skills (e.g., empathy, integrity, teamwork) can be an elusive endeavor. To tap into this content, behavioral assessments are appropriate, and situational judgment tests (SJTs) offer a more approachable solution than true interactive assessments. SJTs can be scored in multiple ways to achieve different goals, and typically require a lower time and cost investment to create. To be fair, robust SME input is required to ensure the scenarios and response options have high fidelity, but delivering the scenarios in text form is less expensive than producing video vignettes or live simulations. Because SJTs can assess complex constructs efficiently, they pose a powerful alternative to more labor-intensive methods such as objective structured clinical examinations (OSCEs) or multiple mini interviews (MMIs). SJT development is unlike knowledge exam development, notably because there is no factually correct answer. The purpose of this session is to equip attendees with the ability to contemplate and implement SJTs. Other behavioral assessment options will be discussed as well, including research findings that support their use for assessment of professionalism, step-by-step guidelines for building an SJT, considerations for scoring, and benefits for examinees and stakeholders.