Use of Artificial Intelligence to Assess Trainee Communication Compared to Human Assessment
ConditionCommunication Skills
Trial statusRecruiting
Trial phaseNot applicable
Trial typeInterventional
Biological sexAll
Age18-99
SponsorStanford University
About this trial
This study will evaluate whether a web-based artificial intelligence platform (AI) (Clinical Mind AI \[CMAI\] Stanford, CA), can assess communication skills in anesthesiology trainees, including residents and fellows, in the setting of disclosing medical errors. All participants will participate in an AI-generated simulation by using the platform remotely, and CMAI will assess trainee performance immediately after the simulation.
Eligibility criteria
This trial accepts healthy volunteersQualifiers
Participant's must be 18 years of age or older
Graduate medical trainess such as residents and fellows at all levels of training excluding PGY1 trainees at Department of Anesthesiology, Perioperative, and Pain Medicine at Stanford University
Able to speak and understand English
Willing and able to provide consent to participate in research
Disqualifiers
Non-English speaker
unable to provide consent to participate in research
individual with no access to the necessary technology (internet, computer/smart device) required for participating in the digital, voice-based interaction with the AI platform
Trial design
Design model
Single group
Treatments tested in this trial
Communication Assessment through AI and Human
Other interventionParticipants will remotely engage in an AI-generated simulation that presents an voice-based conversation with a simulated parent of a pediatric patient. The scenario involves discussing a clinical scenario concerning their child. Following the simulation, the participant's communication skills will be assessed using an artificial intelligence (AI) system trained to evaluate the key aspects of communication using standardized rating scales. In addition, two human evaluators will assess the participant's communication skills using the same rating scales.
Treatment groups
45 Participants
are divided into 1 treatment groupGroup A: Voice Artificial Intelligence (AI) GroupExperimental treatment 1 intervention
Trial outcomes
Primary outcomes
1
Reliability of AI Conversational Performance Assessment
The primary outcome is to evaluate the reliability of the Clinical Mind AI (CMAI) platform in accurately assessing the communication skills of medical education trainees during a simulated interaction. To measure this outcome, the CMAI will use the Breaking Bad News Assessment Schedule (BBAS), a validated tool assessing participant's communication skills in five domains: 1) Setting the scene, 2) Breaking the news, 3) Eliciting concerns, 4) Informative giving, and 5) Empathy and Support. It includes 17 items, each with sub questions, rated on a 5-point Likert scale from (1-5), where score meanings may vary by each question. The scoring will be guided by Breaking Bad News Rubric, which outlines performance criteria for each item on the BBAS. In addition, two trained human evaluators will independently asses the participant's communication skills using the same BBAS tool and rubric. Lastly, both CMAI and human scores will be compared to determine the reliability of the CMAI platform.
Time frame
Immediately after the simulationSecondary outcomes
1
Usability of CMAI Simulation Platform
The perceived usability of the CMAI platform will be evaluated using a usability questionnaire. This questionnaire consists of 14 items, each offering five different response options. Participants will rate each item on a scale from 1 - 5, where score meanings may vary by each question.
Time frame
immediatley after the simulation2
Satisfaction of CMAI Simulation
Participant's satisfaction levels will be evaluated using a modified version of the Questionnaire on Satisfaction with Teaching Innovation (QSTI) Survey. The survey consists of five items, each rated on a scale of 1-5, where 1= Strongly Disagree and 5 = Strongly Agree. Higher scores indicate increased satisfaction levels.
Time frame
immediately after the simulationSponsors and contacts
Click on the lead sponsor to view all of their trials.
Source ClinicalTrials.gov