If OSCE is not a valid test ofClinical Skills-What Next?

NBE Bulletin Volume 1 Number 3 Oct-December, 2005 Page 4

Jacob Puliyel

If OSCE is not a valid test ofClinical Skills-What Next?Jacob M. Puliyel MD MRCP M Phil, Head ofDepartment of Paediatrics, St. StephensHospital, Tis Hazari, DelhiI worked for the University of Manchester in the UKin the nineties when OSCE was a novel methodintroduced to evaluate undergraduates in Pediatrics.I spent my time teaching students clinical pediatrics-distinguishing between different cardiacmurmurs and listening torespiratory sounds. Mystudent did poorly.Then I had opportunityto conduct an OSCE. After that, I understood thesystem. I stopped teaching pediatrics andtrained my students to pass the examination:‘When asked to examine the cardiovascular systemput your stethoscope on the apex, the tricuspid area,the pulmonary area and the aortic area. Put thestethoscope on the neck and turn the patient over toput the stethoscope in the inter-scapular area also-and so on. No time was wasted on the complexsubject of murmurs or respiratory sounds. Thecandidates did brilliantly- but were they betterdoctors? I am certain they were not!We have started the OSCE for the DNB. Soon we willhave coaching classes for the OSCE-they will teachcandidates like I taught at Manchester:‘You may be asked to examine the respiratory systemin a manikin.Remember always introduce your self to the manikin!Ask permission from the manikin to open its shirt!This will fetch you two marks out of the 10 marksallotted for examination of the respiratory system. Ifthe pass percentile is 80% this may make thedifference between pass and fail for this section.Students who are good clinicians get no credit fortheir skills. Do we want this situation with a post-graduate examination? It is suggested that using acombination of the old system with the new, cansolve this problem. I do not think it helps. If the oldsystem is biased, giving 50% marks to that systemcan make or break a candidate. On the same grounds,if the new system is not a valid test of clinical skills,apportioning 50% marks to that, is wrong.NBE TNBE TNBE TNBE TNBE Teachers’ Peachers’ Peachers’ Peachers’ Peachers’ PerspectiveserspectiveserspectiveserspectiveserspectivesCan we improve the system?If the money and energy spent on each OSCE, isspent on improving the old system we could have afairer test of clinical competence. Examiners can betaught to score candidates independently withoutconsulting one another. 2 examiners will evaluatethe candidate for short cases and another 2 for thelong case. The examiners are asked to assess if in hisopinion, the candidate deserves to pass theexamination- not whether he made a correctdiagnosis. Bedside manners, clinical skills anddeductive thinking all need to be evaluated.These scores can be fed to a computer. The computerprograms can find the 3 examiners who have thebest-matched evaluation score for the candidate.The score for the fourth examiner can be disregarded.If 3 of the examiners say a candidate deserves topass he is passed. If the jury is divided evenly thecomputer program will add up the candidates best3 scores to see if he has passed.The same computer program can evaluateexaminers, to see if one examiner consistentlygives lover or higher scores, compared to peersor if he gives scores at random. Examiners whoconsistently give lower or higher marks can havetheir scoring automatically corrected by the computerprogram. Examiners, who score randomly, may bebiased or inattentive and can be taken off theexamination panel. Examiners will be on their toesbecause of this. This will eliminate many of the vicesof the old system. Examiners will no longer have adivine right to act arbitrarily but they will be judgedby the computer program.Such a program is novel and not been tested outbut it has the potential to be fair and open. Insteadof importing methods of evaluation, this is ouropportunity to develop something new that othercan copy!