Although a small study, the one I am going to describe here “Surgical Skill and Complication Rates after Bariatric Surgery” (NEJM 369(15) Oct. 10, 2013) is very clever and, if the results hold up, may provide a new way to evaluate and continually improve the skills of some surgeons.
John Birkmeyer, MD and his colleagues in Michigan got 20 surgeons to submit videos of themselves performing a technically challenging operation, laparoscopic gastric bypass. Other bariatric surgeons, 33 of them altogether, then viewed the videos and rated technical skills from 1( the equivalent of a general surgery chief resident to 5 (master bariatric surgeon) with 3 being considered average.
The ratings were applied to five domains of surgical skill: gentleness, tissue exposure, instrument handling, time and motion, and flow of operation. The total score was a simple average of the individual domain scores.
Videos were edited to focus on critically important aspects of the operation: the creation of the gastric pouch, the gastrojejunostomy, and the jejunojejunostomy. Each surgeon that submitted a video had at least 10 peers rating his or her technical skills, however, the exact makeup of the peer group varied for each video. Neither the rating surgeons nor the researchers were aware of the identity of the videoed surgeon.
The technical skills varied significantly with scores ranging from 2.6 to 4.8. The mean rating for the five docs in the lower quartile was 2.9 compared to 4.4 for those in the top quartile. The authors performed sensitivity analyses in two ways – one by rating the video of a second operation from each surgeon and the other by having some non-Michigan surgeons rate the first video. They found a high correlation with the initial results in both tests.
Surgical skill was not related to years in practice
Interestingly, surgical skill was not related to years in practice, whether a relevant fellowship had been completed, or whether the surgeon practiced in a teaching or community hospital. It was, however, strongly correlated with surgical volume. The best performing quartile performed triple the number of laparoscopic gastric bypass operations compared to the lowest quartile docs – a difference that was highly statistically significant.
By mining the clinical registry of the Michigan Bariatric Surgery Collaborative, the researchers were able to look at the rates of a number of clinically important outcome measures for each of the study surgeons. They found that the lower quartile docs had a longer operation time and a higher overall complication rate than those in the top quartile (14.5% vs 5.2%) and a higher mortality rate (0.26% vs 0.05%). They also had higher surgical (e.g., surgical site infections) and medical complications (e.g., pulmonary complications). Rates of reoperation, readmissions, and return visits to the ED were also greater in the lower skills group.
It is hard to say how much of the difference could be attributed to patient factors, particularly co-morbidities. All of these patients by definition are quite obese and so technically challenging. Although similar in terms of age, sex, and BMI, patients operated on by the bottom quartile docs had a higher prevalence of cardiovascular disease, sleep apnea, and mobility disorders, but a lower prevalence of smoking and musculoskeletal disorders.
The accompanying editorial
An accompanying editorial by Danny Jacobs, MD, MPH, “Cut Well, Sew Well, Do Well?” (NEJM 369(15)p.1466) raises some issues with regard to risk adjustment and patient coexisting conditions, but also points out that “surgeons should enhance their operative effectiveness by any means necessary.” If the study results hold up, video evaluations of surgical skills could become not only a great evaluative tool but also a great source of continuing surgical education.