Robot learns how to be a surgeon

This handout photo released by the University of Maryland School of Medicine on January 10, 2022 shows surgeons performing a transplant of a heart from a genetically modified pig to patient David Bennett, Sr – Copyright AFP/File JIM WATSON

A robot, trained by watching videos of experienced surgeons at work, executed the same surgical procedures as skilfully as the human doctors.

The process of ‘imitation learning’ to train surgical robots removes the necessity to program robots with each move required during a medical procedure and brings the field of robotic surgery closer to true autonomy, where robots can perform complex surgeries without any human help.

The research has been led by Johns Hopkins University and it was presented to the November 2024 Conference on Robot Learning in Munich.

The device tested was the da Vinci Surgical System robot and the trials involved performing some fundamental surgical procedures: manipulating a needle; lifting body tissue, and suturing.

The model combined imitation learning with the same machine learning architecture that underpins ChatGPT. Imitation learning is a type of reinforcement learning, where an agent learns to perform a task by supervised learning from expert demonstrations.

Where ChatGPT works with words and text, this model speaks “robot” with kinematics, a language that breaks down the angles of robotic motion into mathematics.

For the training process, the researchers fed their model hundreds of videos recorded from wrist cameras placed on the arms of da Vinci robots during surgical procedures.

These videos, recorded by surgeons all over the world, are used for post-operative analysis and then archived.

Nearly 7,000 da Vinci robots are used worldwide, and more than 50,000 surgeons are trained on the system, creating a large archive of data for robots to “imitate.”

While the da Vinci system is widely used, it is relatively imprecise. However, the scientists found a way to make the flawed input work. This was achieved by training the model to perform relative movements rather than absolute actions, which are inaccurate.

The key was an update to the image input which permits the AI system to find the right actions. The model has the potential to quickly train a robot to perform any type of surgical procedure.