Generalization of spatio-temporal deep learning for vision-based force estimation

Robot-assisted minimally-invasive surgery is increasingly used in clinical practice. Force feedback offers potential to develop haptic feedback for surgery systems. Forces can be estimated in a vision-based way by capturing deformation observed in 2D-image sequences with deep learning models. Variations in tissue appearance and mechanical properties likely influence force estimation methods' generalization. In this work, we study the generalization capabilities of different spatial and spatio-temporal deep learning methods across different tissue samples. We acquire several data-sets using a clinical laparoscope and use both purely spatial and also spatio-temporal deep learning models. The results of this work show that generalization across different tissues is challenging. Nevertheless, we demonstrate that using spatio-temporal data instead of individual frames is valuable for force estimation. In particular, processing spatial and temporal data separately by a combination of a ResNet and GRU architecture shows promising results with a mean absolute error of 15.450 compared to 19.744 mN of a purely spatial CNN.

Subjects

deep learning

laparoscopic imaging

spatio-temporal data

vision-based force estimation

DDC Class

600: Technik

610: Medizin

More Funding Information

The laparoscopic system was provided by Olympus.

Publication version

publishedVersion

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

[23645504 - Current Directions in Biomedical Engineering] Generalization of spatio-temporal deep learning for vision-based force estimation.pdf

Size

2.5 MB

Format

Adobe PDF

Options

Generalization of spatio-temporal deep learning for vision-based force estimation