26 results
Filters
Settings
Now showing1 - 10 of 26
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Surgical instrument-tissue interaction recognition with multi-task-attention video transformerPurpose The recognition of surgical instrument-tissue interactions can enhance the surgical workflow analysis, improve automated safety systems and enable skill assessment in minimally invasive surgery. However, current deep learning methods for surgical instrument-tissue interaction recognition often rely on static images or coarse temporal sampling, limiting their ability to capture rapid surgical dynamics. Therefore, this study systematically investigates the impact of incorporating fine-grained temporal context into deep learning models for interaction recognition. Methods We conduct extensive experiments with multiple curated video-based datasets to investigate the influence of fine-grained temporal context for the task of instrument-tissue interaction recognition using video transformer with spatio-temporal feature extraction capabilities. Additionally, we propose a multi-task-attention module that utilizes cross-attention and a gating mechanism to improve communication between the subtasks of identifying the surgical instrument, atomic action, and anatomical target. Results Our study demonstrates the benefit of utilizing the fine-grained temporal context for recognition of instrument-tissue interactions, with an optimal sampling rate of 6-8 Hz identified for the examined datasets. Furthermore, our proposed MTAM significantly outperforms state-of-the-art multi-task video transformer on the CholecT45-Vid and GraSP-Vid datasets, achieving relative increases of 4.8% and 5.9% in surgical instrument-tissue interaction recognition, respectively. Conclusions In this work, we demonstrate the benefits of using a fine-grained temporal context rather than static images or coarse temporal context for the task of surgical instrument-tissue interaction recognition. We also show that leveraging cross-attention with spatio-temporal features from various subtasks leads to improved surgical instrument-tissue interaction recognition performance. The project is available at: https://lennart-maack.github.io/InstrTissRec-MTAMPublicationtype: Journal Article23 10 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , A mobile robotic approach to autonomous surface scanning in legal medicinePurpose : Comprehensive legal medicine documentation includes internal and external examination of the corpse. Typically, this documentation is conducted manually during conventional autopsy. Systematic digital documentation would be desirable, especially for external wound examination, which is becoming more relevant for legal medicine analysis. For this purpose, RGB surface scanning has been introduced. While manual full-surface scanning using a handheld camera is time-consuming and operator-dependent, floor or ceiling-mounted robotic systems require specialized rooms. Hence, we consider whether a mobile robotic system can be used for external documentation. Methods : We develop a mobile robotic system that enables full-body RGB-D surface scanning. Our work includes a detailed configuration space analysis to identify the environmental parameters that must be considered for a successful surface scan. We validate our findings through an experimental study in the lab and demonstrate the systems application in legal medicine. Results : Our configuration space analysis shows that a good trade-off between coverage and time is reached with three robot base positions, leading to a coverage of 94.96 %. Experiments validate the effectiveness of the system in accurately capturing body surface geometry with an average surface coverage of 96.90±3.16 % and 92.45±1.43 % for a body phantom and actual corpses, respectively. Conclusion : This work demonstrates the potential of a mobile robotic system to automate RGB-D surface scanning in legal medicine, complementing post-mortem CT scans for inner documentation. Our results indicate that the proposed system can contribute to more efficient, autonomous legal medicine documentation, reducing the need for manual intervention.Publicationtype: Journal Article16 9 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , PolypNextLSTM: a lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM(Springer, 2024-08-08); ; ; ; Purpose: Commonly employed in polyp segmentation, single-image UNet architectures lack the temporal insight clinicians gain from video data in diagnosing polyps. To mirror clinical practices more faithfully, our proposed solution, PolypNextLSTM, leverages video-based deep learning, harnessing temporal information for superior segmentation performance with least parameter overhead, making it possibly suitable for edge devices. Methods: PolypNextLSTM employs a UNet-like structure with ConvNext-Tiny as its backbone, strategically omitting the last two layers to reduce parameter overhead. Our temporal fusion module, a Convolutional Long Short Term Memory (ConvLSTM), effectively exploits temporal features. Our primary novelty lies in PolypNextLSTM, which stands out as the leanest in parameters and the fastest model, surpassing the performance of five state-of-the-art image and video-based deep learning models. The evaluation of the SUN-SEG dataset spans easy-to-detect and hard-to-detect polyp scenarios, along with videos containing challenging artefacts like fast motion and occlusion. Results: Comparison against 5 image-based and 5 video-based models demonstrates PolypNextLSTM’s superiority, achieving a Dice score of 0.7898 on the hard-to-detect polyp test set, surpassing image-based PraNet (0.7519) and video-based PNS+ (0.7486). Notably, our model excels in videos featuring complex artefacts such as ghosting and occlusion. Conclusion: PolypNextLSTM, integrating pruned ConvNext-Tiny with ConvLSTM for temporal fusion, not only exhibits superior segmentation performance but also maintains the highest frames per speed among evaluated models. Code can be found here: https://github.com/mtec-tuhh/PolypNextLSTM.Publicationtype: Journal Article19 21 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Needle tracking in low-resolution ultrasound volumes using deep learning(Springer, 2024-07-13); ; ; ; Purpose: Clinical needle insertion into tissue, commonly assisted by 2D ultrasound imaging for real-time navigation, faces the challenge of precise needle and probe alignment to reduce out-of-plane movement. Recent studies investigate 3D ultrasound imaging together with deep learning to overcome this problem, focusing on acquiring high-resolution images to create optimal conditions for needle tip detection. However, high-resolution also requires a lot of time for image acquisition and processing, which limits the real-time capability. Therefore, we aim to maximize the US volume rate with the trade-off of low image resolution. We propose a deep learning approach to directly extract the 3D needle tip position from sparsely sampled US volumes. Methods: We design an experimental setup with a robot inserting a needle into water and chicken liver tissue. In contrast to manual annotation, we assess the needle tip position from the known robot pose. During insertion, we acquire a large data set of low-resolution volumes using a 16 × 16 element matrix transducer with a volume rate of 4 Hz. We compare the performance of our deep learning approach with conventional needle segmentation. Results: Our experiments in water and liver show that deep learning outperforms the conventional approach while achieving sub-millimeter accuracy. We achieve mean position errors of 0.54 mm in water and 1.54 mm in liver for deep learning. Conclusion: Our study underlines the strengths of deep learning to predict the 3D needle positions from low-resolution ultrasound volumes. This is an important milestone for real-time needle navigation, simplifying the alignment of needle and ultrasound probe and enabling a 3D motion analysis.Publicationtype: Journal Article18 81 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , BrainLossNet : a fast, accurate and robust method to estimate brain volume loss from longitudinal MRI(Springer, 2024-06-16); ; ; ; Purpose: MRI-derived brain volume loss (BVL) is widely used as neurodegeneration marker. SIENA is state-of-the-art for BVL measurement, but limited by long computation time. Here we propose “BrainLossNet”, a convolutional neural network (CNN)-based method for BVL-estimation. Methods: BrainLossNet uses CNN-based non-linear registration of baseline(BL)/follow-up(FU) 3D-T1w-MRI pairs. BVL is computed by non-linear registration of brain parenchyma masks segmented in the BL/FU scans. The BVL estimate is corrected for image distortions using the apparent volume change of the total intracranial volume. BrainLossNet was trained on 1525 BL/FU pairs from 83 scanners. Agreement between BrainLossNet and SIENA was assessed in 225 BL/FU pairs from 94 MS patients acquired with a single scanner and 268 BL/FU pairs from 52 scanners acquired for various indications. Robustness to short-term variability of 3D-T1w-MRI was compared in 354 BL/FU pairs from a single healthy men acquired in the same session without repositioning with 116 scanners (Frequently-Traveling-Human-Phantom dataset, FTHP). Results: Processing time of BrainLossNet was 2–3 min. The median [interquartile range] of the SIENA-BrainLossNet BVL difference was 0.10% [− 0.18%, 0.35%] in the MS dataset, 0.08% [− 0.14%, 0.28%] in the various indications dataset. The distribution of apparent BVL in the FTHP dataset was narrower with BrainLossNet (p = 0.036; 95th percentile: 0.20% vs 0.32%). Conclusion: BrainLossNet on average provides the same BVL estimates as SIENA, but it is significantly more robust, probably due to its built-in distortion correction. Processing time of 2–3 min makes BrainLossNet suitable for clinical routine. This can pave the way for widespread clinical use of BVL estimation from intra-scanner BL/FU pairs.Publicationtype: Journal Article8 11 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Self-supervised learning for classifying paranasal anomalies in the maxillary sinus(Springer, 2024-06-08); ; ; ; Purpose: Paranasal anomalies, frequently identified in routine radiological screenings, exhibit diverse morphological characteristics. Due to the diversity of anomalies, supervised learning methods require large labelled dataset exhibiting diverse anomaly morphology. Self-supervised learning (SSL) can be used to learn representations from unlabelled data. However, there are no SSL methods designed for the downstream task of classifying paranasal anomalies in the maxillary sinus (MS). Methods: Our approach uses a 3D convolutional autoencoder (CAE) trained in an unsupervised anomaly detection (UAD) framework. Initially, we train the 3D CAE to reduce reconstruction errors when reconstructing normal maxillary sinus (MS) image. Then, this CAE is applied to an unlabelled dataset to generate coarse anomaly locations by creating residual MS images. Following this, a 3D convolutional neural network (CNN) reconstructs these residual images, which forms our SSL task. Lastly, we fine-tune the encoder part of the 3D CNN on a labelled dataset of normal and anomalous MS images. Results: The proposed SSL technique exhibits superior performance compared to existing generic self-supervised methods, especially in scenarios with limited annotated data. When trained on just 10% of the annotated dataset, our method achieves an area under the precision-recall curve (AUPRC) of 0.79 for the downstream classification task. This performance surpasses other methods, with BYOL attaining an AUPRC of 0.75, SimSiam at 0.74, SimCLR at 0.73 and masked autoencoding using SparK at 0.75. Conclusion: A self-supervised learning approach that inherently focuses on localizing paranasal anomalies proves to be advantageous, particularly when the subsequent task involves differentiating normal from anomalous maxillary sinuses. Access our code at https://github.com/mtec-tuhh/self-supervised-paranasal-anomaly.Publicationtype: Journal Article14 59 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Multiple instance ensembling for paranasal anomaly classification in the maxillary sinus(Springer, 2023-07-21); ; ; ; Purpose : Paranasal anomalies are commonly discovered during routine radiological screenings and can present with a wide range of morphological features. This diversity can make it difficult for convolutional neural networks (CNNs) to accurately classify these anomalies, especially when working with limited datasets. Additionally, current approaches to paranasal anomaly classification are constrained to identifying a single anomaly at a time. These challenges necessitate the need for further research and development in this area. Methods : We investigate the feasibility of using a 3D convolutional neural network (CNN) to classify healthy maxillary sinuses (MS) and MS with polyps or cysts. The task of accurately localizing the relevant MS volume within larger head and neck Magnetic Resonance Imaging (MRI) scans can be difficult, but we develop a strategy which includes the use of a novel sampling technique that not only effectively localizes the relevant MS volume, but also increases the size of the training dataset and improves classification results. Additionally, we employ a Multiple Instance Ensembling (MIE) prediction method to further boost classification performance. Results : With sampling and MIE, we observe that there is consistent improvement in classification performance of all 3D ResNet and 3D DenseNet architecture with an average AUPRC percentage increase of 21.86 ± 11.92% and 4.27 ± 5.04% by sampling and 28.86 ± 12.80% and 9.85 ± 4.02% by sampling and MIE, respectively. Conclusion : Sampling and MIE can be effective techniques to improve the generalizability of CNNs for paranasal anomaly classification. We demonstrate the feasibility of classifying anomalies in the MS. We propose a data enlarging strategy through sampling alongside a novel MIE strategy that proves to be beneficial for paranasal anomaly classification in the MS.Publicationtype: Journal ArticleScopus© Citations 2 27 104 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , AI-based optimization for US-guided radiation therapy of the prostateObjectives: Fast volumetric ultrasound presents an interesting modality for continuous and real-time intra-fractional target tracking in radiation therapy of lesions in the abdomen. However, the placement of the ultrasound probe close to the target structures leads to blocking some beam directions. Methods: To handle the combinatorial complexity of searching for the ultrasound-robot pose and the subset of optimal treatment beams, we combine CNN-based candidate beam selection with simulated annealing for setup optimization of the ultrasound robot, and linear optimization for treatment plan optimization into an AI-based approach. For 50 prostate cases previously treated with the CyberKnife, we study setup and treatment plan optimization when including robotic ultrasound guidance. Results: The CNN-based search substantially outperforms previous randomized heuristics, increasing coverage from 93.66 to 97.20% on average. Moreover, in some cases the total MU was also reduced, particularly for smaller target volumes. Results after AI-based optimization are similar for treatment plans with and without beam blocking due to ultrasound guidance. Conclusions: AI-based optimization allows for fast and effective search for configurations for robotic ultrasound-guided radiation therapy. The negative impact of the ultrasound robot on the plan quality can successfully be mitigated resulting only in minor differences.Publicationtype: Journal ArticleScopus© Citations 1 59 82 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Systematic analysis of volumetric ultrasound parameters for markerless 4D motion tracking(Springer, 2022-05-21); ; ; ; Objectives: Motion compensation is an interesting approach to improve treatments of moving structures. For example, target motion can substantially affect dose delivery in radiation therapy, where methods to detect and mitigate the motion are widely used. Recent advances in fast, volumetric ultrasound have rekindled the interest in ultrasound for motion tracking. We present a setup to evaluate ultrasound based motion tracking and we study the effect of imaging rate and motion artifacts on its performance. Methods: We describe an experimental setup to acquire markerless 4D ultrasound data with precise ground truth from a robot and evaluate different real-world trajectories and system settings toward accurate motion estimation. We analyze motion artifacts in continuously acquired data by comparing to data recorded in a step-and-shoot fashion. Furthermore, we investigate the trade-off between the imaging frequency and resolution. Results: The mean tracking errors show that continuously acquired data leads to similar results as data acquired in a step-and-shoot fashion. We report mean tracking errors up to 2.01 mm and 1.36 mm on the continuous data for the lower and higher resolution, respectively, while step-and-shoot data leads to mean tracking errors of 2.52 mm and 0.98 mm. Conclusions: We perform a quantitative analysis of different system settings for motion tracking with 4D ultrasound. We can show that precise tracking is feasible and additional motion in continuously acquired data does not impair the tracking. Moreover, the analysis of the frequency resolution trade-off shows that a high imaging resolution is beneficial in ultrasound tracking.Publicationtype: Journal ArticleScopus© Citations 1 33 89 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Three-dimensional deep learning with spatial erasing for unsupervised anomaly segmentation in brain MRI(Springer, 2021-07-12); ; ; ; Purpose: Brain Magnetic Resonance Images (MRIs) are essential for the diagnosis of neurological diseases. Recently, deep learning methods for unsupervised anomaly detection (UAD) have been proposed for the analysis of brain MRI. These methods rely on healthy brain MRIs and eliminate the requirement of pixel-wise annotated data compared to supervised deep learning. While a wide range of methods for UAD have been proposed, these methods are mostly 2D and only learn from MRI slices, disregarding that brain lesions are inherently 3D and the spatial context of MRI volumes remains unexploited. Methods: We investigate whether using increased spatial context by using MRI volumes combined with spatial erasing leads to improved unsupervised anomaly segmentation performance compared to learning from slices. We evaluate and compare 2D variational autoencoder (VAE) to their 3D counterpart, propose 3D input erasing, and systemically study the impact of the data set size on the performance. Results: Using two publicly available segmentation data sets for evaluation, 3D VAEs outperform their 2D counterpart, highlighting the advantage of volumetric context. Also, our 3D erasing methods allow for further performance improvements. Our best performing 3D VAE with input erasing leads to an average DICE score of 31.40% compared to 25.76% for the 2D VAE. Conclusions: We propose 3D deep learning methods for UAD in brain MRI combined with 3D erasing and demonstrate that 3D methods clearly outperform their 2D counterpart for anomaly segmentation. Also, our spatial erasing method allows for further performance improvements and reduces the requirement for large data sets.Publicationtype: Journal ArticleScopus© Citations 19 133 215