Computer Assisted Radiology and Surgery

A “pickup” stereoscopic camera with visual-motor aligned control for the da Vinci surgical system: a preliminary study Abstract Purpose The current state-of-the-art surgical robotic systems use only a single endoscope to view the surgical field. Research has been conducted to introduce additional cameras to the surgical system, giving rise to new camera angles that cannot be achieved using the endoscope alone. While this additional visualization certainly aids in surgical performance, current systems lack visual-motor compatibility with respect to the additional camera views. We propose a new system that overcomes this limitation. Methods In this paper, we introduce a novel design of an additional “pickup” camera that can be integrated into the da Vinci Surgical System. We also introduce a solution to work comfortably in the various arbitrary views this camera provides by eliminating visual-motor misalignment. This is done by changing the working frame of the surgical instruments to work with respect to the coordinate system at the “pickup” camera instead of the endoscope. Results Human user trials ( \(N=14\) ) were conducted to evaluate the effect of visual-motor alignment with respect to the “pickup” camera on surgical performance. An inanimate surgical peg transfer task from the validated Fundamentals of Laparoscopic Surgery (FLS) Training Curriculum was used, and an improvement of 73% in task completion time and 80% in accuracy was observed with the visual-motor alignment over the case without it. Conclusion Our study shows that there is a requirement to achieve visual-motor alignment when utilizing views from external cameras in current clinical surgical robotics setups. We introduce a complete system that provides additional camera views with visual-motor aligned control. Such a system would be useful in existing surgical procedures and could also impact surgical planning and navigation.

EasyLabels: weak labels for scene segmentation in laparoscopic videos Abstract Purpose We present a different approach for annotating laparoscopic images for segmentation in a weak fashion and experimentally prove that its accuracy when trained with partial cross-entropy is close to that obtained with fully supervised approaches. Methods We propose an approach that relies on weak annotations provided as stripes over the different objects in the image and partial cross-entropy as the loss function of a fully convolutional neural network to obtain a dense pixel-level prediction map. Results We validate our method on three different datasets, providing qualitative results for all of them and quantitative results for two of them. The experiments show that our approach is able to obtain at least \(90\%\) of the accuracy obtained with fully supervised methods for all the tested datasets, while requiring \(\sim 13\) \(\times \) less time to create the annotations compared to full supervision. Conclusions With this work, we demonstrate that laparoscopic data can be segmented using very few annotated data while maintaining levels of accuracy comparable to those obtained with full supervision.

Dynamic, patient-specific mitral valve modelling for planning transcatheter repairs Abstract Purpose: Transcatheter, beating heart repair techniques for mitral valve regurgitation is a very active area of development. However, it is difficult to both simulate and predict the clinical outcomes of mitral repairs, owing to the complexity of mitral valve geometry and the influence of hemodynamics. We aim to produce a workflow for manufacturing dynamic patient-specific models to simulate the mitral valve for transcatheter repair applications. Methods: In this paper, we present technology and associated workflow, for using transesophageal echocardiography to generate dynamic physical replicas of patient valves. We validate our workflow using six patient datasets representing patients with unique or particularly challenging pathologies as selected by a cardiologist. The dynamic component of the models and their resultant potential as procedure planning tools is due to a dynamic pulse duplicator that permits the evaluation of the valve models experiencing realistic hemodynamics. Results: Early results indicate the workflow has excellent anatomical accuracy and the ability to replicate regurgitation pathologies, as shown by colour Doppler ultrasound and anatomical measurements comparing patients and models. Analysis of all measurements successfully resulted in t critical two-tail > t stat and p values > 0.05, thus demonstrating no statistical difference between the patients and models, owing to high fidelity morphological replication. Conclusions: Due to the combination of a dynamic environment and patient-specific modelling, this workflow demonstrates a promising technology for simulating the complete morphology of mitral valves undergoing transcatheter repairs.

Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks Abstract Purpose In surgical navigation, pre-operative organ models are presented to surgeons during the intervention to help them in efficiently finding their target. In the case of soft tissue, these models need to be deformed and adapted to the current situation by using intra-operative sensor data. A promising method to realize this are real-time capable biomechanical models. Methods We train a fully convolutional neural network to estimate a displacement field of all points inside an organ when given only the displacement of a part of the organ’s surface. The network trains on entirely synthetic data of random organ-like meshes, which allows us to use much more data than is otherwise available. The input and output data are discretized into a regular grid, allowing us to fully utilize the capabilities of convolutional operators and to train and infer in a highly parallelized manner. Results The system is evaluated on in-silico liver models, phantom liver data and human in-vivo breathing data. We test the performance with varying material parameters, organ shapes and amount of visible surface. Even though the network is only trained on synthetic data, it adapts well to the various cases and gives a good estimation of the internal organ displacement. The inference runs at over 50 frames per second. Conclusion We present a novel method for training a data-driven, real-time capable deformation model. The accuracy is comparable to other registration methods, it adapts very well to previously unseen organs and does not need to be re-trained for every patient. The high inferring speed makes this method useful for many applications such as surgical navigation and real-time simulation.

Extending pretrained segmentation networks with additional anatomical structures Abstract Purpose For comprehensive surgical planning with sophisticated patient-specific models, all relevant anatomical structures need to be segmented. This could be achieved using deep neural networks given sufficiently many annotated samples; however, datasets of multiple annotated structures are often unavailable in practice and costly to procure. Therefore, being able to build segmentation models with datasets from different studies and centers in an incremental fashion is highly desirable. Methods We propose a class-incremental framework for extending a deep segmentation network to new anatomical structures using a minimal incremental annotation set. Through distilling knowledge from the current network state, we overcome the need for a full retraining. Results We evaluate our methods on 100 MR volumes from SKI10 challenge with varying incremental annotation ratios. For 50% incremental annotations, our proposed method suffers less than 1% Dice score loss in retaining old-class performance, as opposed to 25% loss of conventional finetuning. Our framework inherently exploits transferable knowledge from previously trained structures to incremental tasks, demonstrated by results superior even to non-incremental training: In a single volume one-shot incremental learning setting, our method outperforms vanilla network performance by>11% in Dice. Conclusions With the presented method, new anatomical structures can be learned while retaining performance for older structures, without a major increase in complexity and memory footprint, hence suitable for lifelong class-incremental learning. By leveraging information from older examples, a fraction of annotations can be sufficient for incrementally building comprehensive segmentation models. With our meta-method, a deep segmentation network is extended with only a minor addition per structure, thus can be applicable also for future network architectures.

Design optimization of a contact-aided continuum robot for endobronchial interventions based on anatomical constraints Abstract Purpose A laser-profiled continuum robot (CR) with a series of interlocking joints has been developed in our center to reach deeper areas of the airways. However, it deflects with constant curvature, which thus increases the difficulty of entering specific bronchi without relying on the tissue reaction forces. This paper aims to propose an optimization framework to find the best design parameters for nonconstant curvature CRs to reach distal targets while attempting to avoid the collision with the surrounding tissue. Methods First, the contact-aided compliant mechanisms (CCMs) are integrated with the continuum robot to achieve the nonconstant curvature. Second, forward kinematics considering CCMs is built. Third, inverse kinematics is implemented to steer the robot tip toward the desired targets within the confined anatomy. Finally, an optimization framework is proposed to find the best robot design to reach the target with the least collision to the bronchi walls. Results Experiments are carried out to verify the feasibility of CCMs to enable the nonconstant curvature deflection, and simulations demonstrate a lower cost function value to reach a target for the nonconstant curvature optimized design with respect to the standard constant curvature robot (0.11 vs. 2.66). In addition, the higher capacity of the optimized design to complete the task is validated by interventional experiments using fluoroscopy. Conclusion Results demonstrate the effectiveness of the proposed framework to find an optimized CR with nonconstant curvature to perform safer interventions to reach distal targets.

An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions Abstract Purpose The registration of preoperative 3D images to intra-operative laparoscopic 2D images is one of the main concerns for augmented reality in computer-assisted surgery. For laparoscopic liver surgery, while several algorithms have been proposed, there is neither a public dataset nor a systematic evaluation methodology to quantitatively evaluate registration accuracy. Method Our main contribution is to provide such a dataset with an in vivo porcine model. It is used to evaluate a state-of-the-art registration algorithm that is capable of simultaneous registration and soft-body collision reasoning. Results The dataset consists of 13 deformed liver states, with corresponding exploration videos and interventional CT acquisitions with 60 small artificial fiducials located on the surface of the liver and distributed within the parenchyma, where a precise registration is crucial for augmented reality. This dataset will be made public. Using this dataset, we show that collision reasoning improves performance of registration for strong deformation and independent lobe motion. Conclusion This dataset addresses the lack of public datasets in this field. As an example of use, we present and evaluate a state-of-the-art energy-based approach and a novel extension that handles self-collisions.

Preliminary results of DSA denoising based on a weighted low-rank approach using an advanced neurovascular replication system Abstract Purpose 2D digital subtraction angiography (DSA) has become an important technique for interventional neuroradiology tasks, such as detection and subsequent treatment of aneurysms. In order to provide high-quality DSA images, usually undiluted contrast agent and a high X-ray dose are used. The iodinated contrast agent puts a burden on the patients’ kidneys while the use of high-dose X-rays expose both patients and medical staff to a considerable amount of radiation. Unfortunately, reducing either the X-ray dose or the contrast agent concentration usually results in a sacrifice of image quality. Materials and methods To denoise a frame, the proposed spatiotemporal denoising method utilizes the low-rank nature of a spatially aligned temporal sequence where variation is introduced by the flow of contrast agent through a vessel tree of interest. That is, a constrained weighted rank-1 approximation of the stack comprising the frame to be denoised and its temporal neighbors is computed where the weights are used to prevent the contribution of non-similar pixels toward the low-rank approximation. The method has been evaluated using a vascular flow phantom emulating cranial arteries into which contrast agent can be manually injected (Vascular Simulations Replicator, Vascular Simulations, Stony Brook NY, USA). For the evaluation, image sequences acquired at different dose levels as well as different contrast agent concentrations have been used. Results Qualitative and quantitative analyses have shown that with the proposed approach, the dose and the concentration of the contrast agent could both be reduced by about 75%, while maintaining the required image quality. Most importantly, it has been observed that the DSA images obtained using the proposed method have the closest resemblance to typical DSA images, i.e., they preserve the typical image characteristics best. Conclusion Using the proposed denoising approach, it is possible to improve the image quality of low-dose DSA images. This improvement could enable both a reduction in contrast agent and radiation dose when acquiring DSA images, thereby benefiting patients as well as clinicians. Since the resulting images are free from artifacts and as the inherent characteristics of the images are also preserved, the proposed method seems to be well suited for clinical images as well.

High-precision evaluation of electromagnetic tracking Abstract Purpose Navigation in high-precision minimally invasive surgery (HP-MIS) demands high tracking accuracy in the absence of line of sight (LOS). Currently, no tracking technology can satisfy this requirement. Electromagnetic tracking (EMT) is the best tracking paradigm in the absence of LOS despite limited accuracy and robustness. Novel evaluation protocols are needed to ensure high-precision and robust EMT for navigation in HP-MIS. Methods We introduce a novel protocol for EMT measurement evaluation featuring a high-accuracy phantom based on LEGO \(^{\circledR }\) , which is calibrated by a coordinate measuring machine to ensure accuracy. Our protocol includes relative sequential positions and an uncertainty estimation of positioning. We show effects on distortion compensation using a learned interpolation model. Results Our high-precision protocol clarifies properties of errors and uncertainties of EMT for high-precision use cases. For EMT errors reaching clinically relevant 0.2 mm, our design is 5–10 times more accurate than previous protocols with 95% confidence margins of 0.02 mm. This high-precision protocol ensures the performance improvement in compensated EMT by 0.05 mm. Conclusion Our protocol improves the reliability of EMT evaluations because of significantly lower protocol-inherent uncertainties. To reduce patient risk in HP-MIS and to evaluate magnetic field distortion compensation, more high-accuracy protocols such as the one proposed here are required.

Pedicle screw navigation using surface digitization on the Microsoft HoloLens Abstract Purpose In spinal fusion surgery, imprecise placement of pedicle screws can result in poor surgical outcome or may seriously harm a patient. Patient-specific instruments and optical systems have been proposed for improving precision through surgical navigation compared to freehand insertion. However, existing solutions are expensive and cannot provide in situ visualizations. Recent technological advancement enabled the production of more powerful and precise optical see-through head-mounted displays for the mass market. The purpose of this laboratory study was to evaluate whether such a device is sufficiently precise for the navigation of lumbar pedicle screw placement. Methods A novel navigation method, tailored to run on the Microsoft HoloLens, was developed. It comprises capturing of the intraoperatively reachable surface of vertebrae to achieve registration and tool tracking with real-time visualizations without the need of intraoperative imaging. For both surface sampling and navigation, 3D printable parts, equipped with fiducial markers, were employed. Accuracy was evaluated within a self-built setup based on two phantoms of the lumbar spine. Computed tomography (CT) scans of the phantoms were acquired to carry out preoperative planning of screw trajectories in 3D. A surgeon placed the guiding wire for the pedicle screw bilaterally on ten vertebrae guided by the navigation method. Postoperative CT scans were acquired to compare trajectory orientation (3D angle) and screw insertion points (3D distance) with respect to the planning. Results The mean errors between planned and executed screw insertion were \(3.38^{\circ }\pm {1.73}^{\circ }\) for the screw trajectory orientation and 2.77±1.46 mm for the insertion points. The mean time required for surface digitization was 125±27 s. Conclusions First promising results under laboratory conditions indicate that precise lumbar pedicle screw insertion can be achieved by combining HoloLens with our proposed navigation method. As a next step, cadaver experiments need to be performed to confirm the precision on real patient anatomy.