检索结果-内蒙古大学图书馆

Precise Repositioning of Robotic Ultrasound: Improving Registration-based Motion Compensation using Ultrasound Confidence Optimization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jiang, Zhongliang Danis, Nehil Bi, Yuan Zhou, Mingchuan Kroenke, Markus Wendler, Thomas Navab, Nassir Computer Aided Medical Procedures and Augmented Reality Technical University of Munich Garching Germany The College of Biosystems Engineering and Food Science Zhejiang University Hangzhou China The Department of Radiology and Nuclear Medicine German Heart Center Munich Technical University of Munich Germany The Laboratory for Computational Sensing and Robotics Johns Hopkins University BaltimoreMD United States

Robotic ultrasound (US) imaging has been seen as a promising solution to overcome the limitations of free-hand US examinations, i.e., inter-operator variability. However, the fact that robotic US systems cannot react to subject movements during scans limits their clinical acceptance. Regarding human sonographers, they often react to patient movements by repositioning the probe or even restarting the acquisition, in particular for the scans of anatomies with long structures like limb arteries. To realize this characteristic, we proposed a vision-based system to monitor the subject’s movement and automatically update the scan trajectory thus seamlessly obtaining a complete 3D image of the target anatomy. The motion monitoring module is developed using the segmented object masks from RGB images. Once the subject is moved, the robot will stop and recompute a suitable trajectory by registering the surface point clouds of the object obtained before and after the movement using the iterative closest point algorithm. Afterward, to ensure optimal contact conditions after repositioning US probe, a confidence-based fine-tuning process is used to avoid potential gaps between the probe and contact surface. Finally, the whole system is validated on a human-like arm phantom with an uneven surface, while the object segmentation network is also validated on volunteers. The results demonstrate that the presented system can react to object movements and reliably provide accurate 3D images. Copyright © 2022, The Authors. All rights reserved.

关键词： Blood vessels

Motion-Aware Robotic 3D Ultrasound

学校读者我要写书评

暂无评论

Motion-Aware Robotic 3D Ultrasound

IEEE International Conference on robotics and Automation (ICRA)

作者： Zhongliang Jiang Hanyu Wang Zhenyu Li Matthias Grimm Mingchuan Zhou Ulrich Eck Sandra V. Brecht Tim C. Lueth Thomas Wendler Nassir Navab Chair for Computer Aided Medical Procedures and Augmented Reality Technical University of Munich Germany College of Biosystems of Engineering and Food Science Zhejiang University China Institute of Micro Technology and Medical Device Technology Technical University of Munich Germany Laboratory for Computational Sensing and Robotics Johns Hopkins University Baltimore MD USA

Robotic three-dimensional (3D) ultrasound (US) imaging has been employed to overcome the drawbacks of traditional US examinations, such as high inter-operator variability and lack of repeatability. However, object movement remains a challenge as unexpected motion decreases the quality of the 3D compounding. Furthermore, attempted adjustment of objects, e.g., adjusting limbs to display the entire limb artery tree, is not allowed for conventional robotic US systems. To address this challenge, we propose a vision-based robotic US system that can monitor the object’s motion and automatically update the sweep trajectory to provide 3D compounded images of the target anatomy seamlessly. To achieve these functions, a depth camera is employed to extract the manually planned sweep trajectory after which the normal direction of the object is estimated using the extracted 3D trajectory. Subsequently, to monitor the movement and further compensate for this motion to accurately follow the trajectory, the position of firmly attached passive markers is tracked in real-time. Finally, a stepwise compounding was performed. The experiments on a gel phantom demonstrate that the system can resume a sweep when the object is not stationary during scanning.

关键词： Visualization Three-dimensional displays Ultrasonic imaging Tracking Robot vision systems Skin Trajectory

Motion-aware robotic 3D ultrasound

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jiang, Zhongliang Wang, Hanyu Li, Zhenyu Grimm, Matthias Zhou, Mingchuan Eck, Ulrich Brecht, Sandra V. Lueth, Tim C. Wendler, Thomas Navab, Nassir Chair for Computer Aided Medical Procedures and Augmented Reality Technical University of Munich Germany College of Biosystems of Engineering and Food Science Zhejiang University China Institute of Micro Technology and Medical Device Technology Technical University of Munich Germany Laboratory for Computational Sensing and Robotics Johns Hopkins University BaltimoreMD United States

— Robotic three-dimensional (3D) ultrasound (US) imaging has been employed to overcome the drawbacks of traditional US examinations, such as high inter-operator variability and lack of repeatability. However, object movement remains a challenge as unexpected motion decreases the quality of the 3D compounding. Furthermore, attempted adjustment of objects, e.g., adjusting limbs to display the entire limb artery tree, is not allowed for conventional robotic US systems. To address this challenge, we propose a vision-based robotic US system that can monitor the object’s motion and automatically update the sweep trajectory to provide 3D compounded images of the target anatomy seamlessly. To achieve these functions, a depth camera is employed to extract the manually planned sweep trajectory after which the normal direction of the object is estimated using the extracted 3D trajectory. Subsequently, to monitor the movement and further compensate for this motion to accurately follow the trajectory, the position of firmly attached passive markers is tracked in real-time. Finally, a stepwise compounding was performed. The experiments on a gel phantom demonstrate that the system can resume a sweep when the object is not stationary during scanning. Copyright © 2021, The Authors. All rights reserved.

关键词： Trajectories

Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing

学校读者我要写书评

暂无评论

Robotic Navigation Autonomy for Subretinal Injection via Int...

IEEE International Conference on robotics and Automation (ICRA)

作者： Shervin Dehghani Michael Sommersperger Peiyao Zhang Alejandro Martin-Gomez Benjamin Busam Peter Gehlbach Nassir Navab M. Ali Nasseri Iulian Iordachita Department of Computer Science Technische Universität München München Germany Laboratory for Computational Sensing and Robotics Johns Hopkins University Baltimore MD USA Wilmer Eye Institute Johns Hopkins Hospital Baltimore MD USA Computer Aided Medical Procedures & Augmented Reality Technical University of Munich Munich Germany Whiting School of Engineering Johns Hopkins University Baltimore MD USA Augenklinik und Poliklinik Klinikum rechts der Isar der Technische Universität München München Germany

In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT intro-duces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems.

关键词：

STTAR: Surgical Tool Tracking using off-the-shelf Augmented Reality Head-Mounted Displays

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Martin-Gomez, Alejandro Li, Haowei Song, Tianyu Yang, Sheng Wang, Guangzhi Ding, Hui Navab, Nassir Zhao, Zhe Armand, Mehran The Laboratory for Computational Sensing and Robotics Whiting School of Engineering Johns Hopkins University United States The Department of Orthopaedic Surgery Johns Hopkins University School of Medicine United States The Department of Biomedical Engineering Tsinghua University China The Department of Orthopaedics Beijing Tsinghua Changgung Hospital School of Clinical Medicine Tsinghua University China Computer Aided Medical Procedures and Augmented Reality Department of Informatics Technical University of Munich Germany

The use of Augmented Reality (AR) for navigation purposes has shown beneficial in assisting physicians during the performance of surgical procedures. These applications commonly require knowing the pose of surgical tools and patients to provide visual information that surgeons can use during the performance of the task. Existing medical-grade tracking systems use infrared cameras placed inside the Operating Room (OR) to identify retro-reflective markers attached to objects of interest and compute their pose. Some commercially available AR Head-Mounted Displays (HMDs) use similar cameras for self-localization, hand tracking, and estimating the objects’ depth. This work presents a framework that uses the built-in cameras of AR HMDs to enable accurate tracking of retro-reflective markers, such as those used in surgical procedures, without the need to integrate any additional components. This framework is also capable of simultaneously tracking multiple tools. Our results show that the tracking and detection of the markers can be achieved with an accuracy of 0.09 ± 0.06 mm on lateral translation, 0.42 ± 0.32 mm on longitudinal translation and 0.80 ± 0.39◦ for rotations around the vertical axis. Furthermore, to showcase the relevance of the proposed framework, we evaluate the system’s performance in the context of surgical procedures. This use case was designed to replicate the scenarios of k-wire insertions in orthopedic procedures. For evaluation, two surgeons and one biomedical researcher were provided with visual navigation, and each performed 21 injections. Results from this use case provide comparable accuracy to those reported in the literature for AR-based navigation procedures. © 2022, CC BY.

关键词： Augmented reality

Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Dehghani, Shervin Sommersperger, Michael Zhang, Peiyao Martin-Gomez, Alejandro Busam, Benjamin Gehlbach, Peter Navab, Nassir Nasseri, M. Ali Iordachita, Iulian Department of Computer Science Technische Universität München München85748 Germany Augenklinik und Poliklinik Klinikum rechts der Isar der Technische Universität München München81675 Germany Wilmer Eye Institute Johns Hopkins Hospital BaltimoreMD United States Laboratory for Computational Sensing and Robotics Johns Hopkins Universit Baltimore MD United States Computer Aided Medical Procedures & Augmented Reality Technical University of Munich Munich85748 Germany The Whiting School of Engineering Johns Hopkins University BaltimoreMD United States

In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT introduces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems. © 2023, CC BY-SA.

关键词： Semantics

Collaboration analysis using deep learning

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Guo, Zhang Yu, Kevin Pearlman, Rebecca Navab, Nassir Barmaki, Roghayeh Department of Computer and Information Sciences University of Delaware Chair for Computer Aided Medical Procedures and Augmented Reality Technical University of Munich Department of Biology Krieger School of Arts and Sciences Johns Hopkins University Laboratory of Computational Sensing and Robotics Johns Hopkins University

The analyses of collaborative learning process is one of the growing field of education research, which has many different analytic solutions. In this paper, we provided a new solution to improve the automated collaborative learning analyses using deep neural networks. Instead of using selfreported questionnaires, which are subject to bias and noise, we automatically extract group-working information by object recognition results using Mask R-CNN method. This process is based on detecting the people and other objects from pictures and video clips of the collaborative learning process, then evaluate the mobile learning performance using the collaborative indicators. We tested our approach to automatically evaluate the group-work collaboration in a controlled study of 33 dyads while performing an anatomy body painting intervention. The results indicate that our approach recognizes the differences of collaborations among teams of treatment and control groups in the case study;F(1, 32)= 4.83, p MSC Codes 68U10 Copyright © 2019, The Authors. All rights reserved.

关键词： Object recognition

Exploiting partial structural symmetry for patient-specific image augmentation in trauma interventions

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Fotouhi, Javad Unberath, Mathias Taylor, Giacomo Farashahi, Arash Ghaani Bier, Bastian Taylor, Russell H. Osgood, Greg M. Armand, Mehran Navab, Nassir Computer Aided Medical Procedures Johns Hopkins University Department of Mechanical Engineering Johns Hopkins University Laboratory for Computational Sensing and Robotics Johns Hopkins University Department of Orthopaedic Surgery Johns Hopkins Hospital Applied Physics Laboratory Johns Hopkins University Computer Aided Medical Procedures Technische Universität München

In unilateral pelvic fracture reductions, surgeons attempt to reconstruct the bone fragments such that bilateral symmetry in the bony anatomy is restored. We propose to exploit this"structurally symmetric" nature of the pelvic bone, and provide intra-operative image augmentation to assist the surgeon in repairing dislocated fragments. The main challenge is to automatically estimate the desired plane of symmetry within the patient’s pre-operative CT. We propose to estimate this plane using a non-linear optimization strategy, by minimizing Tukey’s biweight robust estimator, relying on the partial symmetry of the anatomy. Moreover, a regularization term is designed to enforce the similarity of bone density histograms on both sides of this plane, relying on the biological fact that, even if injured, the dislocated bone segments remain within the body. The experimental results demonstrate the performance of the proposed method in estimating this"plane of partial symmetry" using CT images of both healthy and injured anatomy. Examples of unilateral pelvic fractures are used to show how intra-operative X-ray images could be augmented with the forward-projections of the mirrored anatomy, acting as objective road-map for fracture reduction procedures. Copyright © 2018, The Authors. All rights reserved.

关键词： computerized tomography

Augmented reality-based feedback for technician-in-the-loop c-arm repositioning

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Unberath, Mathias Fotouhi, Javad Hajek, Jonas Maier, Andreas Osgood, Greg Taylor, Russell Armand, Mehran Navab, Nassir Computer Aided Medical Procedures Johns Hopkins University Pattern Recognition Laboratory FAU Erlangen-Nuremberg Orthopaedic Trauma Johns Hopkins University Laboratory for Computational Sensing and Robotics Johns Hopkins University Applied Physics Laboratory Johns Hopkins University

Interventional C-arm imaging is crucial to percutaneous orthopedic procedures as it enables the surgeon to monitor the progress of surgery on the anatomy level. Minimally invasive interventions require repeated acquisition of X-ray images from different anatomical views to verify tool placement. Achieving and reproducing these views often comes at the cost of increased surgical time and radiation dose to both patient and staff. This work proposes a marker-free"technician-in-the-loop" Augmented Reality (AR) solution for C-arm repositioning. The X-ray technician operating the C-arm interventionally is equipped with a head-mounted display capable of recording desired C-arm poses in 3D via an integrated infrared sensor. For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician. We conduct experiments in a setting simulating orthopedic trauma surgery. Our proof-of-principle findings indicate that the proposed system can decrease the 2.76 X-ray images required per desired view down to zero, suggesting substantial reductions of radiation dose during C-arm repositioning. The proposed AR solution is a first step towards facilitating communication between the surgeon and the surgical staff, improving the quality of surgical image acquisition, and enabling context-aware guidance for surgery rooms of the future. The concept of technician-in-the-loop design will become relevant to various interventions considering the expected advancements of sensing and wearable computing in the near future. Copyright © 2018, The Authors. All rights reserved.

关键词： Robotic arms