The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Cameron N Riviere - One of the best experts on this subject based on the ideXlab platform.
-
vision based control of a handheld surgical micromanipulator with virtual fixtures
International Conference on Robotics and Automation, 2013Co-Authors: Brian C Becker, Robert A Maclachlan, Louis A Lobes, Gregory D Hager, Cameron N RiviereAbstract:Performing Micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate Micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p <; 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p <; 0.05).
-
handheld Micromanipulation with vision based virtual fixtures
International Conference on Robotics and Automation, 2011Co-Authors: Brian C Becker, Robert A Maclachlan, Gregory D Hager, Cameron N RiviereAbstract:Precise movement during Micromanipulation becomes difficult in submillimeter workspaces, largely due to the destabilizing influence of tremor. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to greatly improve micromanipulator precision. In this paper, we derive a control law for position-based virtual fixtures within the framework of an active handheld micromanipulator, where the fixtures are generated in real-time from microscope video. Additionally, we develop motion scaling behavior centered on virtual fixtures as a simple and direct extension to our formulation. We demonstrate that hard and soft (motion-scaled) virtual fixtures outperform state-of-the-art tremor cancellation performance on a set of artificial but medically relevant tasks: holding, move-and-hold, curve tracing, and volume restriction.
Yu Sun - One of the best experts on this subject based on the ideXlab platform.
-
robotic Micromanipulation fundamentals and applications
Social Science Research Network, 2019Co-Authors: Zhuoran Zhang, Xian Wang, Jun Liu, Changsheng Dai, Yu SunAbstract:Robotic Micromanipulation is a relatively young field. However, after three decades of development and evolution, the fundamental physics; techniques for sensing, actuation, and control; tool sets ...
-
A Micromanipulation system for single cell deposition
Proceedings - IEEE International Conference on Robotics and Automation, 2010Co-Authors: Zhe Lu, Li Dan You, Christopher Moraes, Craig A Simmons, Yan Zhao, Yu SunAbstract:Many microfabricated devices have been developed to quantify cellular response to a multitude of stimuli at a single-cell level in a high throughput manner. These single-cell studies require cells to be individually positioned at defined locations on a microdevice. This paper presents a Micromanipulation system for automated pick-place of single cells. Integrating computer vision and motion control algorithms, the system visually tracks a cell in real time and controls multiple motion devices coordinately. Via fine manipulation of picoliter fluids and pressure of a few Pascals, the system accurately picks up a single cell, transfers the cell, and deposits it at a target location at a speed of 15-30 sec/cell. The Micromanipulation system has the advantages of non-invasiveness, high specificity, and high precision. It is suitable to pick-place both non-labeled and labeled cells and applicable to standard cell culture substrates and microdevices with an open top.
-
microrobotics for molecular biology springer tracts in advanced robotics
Springer tracts in advanced robotics, 2005Co-Authors: Bradley J Nelson, Yu Sun, Michael Allen GremingerAbstract:Recent advances in molecular biology such as cloning demonstrate that increasingly complex Micromanipulation strategies for manipulating individual biological cells are required. From a robotics standpoint, the manipulation of biological cells, sometimes referred to as biomanipulation, presents several interesting research issues that extend well beyond cell manipulation. Biological cells are highly deformable objects, and the material properties of these objects are not well quantified, so developing strategies for manipulating deformable objects must be addressed. Most biological cells are between 1μm and 100μm in diameter, depending on the cell type, so Micromanipulation issues must be explored, including the appropriate use of high resolution, low depth-of-field vision feedback and very low magnitude multi-axis force feedback. By pursuing robotic manipulation of biological cells, many interesting robotics research avenues in Micromanipulation, deformable object handling, multi-sensor integration, and force and vision feedback assimilation must be explored. This paper explores the visual tracking of biological cells using physics-based models and the measurement of applied force fields using a new cell deformation model with visual feedback. A multi-axis MEMS based force sensor is used to determine applied forces and develop models of cell deformation. Robust tracking of cell deformation is shown and real-time determination of applied force fields is demonstrated. In addition, the system developed has been used to quantitate for the first time a phenomenon known as “zona hardening” during mouse oocyte fertilization.
-
microrobotics for molecular biology manipulating deformable objects at the microscale
ISRR, 2005Co-Authors: Bradley J Nelson, Yu Sun, Michael Allen GremingerAbstract:Recent advances in molecular biology such as cloning demonstrate that increasingly complex Micromanipulation strategies for manipulating individual biological cells are required. From a robotics standpoint, the manipulation of biological cells, sometimes referred to as biomanipulation, presents several interesting research issues that extend well beyond cell manipulation. Biological cells are highly deformable objects, and the material properties of these objects are not well quantified, so developing strategies for manipulating deformable objects must be addressed. Most biological cells are between 1μm and 100μm in diameter, depending on the cell type, so Micromanipulation issues must be explored, including the appropriate use of high resolution, low depth-of-field vision feedback and very low magnitude multi-axis force feedback. By pursuing robotic manipulation of biological cells, many interesting robotics research avenues in Micromanipulation, deformable object handling, multi-sensor integration, and force and vision feedback assimilation must be explored. This paper explores the visual tracking of biological cells using physics-based models and the measurement of applied force fields using a new cell deformation model with visual feedback. A multi-axis MEMS-based force sensor is used to determine applied forces and develop models of cell deformation. Robust tracking of cell deformation is shown and real-time determination of applied force fields is demonstrated. In addition, the system developed has been used to quantitate for the first time a phenomenon known as “zona hardening” during mouse oocyte fertilization.
Bradley J Nelson - One of the best experts on this subject based on the ideXlab platform.
-
rodbot a rolling microrobot for Micromanipulation
International Conference on Robotics and Automation, 2015Co-Authors: Roel Pieters, Hsiwen Tung, Samuel Charreyron, David F Sargent, Bradley J NelsonAbstract:We introduce the modelling and control of a rolling microrobot. The microrobot is capable of manipulating micro-objects through the use of a magnetic visual control system. This system consists of a rod-shaped microrobot, a magnetic actuation system and a visual control system. Motion of the rolling microrobot on a supporting surface is induced by a rotating magnetic field. As the robot is submerged in a liquid this motion creates a rising flow in front, a sinking flow behind, and a vortex above the robot, thus enabling non-contact transportation of micro-objects. Besides this fluid-vortex approach, the microrobot is also able to manipulate micro-objects via a pushing strategy. We present the design and modelling of the 50×60×300 µm micro-agent, the visual control system, and an experimental analysis of the Micromanipulation and control methods.
-
artificial bacterial flagella for Micromanipulation
Lab on a Chip, 2010Co-Authors: Li Zhang, Kathrin E Peyer, Bradley J NelsonAbstract:This article presents an overview of recent developments in artificial bacterial flagella (ABFs) and discusses challenges and opportunities in pursuing applications. These helical swimmers possess several advantageous characteristics, such as high swimming velocity and precise motion control indicating their potential for diverse applications. One application is the manipulation of small objects within liquid, which is the focus of this review. Preliminary results have shown that ABFs are capable of performing microobject manipulation either directly by mechanical contact or indirectly by generating a localized fluid flow. The latter approach can be used for batch manipulation without direct contact, also implying possibilities for flow control in lab-on-a-chip systems. Miniaturized helical swimmers are also promising for biomedical applications, such as targeted drug delivery and implantation or removal of tissues and other objects.
-
microrobotics for molecular biology springer tracts in advanced robotics
Springer tracts in advanced robotics, 2005Co-Authors: Bradley J Nelson, Yu Sun, Michael Allen GremingerAbstract:Recent advances in molecular biology such as cloning demonstrate that increasingly complex Micromanipulation strategies for manipulating individual biological cells are required. From a robotics standpoint, the manipulation of biological cells, sometimes referred to as biomanipulation, presents several interesting research issues that extend well beyond cell manipulation. Biological cells are highly deformable objects, and the material properties of these objects are not well quantified, so developing strategies for manipulating deformable objects must be addressed. Most biological cells are between 1μm and 100μm in diameter, depending on the cell type, so Micromanipulation issues must be explored, including the appropriate use of high resolution, low depth-of-field vision feedback and very low magnitude multi-axis force feedback. By pursuing robotic manipulation of biological cells, many interesting robotics research avenues in Micromanipulation, deformable object handling, multi-sensor integration, and force and vision feedback assimilation must be explored. This paper explores the visual tracking of biological cells using physics-based models and the measurement of applied force fields using a new cell deformation model with visual feedback. A multi-axis MEMS based force sensor is used to determine applied forces and develop models of cell deformation. Robust tracking of cell deformation is shown and real-time determination of applied force fields is demonstrated. In addition, the system developed has been used to quantitate for the first time a phenomenon known as “zona hardening” during mouse oocyte fertilization.
-
microrobotics for molecular biology manipulating deformable objects at the microscale
ISRR, 2005Co-Authors: Bradley J Nelson, Yu Sun, Michael Allen GremingerAbstract:Recent advances in molecular biology such as cloning demonstrate that increasingly complex Micromanipulation strategies for manipulating individual biological cells are required. From a robotics standpoint, the manipulation of biological cells, sometimes referred to as biomanipulation, presents several interesting research issues that extend well beyond cell manipulation. Biological cells are highly deformable objects, and the material properties of these objects are not well quantified, so developing strategies for manipulating deformable objects must be addressed. Most biological cells are between 1μm and 100μm in diameter, depending on the cell type, so Micromanipulation issues must be explored, including the appropriate use of high resolution, low depth-of-field vision feedback and very low magnitude multi-axis force feedback. By pursuing robotic manipulation of biological cells, many interesting robotics research avenues in Micromanipulation, deformable object handling, multi-sensor integration, and force and vision feedback assimilation must be explored. This paper explores the visual tracking of biological cells using physics-based models and the measurement of applied force fields using a new cell deformation model with visual feedback. A multi-axis MEMS-based force sensor is used to determine applied forces and develop models of cell deformation. Robust tracking of cell deformation is shown and real-time determination of applied force fields is demonstrated. In addition, the system developed has been used to quantitate for the first time a phenomenon known as “zona hardening” during mouse oocyte fertilization.
-
adhesion force modeling and measurement for Micromanipulation
Microrobotics and micromanipulation. Conference, 1998Co-Authors: Yu Zhou, Bradley J NelsonAbstract:It is well known that surface effect forces such as van der Waals, electrostatic, and surface tension forces dominate part interactions as part dimensions fall below approximately 100 microns. Many researchers have suggested manipulation strategies that either diminish the effect of these forces or use these forces to advantage. There is little work, however, that comprehensively analyzes, both theoretically and experimentally, the exact contributions of such phenomena as surface roughness, material properties, environmental conditions, etc. to part interactions at microscales. This paper describes our work in developing a high resolution force sensor using optical beam deflection techniques for characterizing object interactions at the microscale. The interactions among a variety of micropart shapes and materials of varying surface roughness and conductivity were analyzed under various environmental conditions. Experimental results of this analysis are presented.© (1998) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
Brian C Becker - One of the best experts on this subject based on the ideXlab platform.
-
vision based control of a handheld surgical micromanipulator with virtual fixtures
International Conference on Robotics and Automation, 2013Co-Authors: Brian C Becker, Robert A Maclachlan, Louis A Lobes, Gregory D Hager, Cameron N RiviereAbstract:Performing Micromanipulation and delicate operations in submillimeter workspaces is difficult because of destabilizing tremor and imprecise targeting. Accurate Micromanipulation is especially important for microsurgical procedures, such as vitreoretinal surgery, to maximize successful outcomes and minimize collateral damage. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to further improve performance. In this paper, we derive a virtual fixture framework for active handheld micromanipulators that is based on high-bandwidth position measurements rather than forces applied to a robot handle. For applicability in surgical environments, the fixtures are generated in real time from microscope video during the procedure. Additionally, we develop motion scaling behavior around virtual fixtures as a simple and direct extension to the proposed framework. We demonstrate that virtual fixtures significantly outperform tremor cancellation algorithms on a set of synthetic tracing tasks (p <; 0.05). In more medically relevant experiments of vein tracing and membrane peeling in eye phantoms, virtual fixtures can significantly reduce both positioning error and forces applied to tissue (p <; 0.05).
-
handheld Micromanipulation with vision based virtual fixtures
International Conference on Robotics and Automation, 2011Co-Authors: Brian C Becker, Robert A Maclachlan, Gregory D Hager, Cameron N RiviereAbstract:Precise movement during Micromanipulation becomes difficult in submillimeter workspaces, largely due to the destabilizing influence of tremor. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to greatly improve micromanipulator precision. In this paper, we derive a control law for position-based virtual fixtures within the framework of an active handheld micromanipulator, where the fixtures are generated in real-time from microscope video. Additionally, we develop motion scaling behavior centered on virtual fixtures as a simple and direct extension to our formulation. We demonstrate that hard and soft (motion-scaled) virtual fixtures outperform state-of-the-art tremor cancellation performance on a set of artificial but medically relevant tasks: holding, move-and-hold, curve tracing, and volume restriction.
Liangjing Yang - One of the best experts on this subject based on the ideXlab platform.
-
confidence based hybrid tracking to overcome visual tracking failures in calibration less vision guided Micromanipulation
IEEE Transactions on Automation Science and Engineering, 2020Co-Authors: Liangjing Yang, Ishara Paranawithana, Kamal YouceftoumiAbstract:This article proposes a confidence-based approach for combining two visual tracking techniques to minimize the influence of unforeseen visual tracking failures to achieve uninterrupted vision-based control. Despite research efforts in vision-guided Micromanipulation, existing systems are not designed to overcome visual tracking failures, such as inconsistent illumination condition, regional occlusion, unknown structures, and nonhomogenous background scene. There remains a gap in expanding current procedures beyond the laboratory environment for practical deployment of vision-guided Micromanipulation system. A hybrid tracking method, which combines motion-cue feature detection and score-based template matching, is incorporated in an uncalibrated vision-guided workflow capable of self-initializing and recovery during the Micromanipulation. Weighted average, based on the respective confidence indices of the motion-cue feature localization and template-based trackers, is inferred from the statistical accuracy of feature locations and the similarity score-based template matches. Results suggest improvement of the tracking performance using hybrid tracking under the conditions. The mean errors of hybrid tracking are maintained at subpixel level under adverse experimental conditions while the original template matching approach has mean errors of 1.53, 1.73, and 2.08 pixels. The method is also demonstrated to be robust in the nonhomogeneous scene with an array of plant cells. By proposing a self-contained fusion method that overcomes unforeseen visual tracking failures using pure vision approach, we demonstrated the robustness in our developed low-cost Micromanipulation platform. Note to Practitioners —Cell manipulation is traditionally done in highly specialized facilities and controlled environment. Existing vision-based methods do not readily fulfill the need for the unique requirements in cell manipulation including prospective plant cell-related applications. There is a need for robust visual tracking to overcome visual tracking failure during the automated vision-guided Micromanipulation. To address the gap in maintaining continuous tracking for vision-guided Micromanipulation under unforeseen visual tracking failures, we proposed a purely visual data-driven hybrid tracking approach. Our proposed confidence-based approach combines two tracking techniques to minimize the influence of scene uncertainties, hence, achieving uninterrupted vision-based control. Because of its readily deployable design, the method can be generalized for a wide range of vision-guided Micromanipulation applications. This method has the potential to significantly expand the capability of cell manipulation technology to even include prospective applications associated with plant cells, which are yet to be explored.
-
Automatic Vision-Guided Micromanipulation for Versatile Deployment and Portable Setup
IEEE Transactions on Automation Science and Engineering, 2018Co-Authors: Liangjing Yang, Ishara Paranawithana, Kamal Youcef-toumiAbstract:In this paper, an automatic vision-guided Micromanipulation approach to facilitate versatile deployment and portable setup is proposed. This paper is motivated by the importance of Micromanipulation and the limitations in existing automation technology in Micromanipulation. Despite significant advancements in Micromanipulation techniques, there remain bottlenecks in integrating and adopting automation for this application. An underlying reason for the gaps is the difficulty in deploying and setting up such systems. To address this, we identified two important design requirements, namely, portability and versatility of the Micromanipulation platform. A self-contained vision-guided approach requiring no complicated preparation or setup is proposed. This is achieved through an uncalibrated self-initializing workflow algorithm also capable of assisted targeting. The feasibility of the solution is demonstrated on a low-cost portable microscope camera and compact actuated microstages. Results suggest subpixel accuracy in localizing the tool tip during initialization steps. The self-focus mechanism could recover intentional blurring of the tip by autonomously manipulating it 95.3% closer to the focal plane. The average error in visual servo is less than a pixel with our depth compensation mechanism showing better maintaining of similarity score in tracking. Cell detection rate in a 1637-frame video stream is 97.7% with subpixels localization uncertainty. Our work addresses the gaps in existing automation technology in the application of robotic vision-guided Micromanipulation and potentially contributes to the way cell manipulation is performed. Note to Practitioners —This paper introduces an automatic method for Micromanipulation using visual information from microscopy. We design an automatic workflow, which consists of: 1) self-initialization; 2) vision-guided manipulation; and 3) assisted targeting, and demonstrate versatile deployment of the micromanipulator on a portable microscope camera setup. Unlike existing systems, our proposed method does not require any tedious calibration or expensive setup making it mobile and low cost. This overcomes the constraints of traditional practices that confine automated cell manipulation to a laboratory setting. By extending the application beyond the laboratory environment, automated Micromanipulation technology can be made more ubiquitous and expands readily to facilitate field study.
-
scene adaptive fusion of visual and motion tracking for vision guided Micromanipulation in plant cells
Conference on Automation Science and Engineering, 2018Co-Authors: Ishara Paranawithana, Liangjing Yang, Uxuan Tan, Zhong Chen, Kamal YouceftoumiAbstract:This work proposes a fusion mechanism that overcomes the traditional limitations in vision-guided Micromanipulation in plant cells. Despite the recent advancement in vision-guided Micromanipulation, only a handful of research addressed the intrinsic issues related to Micromanipulation in plant cells. Unlike single cell manipulation, the structural complexity of plant cells makes visual tracking extremely challenging. There is therefore a need to complement the visual tracking approach with trajectory data from the manipulator. Fusion of the two sources of data is done by combining the projected trajectory data to the image domain and template tracking data using a score-based weighted averaging approach. Similarity score reflecting the confidence of a particular localization result is used as the basis of the weighted average. As the projected trajectory data of the manipulator is not at all affected by the visual disturbances such as regional occlusion, fusing estimations from two sources leads to improved tracking performance. Experimental results suggest that fusion-based tracking mechanism maintains a mean error of 2.15 pixels whereas template tracking and projected trajectory data has a mean error of 2.49 and 2.61 pixels, respectively. Path B of the square trajectory demonstrated a significant improvement with a mean error of 1.11 pixels with 50% of the tracking ROI occluded by plant specimen. Under these conditions, both template tracking and projected trajectory data show similar performances with a mean error of 2.59 and 2.58 pixels, respectively. By addressing the limitations and unmet needs in the application of plant cell bio-manipulation, we hope to bridge the gap in the development of automatic vision-guided Micromanipulation in plant cells.
-
detect focus track servo dfts a vision based workflow algorithm for robotic image guided Micromanipulation
Other repository, 2017Co-Authors: Liangjing Yang, Kamal Youceftoumi, Uxuan TanAbstract:Robotic image-guided Micromanipulation contributes towards the ease of operation, speed, accuracy, and repeatability in cell manipulation. However, such technology is not fully exploited because of the challenges in the integration of robotic modules with existing microscope systems, and the difficulty in incorporating robot assistance seamlessly into the workflow. In this paper, we propose a vision-based workflow algorithm termed Detect-Focus-Track-Servo (DFTS). It facilitates easy integration of robotic modules. It also supports user interactions while minimizing the need for manual intervention and disruption to workflow through automatic detection, focusing, tracking and servoing. Experimental results suggest satisfactory detection accuracy of 99.0 % at 70 μm tolerance. The robustness test suggests no difference in the accuracy under blurred and cluttered images. The self-focus algorithm is also demonstrated to bring the tip into focus consistently. The track-servo algorithm achieves low sub-pixel uncertainty. By proposing the DFTS workflow algorithm, we hope that the level of autonomy and ease of deployment in robot and vision modules for Micromanipulation can be improved so as to open up new possibilities in the development of robotic image-guided cell manipulation.
-
Towards automatic robot-assisted microscopy: An uncalibrated approach for robotic vision-guided Micromanipulation
2016 IEEE RSJ International Conference on Intelligent Robots and Systems (IROS), 2016Co-Authors: Liangjing Yang, Kamal Youcef-toumiAbstract:Micromanipulation during live microscopic imaging relies heavily on good manual controls, dexterity, and hand-eye coordination. However, unassisted manual operations in these procedures greatly limit the speed, repeatability, and ease of operation. This is especially challenging in the case of microinjection where the insertion path needs to be in precise alignment with the imaging plane to avoid damage to cells. In this paper, we proposed an assistive robotic system that facilitates Micromanipulation under microscopy. This comes in the form of intelligent robotic vision and guided manipulation. Using user-selected patch similarity, the system registers target templates and provides online coordinated depth compensation that ensures in-plane microinjection without the need for any prior calibration. This vision-based auto-registration approach readily integrates to any existing microscope system uncalibrated. It can also work as a standalone imaging solution with any general digital microscope camera. Experiments show that the similarity-score based depth compensation performed better than the uncompensated method. The method was shown to self-recover from an unfocused position. By robotizing conventional microscopy and Micromanipulation procedures, we hope to address traditional latent needs and open up new possibilities in the ways experimental biology is performed.