检索结果-内蒙古大学图书馆

From marker features to multimodal fusion: a review of vision-based tactile sensor design and development

TM-TECHNISCHES MESSEN 2025年第6期92卷 262-276页

作者： Han, Ning Jingjin, Shen Nanjing Univ Posts & Telecommun Coll Artificial Intelligence & Automat Nanjing Peoples R China

vision-based tactile sensors capture visual information from contact surfaces to perceive tactile signals. In recent years, these sensors have been widely used in robotic systems to enhance their tactile perception capabilities. To meet the demands of various application scenarios, numerous hardware designs for these sensors have been developed. This paper reviews the design and development of vision-based tactile sensors. based on their sensing principles and implementation methods, we categorize existing sensors into three main types: vision-based tactile sensors based on marked features, vision-based tactile sensors based on coating geometric features, and vision-tactile modality fusion sensors. For each type, we delve into the core technical challenges, existing solutions, and corresponding hardware implementation strategies. By summarizing the characteristics and solutions of existing sensors, this paper aims to provide researchers with a comprehensive reference to past studies and solutions, while also exploring potential future research directions. Bildverarbeitungsbasierte taktile sensoren erfassen visuelle Informationen von Kontaktfl & auml;chen, um taktile Signale wahrzunehmen. In den letzten Jahren wurden diese sensoren h & auml;ufig in Robotersystemen eingesetzt, um deren taktile Wahrnehmungsf & auml;higkeiten zu verbessern. Um den Anforderungen der verschiedenen Anwendungsszenarien gerecht zu werden, wurden zahlreiche Hardware-Designs f & uuml;r diese sensoren entwickelt. Dieser Beitrag gibt einen & Uuml;berblick & uuml;ber das Design und die Entwicklung von bildverarbeitungsbasierten taktilen sensoren. Basierend auf ihren Erfassungsprinzipien und Implementierungsmethoden kategorisieren wir die vorhandenen sensoren in drei Haupttypen: Bildverarbeitungsbasierte taktile sensoren, die auf markierten Merkmalen basieren, bildverarbeitungsbasierte taktile sensoren, die auf geometrischen Beschichtungsmerkmalen basieren, und bildverarbeitungsbasierte taktile Mo

关键词： vision-based tactile sensor tactile perception hardware design vision-basierter taktiler sensor taktile Wahrnehmung Hardware-Design

来源：评论

学校读者我要写书评

暂无评论

vision-based tactile sensor using depth from defocus for artificial finger in hand prosthesis

引用

ELECTRONICS LETTERS 2016年第20期52卷 1665-U19页

作者： Jiang, Hanjun Zhu, Xiyang Xie, Wenao Guo, Feng Zhang, Chun Wang, Zhihua Tsinghua Univ Inst Microelect Tsinghua Natl Lab Informat Sci & Technol Beijing Peoples R China

A vision-based tactile sensor has been implemented to build an artificial fingers capable of grasping for the hand prosthesis. The tactile sensor hardware is composed of an elastomer sensing part with one layer of markers and an image sensor. By employing the depth from defocus technique, three-dimensional contact force distribution can be reconstructed from the 2D images of only one layer of markers. The proposed hardware and algorithm have been validated through experiments.

关键词： tactile sensors image sensors prosthetics spatial variables measurement image reconstruction medical image processing vision-based tactile sensor artificial finger hand prosthesis elastomer sensing image sensor defocus technique three-dimensional contact force distribution 2D image reconstruction

来源：评论

学校读者我要写书评

暂无评论

A Thin Format vision-based tactile sensor With a Microlens Array (MLA)

引用

IEEE sensorS JOURNAL 2022年第22期22卷 22069-22076页

作者： Chen, Xia Zhang, Guanlan Wang, Michael Yu Yu, Hongyu Hong Kong Univ Sci & Technol Dept Mech & Aerosp Engn Hong Kong Peoples R China Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong Peoples R China Hong Kong Univ Sci & Technol Shenzhen Hong Kong Collaborat Innovat Res Inst Shenzhen 518000 Peoples R China

vision-based tactile sensors have been widely studied in the robotics field for high spatial resolution and compatibility with machine learning algorithms. However, the currently employed sensor's imaging system is bulky, limiting its further application. Here, we present a microlens array (MLA)-based vision system to achieve a low thickness format of the sensor package with high tactile sensing performance. Multiple micromachined microlens units cover the whole elastic touching layer and provide a stitched clear tactile image, enabling high spatial resolution with a thin thickness of 5 mm. The thermal reflow and the soft lithography method ensure the uniform spherical profile and smooth surface of microlenses. Both optical and mechanical characterizations demonstrated the sensor's stable imaging and excellent tactile sensing, enabling precise 3-D tactile information, such as displacement mapping and force distribution with an ultracompact thin structure.

关键词： Lenses tactile sensors sensors Imaging sensor arrays Spatial resolution Optical sensors Microlens array (MLA) miniaturized tactile sensor optical tactile sensor vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

HSVTac: A High-Speed vision-based tactile sensor for Exploring Fingertip tactile Sensitivity

引用

IEEE sensorS JOURNAL 2023年第19期23卷 23431-23439页

作者： Wang, Xiaoxin Yang, Yicheng Zhou, Ziliang Xiang, Guiyao Liu, Honghai Harbin Inst Technol Shenzhen State Key Lab Robot & Syst Shenzhen 518055 Peoples R China Shanghai Jiao Tong Univ Sch Mech Engn State Key Lab Mech Syst & Vibrat Shanghai 200240 Peoples R China

The sensitivity of human tactile perception is essential in sensing texture surfaces via the fingertip. However, direct investigation of the underlying physiological principles is challenging due to their complexity, while bionic vision-based tactile sensing is a promising solution. This article proposed a novel high-speed vision-based tactile sensor, denoted as HSVTac, which emulated the functional characteristics of the human fingertip. A high-speed camera with 2200 frames/s covered the frequency response range of four mechanoreceptor types, which mediated the coding of tactile stimuli. The sinusoidal vibration stimulation was applied to verify that the frequency range reached 0-600 Hz. Then, a sliding contact paradigm was designed, and two experiments were conducted with 54 types of gratings texture surfaces. Variations in gratings' width and sliding direction characterized spatial acuity, and variations in gratings' period and sliding velocity characterized temporal sensitivity. The spatial acuity and temporal sensitivity recognition rates reached 91.20% and 92.59%, respectively. The results demonstrated that the high-frequency information promoted tactile sensitivity and enhanced discrimination of texture surfaces. The proposed device represented a novel approach to investigating human tactile perception and provided the necessary equipment foundation for assessing tactile sensitivity.

关键词： Biomimetic high-speed camera tactile sensitivity texture recognition vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

BiTac: A Soft vision-based tactile sensor With Bidirectional Force Perception for Robots

引用

IEEE sensorS JOURNAL 2023年第9期23卷 9158-9167页

作者： Duong, Lac Van Hanoi Univ Sci & Technol HUST Sch Mech Engn Dept Mechatron Hanoi 112400 Vietnam

Although various designs have been reported in the literature for vision-based tactile sensors, a few of them can detect pressing forces. These designs are often addressed with learning-based approaches that critically rely on training and require large amounts of training data. This study presents a novel visuotactile sensing technology for the development of large-scale vision-based tactile sensors named BiTac, which can accurately reconstruct its 3-D surface shape in real time and is capable of detecting the bidirectional distribution of normal forces applied to its soft sensing surface. The sensor skin was made of soft and elastic materials composed of an array of markers which are tracked efficiently by a compact stereo camera, whose parameters could be calibrated automatically. Notably, the complexity of mapping the displacement fields calculated from the images to the normal force distribution is resolved by the finite element method (FEM), which would be a generalized modeling technique for soft materials. This article comprehensively covers the design, fabrication, theoretical modeling, calibration, and experiments. The hardware capability and tactile perception were assessed quantitatively and qualitatively through real interactions with various objects. With a simple structure and analytical approach, the BiTac sensor can be easily customized and manufactured for different parts of robots, making it a potential design for mass production.

关键词： Skin sensors Force Robot sensing systems Cameras Robots Finite element analysis Contact force sensing soft robotics tactile sensing vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

PaLmTac: A vision-based tactile sensor Leveraging Distributed-Modality Design and Modal-Matching Recognition for Soft Hand Perception

引用

IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 2024年第3期18卷 288-298页

作者： Zhang, Shixin Yang, Yiyong Shan, Jianhua Sun, Fuchun Xue, Hongxiang Fang, Bin China Univ Geosci Sch Engn & Technol Beijing 100083 Peoples R China Beijing Univ Posts & Telecommun Sch Artificial Intelligence Beijing 100876 Peoples R China Anhui Univ Technol Anhui Prov Key Lab Special Heavy Load Robot Maanshan 243002 Peoples R China Tsinghua Univ Inst Artificial Intelligence Beijing Natl Res Ctr Informat Sci & Technol Dept Comp Sci & TechnolState Key Lab Intelligent Beijing 100084 Peoples R China Fudan Univ Inst Engn & Appl Technol Shanghai 200433 Peoples R China

This paper proposes a vision-based tactile sensor (VBTS) embedded into the soft hand palm, named PaLmTac. We adopt a distributed modality design instead of overlaying function layers. On the one hand, the problem of unrelated modality integration (texture and temperature) is solved. On the other hand, combining regional recognition can avoid mixed unrelated information. Herein, a Level-Regional Feature Extraction Network (LRFE-Net) is presented to match the modality design. We leverage feature mapping, regional convolution, and regional vectorization to construct the regional recognition mechanism, which can extract features in parallel and control fusion degrees. The level recognition mechanism balances the learning difficulty of each modality. Compared with the existing VBTSs, the PaLmTac optimizes unrelated modality integration and reduces fusion interference. This paper provides a novel idea of multimodal VBTS design and sensing mechanism, which is expected to be applied to human-computer interaction scenarios based on multimodal fusion.

关键词： Level recognition mechanism regional recognition mechanism soft hand vision-based tactile sensor Level recognition mechanism regional recognition mechanism soft hand vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

An improved DFD method for three-dimensional displacement measurement of vision-based tactile sensor

引用

sensorS AND ACTUATORS A-PHYSICAL 2024年 378卷

作者： Ma, Zenghong Tan, Li Zeng, Wei Du, Xiaoqiang He, Leiying Wu, Chuanyu Zhejiang Sci Tech Univ Sch Mech Engn Hangzhou 310018 Peoples R China Zhejiang Key Lab Transplanting Equipment & Technol Hangzhou 310018 Peoples R China Minist Agr & Rural Affairs Coconstruct Minist & Prov Key Lab Agr Equipment Hilly & Mountainous Areas So Hangzhou 310018 Peoples R China

Currently, traditional tactile sensors based on the principles of capacitance or piezoelectricity have complex structures and difficulty in obtaining tactile information. A vision-based tactile sensor is introduced which can realize visual measurement of three-dimensional displacement in this paper. The vision-based tactile sensor is mainly composed of an elastomer embedded with marker point array, a transparent acrylic plate, 8 LED lights and a micro monocular camera. The elastomer deforms when the tactile sensor contacts an object, and the micro monocular camera is used to capture the elastomer deformation and transmit it to the computer in the form of image, and then the three-dimensional displacement information is obtained by processing the image. In order to more accurately recover the missing dimensional information in the three-dimensional displacement detection of monocular camera, an improved DFD (Depth from Defocus) method based on finite element theory is proposed in this paper. It is verified by experiments that the improved DFD method proposed in this paper can measure the three-dimensional displacement information more accurately compared with the DFD method. In addition, an experiment is conducted to prove the robustness of the improved DFD method on the robotic gripper. The experimental results demonstrate that the three-dimensional displacement measurement method proposed in this paper can provide technical support for the design and development of vision-based tactile sensors.

关键词： Visual measurement Finite element theory vision-based tactile sensor Improved DFD method Three-dimensional displacement

来源：评论

学校读者我要写书评

暂无评论

vision-based tactile sensor Mechanism for the Estimation of Contact Position and Force Distribution Using Deep Learning

引用

sensorS 2021年第5期21卷 1920-1920页

作者： Kakani, Vijay Cui, Xuenan Ma, Mingjie Kim, Hakil Inha Univ Informat & Commun Engn 100 Inharo Incheon 22212 South Korea VisionIn Inc Global R&D Ctr 704 Ace Gasan Tower121 Digital Ro Seoul 08505 South Korea

This work describes the development of a vision-based tactile sensor system that utilizes the image-based information of the tactile sensor in conjunction with input loads at various motions to train the neural network for the estimation of tactile contact position, area, and force distribution. The current study also addresses pragmatic aspects, such as choice of the thickness and materials for the tactile fingertips and surface tendency, etc. The overall vision-based tactile sensor equipment interacts with an actuating motion controller, force gauge, and control PC (personal computer) with a LabVIEW software on it. The image acquisition was carried out using a compact stereo camera setup mounted inside the elastic body to observe and measure the amount of deformation by the motion and input load. The vision-based tactile sensor test bench was employed to collect the output contact position, angle, and force distribution caused by various randomly considered input loads for motion in X, Y, Z directions and RxRy rotational motion. The retrieved image information, contact position, area, and force distribution from different input loads with specified 3D position and angle are utilized for deep learning. A convolutional neural network VGG-16 classification modelhas been modified to a regression network model and transfer learning was applied to suit the regression task of estimating contact position and force distribution. Several experiments were carried out using thick and thin sized tactile sensors with various shapes, such as circle, square, hexagon, for better validation of the predicted contact position, contact area, and force distribution.

关键词： vision-based tactile sensor deep learning contact position contact area force distribution

来源：评论

学校读者我要写书评

暂无评论

WSTac: Interactive Surface Perception based on Whisker-Inspired and Self-Illuminated vision-based tactile sensor

WSTac: Interactive Surface Perception based on Whisker-Inspi...

引用

ACM International Joint Conference on Pervasive and Ubiquitous Computing / ACM International Symposium on Wearable Computing (UbiComp/ISWC)

作者： Lei, Kai Chong Sou, Kit Wa Chan, Wang Sing Yan, Jiayi Ping, Siqi Peng, Dengfeng Ding, Wenbo Zhang, Xiao-Ping Tsinghua Berkeley Shenzhen Inst Shenzhen Peoples R China Tsinghua Univ Shenzhen Int Grad Sch Beijing Peoples R China Shenzhen Univ Key Lab Optoelect Devices & Syst Minist Educ & Guangdong Prov Coll Phys & Optoelect Engn Shenzhen Peoples R China RISC V Int Open Source Lab Shenzhen Peoples R China Ryerson Univ Dept Elect Comp & Biomed Engn Toronto ON Canada

ISBN: (纸本)9798400702006

Modern Visual-based tactile sensors (VBTSs) use cost-effective cameras to track elastomer deformation, but struggle with ambient light interference. Solutions typically involve using internal LEDs and blocking external light, thus adding complexity. Creating a VBTS resistant to ambient light with just a camera and an elastomer remains a challenge. In this work, we introduce WStac, a self-illuminating VBTS comprising a mechanoluminescence (ML) whisker elastomer, camera, and 3D printed parts. The ML whisker elastomer, inspired by the touch sensitivity of vibrissae, offers both light isolation and high ML intensity under stress, thereby removing the necessity for additional LED modules. With the incorporation of machine learning, the sensor effectively utilizes the dynamic contact variations of 25 whiskers to successfully perform tasks like speed regression, directional identification, and texture classification. Videos are available at: https://***/view/wstac/.

关键词： vision-based tactile sensor Mechanoluminescence sensor Data Processing Interactive Perception

来源：评论

学校读者我要写书评

暂无评论

Acquisition of tactile Information by vision-based tactile sensor for Dexterous Handling of Robot Hands

Acquisition of Tactile Information by Vision-based Tactile S...

引用

IEEE/RSJ International Conference on Intelligent Robots and Systems

作者： Ito, Yuji Kim, Youngwoo Nagai, Chikara Obinata, Goro Nagoya Univ Grad Sch Engn Chikusa Ku Furo Cho Nagoya Aichi 4648603 Japan Nagoya Univ EcoTopia Sci Inst Chikusa Ku Nagoya Aichi 4648603 Japan

ISBN: (纸本)9781424466757

In this paper, we propose a new method for calculation of the contact region and the location of an object from the data acquired from vision-based tactile sensor, which touch pad is an elastic transparent membrane of silicon rubber with dotted pattern printed on its inner side. The membrane is filled with translucent red colored water. The algorithm for calculation of the contact region is based on the curvature radius of the elastic membrane and considerations of the conditions for balance between the tensional forces of the membrane and the inner pressure. The location of the object is calculated from the relations between the translational displacement of the object and the displacement of the contacting dots. The rotation angles of the object are estimated from the rotation matrix of the three basis vectors by accumulating changes of the rotation angles of the rotation matrix for applying to the object rolling on the sensor. This paper also provides some experimental results that demonstrate the usefulness of the proposed method. Proposed method can be applied in many robot tasks that require dexterity of the robot hands.

关键词： contact location contact region contacting dots curvature radius data acquisition dexterous handling dexterous manipulators elastic transparent membrane inner pressure membranes object rolling robot hands robot vision rotation matrix silicon rubber tactile information acquisition tactile sensors tensional forces touch pad translational displacement translucent red colored water vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：