检索结果-内蒙古大学图书馆

From marker features to multimodal fusion: a review of vision-based tactile sensor design and development

TM-TECHNISCHES MESSEN 2025年第6期92卷 262-276页

作者： Han, Ning Jingjin, Shen Nanjing Univ Posts & Telecommun Coll Artificial Intelligence & Automat Nanjing Peoples R China

vision-based tactile sensors capture visual information from contact surfaces to perceive tactile signals. In recent years, these sensors have been widely used in robotic systems to enhance their tactile perception capabilities. To meet the demands of various application scenarios, numerous hardware designs for these sensors have been developed. This paper reviews the design and development of vision-based tactile sensors. based on their sensing principles and implementation methods, we categorize existing sensors into three main types: vision-based tactile sensors based on marked features, vision-based tactile sensors based on coating geometric features, and vision-tactile modality fusion sensors. For each type, we delve into the core technical challenges, existing solutions, and corresponding hardware implementation strategies. By summarizing the characteristics and solutions of existing sensors, this paper aims to provide researchers with a comprehensive reference to past studies and solutions, while also exploring potential future research directions. Bildverarbeitungsbasierte taktile sensoren erfassen visuelle Informationen von Kontaktfl & auml;chen, um taktile Signale wahrzunehmen. In den letzten Jahren wurden diese sensoren h & auml;ufig in Robotersystemen eingesetzt, um deren taktile Wahrnehmungsf & auml;higkeiten zu verbessern. Um den Anforderungen der verschiedenen Anwendungsszenarien gerecht zu werden, wurden zahlreiche Hardware-Designs f & uuml;r diese sensoren entwickelt. Dieser Beitrag gibt einen & Uuml;berblick & uuml;ber das Design und die Entwicklung von bildverarbeitungsbasierten taktilen sensoren. Basierend auf ihren Erfassungsprinzipien und Implementierungsmethoden kategorisieren wir die vorhandenen sensoren in drei Haupttypen: Bildverarbeitungsbasierte taktile sensoren, die auf markierten Merkmalen basieren, bildverarbeitungsbasierte taktile sensoren, die auf geometrischen Beschichtungsmerkmalen basieren, und bildverarbeitungsbasierte taktile Mo

关键词： vision-based tactile sensor tactile perception hardware design vision-basierter taktiler sensor taktile Wahrnehmung Hardware-Design

来源：评论

学校读者我要写书评

暂无评论

PaLmTac: A vision-based tactile sensor Leveraging Distributed-Modality Design and Modal-Matching Recognition for Soft Hand Perception

引用

IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 2024年第3期18卷 288-298页

作者： Zhang, Shixin Yang, Yiyong Shan, Jianhua Sun, Fuchun Xue, Hongxiang Fang, Bin China Univ Geosci Sch Engn & Technol Beijing 100083 Peoples R China Beijing Univ Posts & Telecommun Sch Artificial Intelligence Beijing 100876 Peoples R China Anhui Univ Technol Anhui Prov Key Lab Special Heavy Load Robot Maanshan 243002 Peoples R China Tsinghua Univ Inst Artificial Intelligence Beijing Natl Res Ctr Informat Sci & Technol Dept Comp Sci & TechnolState Key Lab Intelligent Beijing 100084 Peoples R China Fudan Univ Inst Engn & Appl Technol Shanghai 200433 Peoples R China

This paper proposes a vision-based tactile sensor (VBTS) embedded into the soft hand palm, named PaLmTac. We adopt a distributed modality design instead of overlaying function layers. On the one hand, the problem of unrelated modality integration (texture and temperature) is solved. On the other hand, combining regional recognition can avoid mixed unrelated information. Herein, a Level-Regional Feature Extraction Network (LRFE-Net) is presented to match the modality design. We leverage feature mapping, regional convolution, and regional vectorization to construct the regional recognition mechanism, which can extract features in parallel and control fusion degrees. The level recognition mechanism balances the learning difficulty of each modality. Compared with the existing VBTSs, the PaLmTac optimizes unrelated modality integration and reduces fusion interference. This paper provides a novel idea of multimodal VBTS design and sensing mechanism, which is expected to be applied to human-computer interaction scenarios based on multimodal fusion.

关键词： Level recognition mechanism regional recognition mechanism soft hand vision-based tactile sensor Level recognition mechanism regional recognition mechanism soft hand vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

An improved DFD method for three-dimensional displacement measurement of vision-based tactile sensor

引用

sensorS AND ACTUATORS A-PHYSICAL 2024年 378卷

作者： Ma, Zenghong Tan, Li Zeng, Wei Du, Xiaoqiang He, Leiying Wu, Chuanyu Zhejiang Sci Tech Univ Sch Mech Engn Hangzhou 310018 Peoples R China Zhejiang Key Lab Transplanting Equipment & Technol Hangzhou 310018 Peoples R China Minist Agr & Rural Affairs Coconstruct Minist & Prov Key Lab Agr Equipment Hilly & Mountainous Areas So Hangzhou 310018 Peoples R China

Currently, traditional tactile sensors based on the principles of capacitance or piezoelectricity have complex structures and difficulty in obtaining tactile information. A vision-based tactile sensor is introduced which can realize visual measurement of three-dimensional displacement in this paper. The vision-based tactile sensor is mainly composed of an elastomer embedded with marker point array, a transparent acrylic plate, 8 LED lights and a micro monocular camera. The elastomer deforms when the tactile sensor contacts an object, and the micro monocular camera is used to capture the elastomer deformation and transmit it to the computer in the form of image, and then the three-dimensional displacement information is obtained by processing the image. In order to more accurately recover the missing dimensional information in the three-dimensional displacement detection of monocular camera, an improved DFD (Depth from Defocus) method based on finite element theory is proposed in this paper. It is verified by experiments that the improved DFD method proposed in this paper can measure the three-dimensional displacement information more accurately compared with the DFD method. In addition, an experiment is conducted to prove the robustness of the improved DFD method on the robotic gripper. The experimental results demonstrate that the three-dimensional displacement measurement method proposed in this paper can provide technical support for the design and development of vision-based tactile sensors.

关键词： Visual measurement Finite element theory vision-based tactile sensor Improved DFD method Three-dimensional displacement

来源：评论

学校读者我要写书评

暂无评论

HSVTac: A High-Speed vision-based tactile sensor for Exploring Fingertip tactile Sensitivity

引用

IEEE sensorS JOURNAL 2023年第19期23卷 23431-23439页

作者： Wang, Xiaoxin Yang, Yicheng Zhou, Ziliang Xiang, Guiyao Liu, Honghai Harbin Inst Technol Shenzhen State Key Lab Robot & Syst Shenzhen 518055 Peoples R China Shanghai Jiao Tong Univ Sch Mech Engn State Key Lab Mech Syst & Vibrat Shanghai 200240 Peoples R China

The sensitivity of human tactile perception is essential in sensing texture surfaces via the fingertip. However, direct investigation of the underlying physiological principles is challenging due to their complexity, while bionic vision-based tactile sensing is a promising solution. This article proposed a novel high-speed vision-based tactile sensor, denoted as HSVTac, which emulated the functional characteristics of the human fingertip. A high-speed camera with 2200 frames/s covered the frequency response range of four mechanoreceptor types, which mediated the coding of tactile stimuli. The sinusoidal vibration stimulation was applied to verify that the frequency range reached 0-600 Hz. Then, a sliding contact paradigm was designed, and two experiments were conducted with 54 types of gratings texture surfaces. Variations in gratings' width and sliding direction characterized spatial acuity, and variations in gratings' period and sliding velocity characterized temporal sensitivity. The spatial acuity and temporal sensitivity recognition rates reached 91.20% and 92.59%, respectively. The results demonstrated that the high-frequency information promoted tactile sensitivity and enhanced discrimination of texture surfaces. The proposed device represented a novel approach to investigating human tactile perception and provided the necessary equipment foundation for assessing tactile sensitivity.

关键词： Biomimetic high-speed camera tactile sensitivity texture recognition vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

BiTac: A Soft vision-based tactile sensor With Bidirectional Force Perception for Robots

引用

IEEE sensorS JOURNAL 2023年第9期23卷 9158-9167页

作者： Duong, Lac Van Hanoi Univ Sci & Technol HUST Sch Mech Engn Dept Mechatron Hanoi 112400 Vietnam

Although various designs have been reported in the literature for vision-based tactile sensors, a few of them can detect pressing forces. These designs are often addressed with learning-based approaches that critically rely on training and require large amounts of training data. This study presents a novel visuotactile sensing technology for the development of large-scale vision-based tactile sensors named BiTac, which can accurately reconstruct its 3-D surface shape in real time and is capable of detecting the bidirectional distribution of normal forces applied to its soft sensing surface. The sensor skin was made of soft and elastic materials composed of an array of markers which are tracked efficiently by a compact stereo camera, whose parameters could be calibrated automatically. Notably, the complexity of mapping the displacement fields calculated from the images to the normal force distribution is resolved by the finite element method (FEM), which would be a generalized modeling technique for soft materials. This article comprehensively covers the design, fabrication, theoretical modeling, calibration, and experiments. The hardware capability and tactile perception were assessed quantitatively and qualitatively through real interactions with various objects. With a simple structure and analytical approach, the BiTac sensor can be easily customized and manufactured for different parts of robots, making it a potential design for mass production.

关键词： Skin sensors Force Robot sensing systems Cameras Robots Finite element analysis Contact force sensing soft robotics tactile sensing vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

Detection of rotational direction of objects using a vision-based tactile sensor 24

Detection of rotational direction of objects using a vision-...

引用

24th International Conference on Control, Automation and Systems

作者： Shim, Minjae Kim, Sung-Phil Ulsan Natl Inst Sci & Technol Dept Biomed Engn Ulsan 44919 South Korea

ISBN: (纸本)9798331517939;9788993215380

tactile perception is essential for manipulating objects safely and effectively, with dynamic tactile sensing playing a crucial role in identifying contact events. While existing studies have focused on attributes such as texture and softness, this study aims to enhance robotic tactile perception by detecting the rotational direction of objects using a vision-based sensor. Dynamic tactile sensing of rotational movements can improve the success rate and stability of grasping, which is crucial for handling objects particularly in visually restricted situations. The experimental setup involved a 6-DoF robotic arm, a gripper, and a vision-based tactile sensor to collect data from various stimulus shapes, inducing rotations in four directions and modeling the contact on the tactile sensor. We decoded tactile sensor images using a convolutional neural network combined with a long short-term memory recurrent neural network (CNN-LSTM) for classifying the rotational direction. The results showed robust performance in classifying rotational directions. Our study demonstrates the potential of integrating vision-based sensors with robotic arms to enhance tactile interaction during object manipulation.

关键词： vision-based tactile sensor dynamic tactile sensing rotational direction

来源：评论

学校读者我要写书评

暂无评论

A Thin Format vision-based tactile sensor With a Microlens Array (MLA)

引用

IEEE sensorS JOURNAL 2022年第22期22卷 22069-22076页

作者： Chen, Xia Zhang, Guanlan Wang, Michael Yu Yu, Hongyu Hong Kong Univ Sci & Technol Dept Mech & Aerosp Engn Hong Kong Peoples R China Hong Kong Univ Sci & Technol Dept Elect & Comp Engn Hong Kong Peoples R China Hong Kong Univ Sci & Technol Shenzhen Hong Kong Collaborat Innovat Res Inst Shenzhen 518000 Peoples R China

vision-based tactile sensors have been widely studied in the robotics field for high spatial resolution and compatibility with machine learning algorithms. However, the currently employed sensor's imaging system is bulky, limiting its further application. Here, we present a microlens array (MLA)-based vision system to achieve a low thickness format of the sensor package with high tactile sensing performance. Multiple micromachined microlens units cover the whole elastic touching layer and provide a stitched clear tactile image, enabling high spatial resolution with a thin thickness of 5 mm. The thermal reflow and the soft lithography method ensure the uniform spherical profile and smooth surface of microlenses. Both optical and mechanical characterizations demonstrated the sensor's stable imaging and excellent tactile sensing, enabling precise 3-D tactile information, such as displacement mapping and force distribution with an ultracompact thin structure.

关键词： Lenses tactile sensors sensors Imaging sensor arrays Spatial resolution Optical sensors Microlens array (MLA) miniaturized tactile sensor optical tactile sensor vision-based tactile sensor

来源：评论

学校读者我要写书评

暂无评论

WSTac: Interactive Surface Perception based on Whisker-Inspired and Self-Illuminated vision-based tactile sensor

WSTac: Interactive Surface Perception based on Whisker-Inspi...

引用

ACM International Joint Conference on Pervasive and Ubiquitous Computing / ACM International Symposium on Wearable Computing (UbiComp/ISWC)

作者： Lei, Kai Chong Sou, Kit Wa Chan, Wang Sing Yan, Jiayi Ping, Siqi Peng, Dengfeng Ding, Wenbo Zhang, Xiao-Ping Tsinghua Berkeley Shenzhen Inst Shenzhen Peoples R China Tsinghua Univ Shenzhen Int Grad Sch Beijing Peoples R China Shenzhen Univ Key Lab Optoelect Devices & Syst Minist Educ & Guangdong Prov Coll Phys & Optoelect Engn Shenzhen Peoples R China RISC V Int Open Source Lab Shenzhen Peoples R China Ryerson Univ Dept Elect Comp & Biomed Engn Toronto ON Canada

ISBN: (纸本)9798400702006

Modern Visual-based tactile sensors (VBTSs) use cost-effective cameras to track elastomer deformation, but struggle with ambient light interference. Solutions typically involve using internal LEDs and blocking external light, thus adding complexity. Creating a VBTS resistant to ambient light with just a camera and an elastomer remains a challenge. In this work, we introduce WStac, a self-illuminating VBTS comprising a mechanoluminescence (ML) whisker elastomer, camera, and 3D printed parts. The ML whisker elastomer, inspired by the touch sensitivity of vibrissae, offers both light isolation and high ML intensity under stress, thereby removing the necessity for additional LED modules. With the incorporation of machine learning, the sensor effectively utilizes the dynamic contact variations of 25 whiskers to successfully perform tasks like speed regression, directional identification, and texture classification. Videos are available at: https://***/view/wstac/.

关键词： vision-based tactile sensor Mechanoluminescence sensor Data Processing Interactive Perception

来源：评论

学校读者我要写书评

暂无评论

Development of a vision-based tactile sensor with micro suction cups

引用

sensorS AND ACTUATORS A-PHYSICAL 2024年 371卷

作者： Shiratori, Toshihiro Kanazawa, Yuma Sakamoto, Jinya Takahashi, Tomokazu Suzuki, Masato Aoyagi, Seiji Kansai Univ Fac Engn Dept Mech Engn 3-3-35 Yamate Cho Suita Osaka 5648680 Japan

tactile sensors capable of measuring contact force distribution are being actively developed for complex manipulation by robotic hands. In recent years, vision-based tactile sensors that estimate contact conditions by observing elastomer deformation with a camera from the backside and analyzing the images have attracted much attention. In the general marker displacement method using a single camera, it is difficult to estimate the contact force in the normal direction with high resolution. By applying a microstructure array (micro suction cup array) to the surface of the elastomer that is in contact with the object, the micro suction cups are deformed by the contact force. A device was fabricated that can estimate the normal contact force to each suction cup from this deformation images using machine learning. The device can vacuum a micro suction cup array and capture its deformation from the backside. The micro suction cup array (2x2) was fabricated by transferring a master mold fabricated on a high-resolution optical fabrication 3D printer onto polydimethylsiloxane. It was shown that UV nanoimprinting with soft molds can be used to fabricate large-area micro suction cup arrays relatively easily. A regression CNN model was created to estimate the normal force applied to each suction cup from the deformation image. An average error between the measured and estimated value was 21 mN, which was sufficient for the accuracy of the estimation.

关键词： vision-based tactile sensor Micro suction cup array UV nanoimprint CNN Machine learning Femtosecond laser

来源：评论

学校读者我要写书评

暂无评论

Design and Evaluation of a Rapid Monolithic Manufacturing Technique for a Novel vision-based tactile sensor: C-Sight

引用

sensorS 2024年第14期24卷 4603页

作者： Fan, Wen Li, Haoran Xing, Yifan Zhang, Dandan Univ Bristol Sch Engn Math & Technol Bristol Robot Lab Bristol BS8 1QU England Univ Bristol Sch Comp Sci Visual Informat Lab Bristol BS8 1QU England

tactile sensing has become indispensable for contact-rich dynamic robotic manipulation tasks. It provides robots with a better understanding of the physical environment, which is a vital supplement to robotic vision perception. Compared with other existing tactile sensors, vision-based tactile sensors (VBTSs) stand out for augmenting the tactile perception capabilities of robotic systems, owing to superior spatial resolution and cost-effectiveness. Despite their advantages, VBTS production faces challenges due to the lack of standardised manufacturing techniques and heavy reliance on manual labour. This limitation impedes scalability and widespread adoption. This paper introduces a rapid monolithic manufacturing technique and evaluates its performance quantitatively. We further develop and assess C-Sight, a novel VBTS sensor manufactured using this technique, focusing on its tactile reconstruction capabilities. Experimental results demonstrate that the monolithic manufacturing technique enhances VBTS production efficiency significantly. Also, the fabricated C-Sight sensor exhibits its reliable tactile perception and reconstruction capabilities, proofing the validity and feasibility of the monolithic manufacturing method.

关键词： vision-based tactile sensor monolithic 3D printing contact reconstruction

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：