检索结果-内蒙古大学图书馆

32nd ACM International Conference on Multimedia, MM 2024

作者： Liu, Jinfu Chen, Chen Liu, Mengyuan State Key Laboratory of General Artificial Intelligence Peking University Shenzhen Graduate School Shenzhen China Center for Research in Computer Vision University of Central Florida OrlandoFL United States

ISBN: (纸本)9798400706868

Skeleton-based action recognition has garnered significant attention due to the utilization of concise and resilient skeletons. Nevertheless, the absence of detailed body information in skeletons restricts performance, while other multimodal methods require substantial inference resources and are inefficient when using multimodal data during both training and inference stages. To address this and fully harness the complementary multimodal features, we propose a novel multi-modality co-learning (MMCL) framework by leveraging the multimodal large language models (LLMs) as auxiliary networks for efficient skeleton-based action recognition, which engages in multi-modality co-learning during the training stage and keeps efficiency by employing only concise skeletons in inference. Our MMCL framework primarily consists of two modules. First, the Feature Alignment Module (FAM) extracts rich RGB features from video frames and aligns them with global skeleton features via contrastive learning. Second, the Feature Refinement Module (FRM) uses RGB images with temporal information and text instruction to generate instructive features based on the powerful generalization of multimodal LLMs. These instructive text features will further refine the classification scores and the refined scores will enhance the model's robustness and generalization in a manner similar to soft labels. Extensive experiments on NTU RGB+D, NTU RGB+D 120 and Northwestern-UCLA benchmarks consistently verify the effectiveness of our MMCL, which outperforms the existing skeleton-based action recognition methods. Meanwhile, experiments on UTD-MHAD and SYSU-Action datasets demonstrate the commendable generalization of our MMCL in zero-shot and domain-adaptive action recognition. Our code is publicly available at: https://***/liujf69/MMCL-Action. © 2024 ACM.

关键词： Musculoskeletal system

来源：评论

学校读者我要写书评

暂无评论

Deep semi-supervised knowledge distillation for overlapping cervical cell instance segmentation 23rd

Deep semi-supervised knowledge distillation for overlapping ...

引用

23rd International Conference on Medical Image Computing and computer-Assisted Intervention, MICCAI 2020

作者： Zhou, Yanning Chen, Hao Lin, Huangjing Heng, Pheng-Ann Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen China

ISBN: (纸本)9783030597092

Deep learning methods show promising results for overlapping cervical cell instance segmentation. However, in order to train a model with good generalization ability, voluminous pixel-level annotations are demanded which is quite expensive and time-consuming for acquisition. In this paper, we propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining (MMT-PSM), which consists of a teacher and a student network during training. Two networks are encouraged to be consistent both in feature and semantic level under small perturbations. The teachers self-ensemble predictions from K-time augmented samples are used to construct the reliable pseudo-labels for optimizing the student. We design a novel strategy to estimate the sensitivity to perturbations for each proposal and select informative samples from massive cases to facilitate fast and effective semantic distillation. In addition, to eliminate the unavoidable noise from the background region, we propose to use the predicted segmentation mask as guidance to enforce the feature distillation in the foreground region. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only, and outperforms state-of-the-art semi-supervised methods. Code: https://***/SIAAAAAA/MMT-PSM. © Springer Nature Switzerland AG 2020.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

Assessing performance of augmented reality-based neurosurgical training

引用

Visual Computing for Industry,Biomedicine,and Art 2019年第1期2卷 45-54页

作者： Wei-Xin Si Xiang-Yun Liao Yin-Ling Qian Hai-Tao Sun Xiang-Dong Chen Qiong Wang Pheng Ann Heng Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institutes of Advanced TechnologyChinese Academy of Sciences1068 Xueyuan AvenueShenzhen University TownShenzhen 518055China Department of Neurosurgery Zhujiang HospitalSouthern Medical UniversityGuangzhou 510282China E.N.T.department of Shenzhen University General Hospital Shenzhen 518055China Department of Computer Science and Engineering the Chinese University of Hong KongHong KongChina

This paper presents a novel augmented reality(AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical *** simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative *** achieve the AR guidance,the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real *** this regard,the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance *** propose a registration method for precise mapping of the virtual and real *** addition,the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor *** this study,we conduct AR-based guidance validation and a user study on the developed simulator,which demonstrate the high accuracy of our AR-based neurosurgery simulator,as well as the AR guidance mode’s potential to improve neurosurgery by simplifying the operation,reducing the difficulty of the operation,shortening the operation time,and increasing the precision of the operation.

关键词： Augmented reality Personalized virtual operative anatomy Neurosurgical training

来源：评论

学校读者我要写书评

暂无评论

Conditional Sequential Modulation for Efficient Global Image Retouching 16th

Conditional Sequential Modulation for Efficient Global Image...

引用

16th European Conference on computer vision, ECCV 2020

作者： He, Jingwen Liu, Yihao Qiao, Yu Dong, Chao ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT - SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Beijing China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China University of Chinese Academy of Sciences Beijing China

ISBN: (纸本)9783030586003

Photo retouching aims at enhancing the aesthetic visual quality of images that suffer from photographic defects such as over/under exposure, poor contrast, inharmonious saturation. Practically, photo retouching can be accomplished by a series of image processing operations. In this paper, we investigate some commonly-used retouching operations and mathematically find that these pixel-independent operations can be approximated or formulated by multi-layer perceptrons (MLPs). Based on this analysis, we propose an extremely light-weight framework - Conditional Sequential Retouching Network (CSRNet) - for efficient global image retouching. CSRNet consists of a base network and a condition network. The base network acts like an MLP that processes each pixel independently and the condition network extracts the global features of the input image to generate a condition vector. To realize retouching operations, we modulate the intermediate features using Global Feature Modulation (GFM), of which the parameters are transformed by condition vector. Benefiting from the utilization of 1 × 1 convolution, CSRNet only contains less than 37 k trainable parameters, which is orders of magnitude smaller than existing learning-based methods. Extensive experiments show that our method achieves state-of-the-art performance on the benchmark MIT-Adobe FiveK dataset quantitively and qualitatively. Code is available at https://***/hejingwenhejingwen/CSRNet. © 2020, Springer Nature Switzerland AG.

关键词： Pixels

来源：评论

学校读者我要写书评

暂无评论

Efficient Image Super-Resolution Using Pixel Attention 16th

Efficient Image Super-Resolution Using Pixel Attention

引用

Workshops held at the 16th European Conference on computer vision, ECCV 2020

作者： Zhao, Hengyuan Kong, Xiangtao He, Jingwen Qiao, Yu Dong, Chao ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Beijing China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China University of Chinese Academy of Sciences Beijing China

ISBN: (纸本)9783030670696

This work aims at designing a lightweight convolutional neural network for image super resolution (SR). With simplicity bare in mind, we construct a pretty concise and effective network with a newly proposed pixel attention scheme. Pixel attention (PA) is similar as channel attention and spatial attention in formulation. The difference is that PA produces 3D attention maps instead of a 1D attention vector or a 2D map. This attention scheme introduces fewer additional parameters but generates better SR results. On the basis of PA, we propose two building blocks for the main branch and the reconstruction branch, respectively. The first one—SC-PA block has the same structure as the Self-Calibrated convolution but with our PA layer. This block is much more efficient than conventional residual/dense blocks, for its two-branch architecture and attention scheme. While the second one—U-PA block combines the nearest-neighbor upsampling, convolution and PA layers. It improves the final reconstruction quality with little parameter cost. Our final model—PAN could achieve similar performance as the lightweight networks—SRResNet and CARN, but with only 272K parameters (17.92% of SRResNet and 17.09% of CARN). The effectiveness of each proposed component is also validated by ablation study. The code is available at https://***/zhaohengyuan1/PAN. © 2020, Springer Nature Switzerland AG.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

An Experimental Study of keypoint Descriptor Fusion

An Experimental Study of Keypoint Descriptor Fusion

引用

IEEE International Conference on robotics and Biomimetics

作者： Yaling Pan Li He Yisheng Guan Hong Zhang Biomimetic Intelligent Robotics Research Laboratory (BIRL) Guangdong University of Technology Guangzhou China Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology Shenzhen China

ISBN: (纸本)9781665481106

Local feature descriptors play a crucial role in computer vision problems, especially robot motion. Existing descriptors are highly accurate, but their performance de-pends on the influence of distracting factors, such as illumi-nation and viewpoint. There is room for further improvement of these descriptors. In this paper, we provide an in-depth analysis of several exciting features of the descriptor fusion model (DFM) we have proposed in our recent work, which uses an autoencoder to combine descriptors and exploit their respective advantages. With this DFM framework, we fur-ther validate that fused descriptors can retain advantageous properties and that our DFM is a generally applicable method with respect to various component descriptors. Specifically, we evaluate multiple combinations of hand-crafted and CNN descriptors concerning their performance on a benchmark dataset with illumination and viewpoint changes to obtain comprehensive experimental results. The results show that the fused descriptors have better matching accuracy than their component descriptors.

关键词： Robot motion computer vision Analytical models Biomimetics Lighting Benchmark testing

来源：评论

学校读者我要写书评

暂无评论

Person Re-Identification for Robot Person Following with Online Continual Learning

arXiv

引用

arXiv 2023年

作者： Ye, Hanjing Zhao, Jieting Zhan, Yu Chen, Weinan He, Li Zhang, Hong Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology China Guangdong University of Technology China

Robot person following (RPF) is a crucial capability in human-robot interaction (HRI) applications, allowing a robot to persistently follow a designated person. In practica RPF scenarios, the person can often be occluded by other objects or people. Consequently, it is necessary to re-identify the person when he/she reappears within the robot’s field of view Previous person re-identification (ReID) approaches to person following rely on a fixed feature extractor. Such an approach often fails to generalize to different viewpoints and lighting conditions in practical RPF environments. In other words, it suffers from the so-called domain shift problem where it cannot re-identify the person when his re-appearance is out of the domain modeled by the fixed feature extractor. To mitigate this problem, we propose a ReID framework for RPF where we use a feature extractor that is optimized online with both short-term and long-term experiences (i.e., recently and previously observed samples during RPF) using the online continua learning (OCL) framework. The long-term experiences are maintained by a memory manager to enable OCL to update the feature extractor. Our experiments demonstrate that even in the presence of severe appearance changes and distractions from visually similar people, the proposed method can still re-identify the person more accurately than the state-of-the-art methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Human robot interaction

来源：评论

学校读者我要写书评

暂无评论

Perspective Phase Angle Model for Polarimetric 3D Reconstruction

arXiv

引用

arXiv 2022年

作者： Chen, Guangcheng He, Li Guan, Yisheng Zhang, Hong Guangdong University of Technology Guangzhou China Southern University of Science and Technology Shenzhen China Shenzhen Key Laboratory of Robotics and Computer Vision China

Current polarimetric 3D reconstruction methods, including those in the well-established shape from polarization literature, are all developed under the orthographic projection assumption. In the case of a large field of view, however, this assumption does not hold and may result in significant reconstruction errors in methods that make this assumption. To address this problem, we present the perspective phase angle (PPA) model that is applicable to perspective cameras. Compared with the orthographic model, the proposed PPA model accurately describes the relationship between polarization phase angle and surface normal under perspective projection. In addition, the PPA model makes it possible to estimate surface normals from only one single-view phase angle map and does not suffer from the so-called π-ambiguity problem. Experiments on real data show that the PPA model is more accurate for surface normal estimation with a perspective camera than the orthographic model. Copyright © 2022, The Authors. All rights reserved.

关键词： Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Multi-Scale Point Octree Encoding Network for Point Cloud Based Place Recognition

Multi-Scale Point Octree Encoding Network for Point Cloud Ba...

引用

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Zhilong Tang Hanjing Ye Hong Zhang Department of Electrical and Electronic Engineering Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology (SUSTech) SUSTech Shenzhen China

Over the past decades, point cloud-based place recognition has garnered significant attention. This research paper presents a pioneering approach, denoted as the Multi-scale Point Octree Encoding Network (MPOE-Net), designed to acquire a discriminative global descriptor for efficient retrieval of places. The key element of the MPOE-Net is the point octree encoding module, which adeptly captures local information for each point by considering its nearest and farthest neighbors. Further enhancing local relationships, a multi-transformer network is introduced, utilizing a novel grouped offset-attention mechanism. To amalgamate the multi-scale attention maps into a comprehensive global descriptor, a multi-NetVLAD layer is incorporated. Through rigorous experimentation across diverse benchmark datasets, our proposed method unequivocally outperforms existing techniques in the realm of point cloud-based place recognition tasks, achieving state-of-the-art results. Our code is released publicly at https://***/Zhilong-Tang/MPOE-Net.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Task-Oriented Grasp Prediction with Visual-Language Inputs

arXiv

引用

arXiv 2023年

作者： Tang, Chao Huang, Dehao Meng, Lingxiao Liu, Weiyu Zhang, Hong Shenzhen Key Laboratory of Robotics and Computer Vision Southern University of Science and Technology Shenzhen China Institute for Robotics and Intelligent Machines Georgia Institute of Technology Atlanta United States

To perform household tasks, assistive robots receive commands in the form of user language instructions for tool manipulation. The initial stage involves selecting the intended tool (i.e., object grounding) and grasping it in a task-oriented manner (i.e., task grounding). Nevertheless, prior researches on visual-language grasping (VLG) focus on object grounding, while disregarding the fine-grained impact of tasks on object grasping. Task-incompatible grasping of a tool will inevitably limit the success of subsequent manipulation steps. Motivated by this problem, this paper proposes GraspCLIP, which addresses the challenge of task grounding in addition to object grounding to enable task-oriented grasp prediction with visual-language inputs. Evaluation on a custom dataset demonstrates that GraspCLIP achieves superior performance over established baselines with object grounding only. The effectiveness of the proposed method is further validated on an assistive robotic arm platform for grasping previously unseen kitchen tools given the task specification. Our presentation video is available at: https://***/watch?v=e1wfYQPeAXU. Copyright © 2023, The Authors. All rights reserved.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：