检索结果-内蒙古大学图书馆

NavCoT: Boosting LLM-Based vision-and-Language Navigation via Learning Disentangled Reasoning

IEEE Transactions on Pattern Analysis and Machine intelligence 2025年第7期47卷 5945-5957页

作者： Bingqian Lin Yunshuang Nie Ziming Wei Jiaqi Chen Shikui Ma Jianhua Han Hang Xu Xiaojun Chang Xiaodan Liang Shenzhen Campus Sun Yat-sen University Shenzhen China Shanghai Jiao Tong University Shanghai China University of Hong Kong Pok Fu Lam Hong Kong Dataa Robotics Company Beijing China Huawei Noah's Ark Lab Shanghai China School of Information Science and Technology University of Science and Technology of China Hefei China Department of Computer Vision Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) Abu Dhabi UAE Peng Cheng Laboratory Shenzhen China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China

vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions. Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability. However, their predominant use in an offline manner usually suffers from substantial domain gap between the VLN task and the LLM training corpus. This paper proposes a novel strategy called Nav igational C hain- o f- T hought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision, leading to a significant mitigation of the domain gap in a cost-effective manner. Specifically, at each timestep, the LLM is prompted to forecast the navigational chain-of-thought by: 1) acting as a world model to imagine the next observation according to the instruction, 2) selecting the candidate observation that best aligns with the imagination, and 3) determining the action based on the reasoning from the prior steps. In this way, the action prediction can be effectively simplified benefiting from the disentangled reasoning. Through constructing formalized labels for training, the LLM can learn to generate desired and reasonable chain-of-thought outputs for improving the action decision. Experimental results across various training settings and popular VLN benchmarks (e.g., Room-to-Room (R2R), Room-across-Room (RxR), Room-for-Room (R4R)) show the significant superiority of NavCoT over the direct action prediction variants. Through simple parameter-efficient finetuning, our NavCoT outperforms a recent GPT4-based approach with $\sim$ 7% relative improvement on the R2R dataset. We believe that NavCoT will help unlock more task-adaptive and scalable LLM-based embodied agents, which are helpful for developing real-world robotics applications.

关键词： Navigation Cognition Training Scalability Glass Planning Large language models History Electronic mail Visualization

来源：评论

学校读者我要写书评

暂无评论

MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation

arXiv

引用

arXiv 2025年

作者： Ding, Yanbo Hu, Xirui Guo, Zhizhi Wang, Yali Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China China Telecom China School of Computer Science and Technology Xi’an Jiaotong University China School of Artificial Intelligence University of Chinese Academy of Sciences China Shanghai Artificial Intelligence Laboratory China

Human image animation has gained increasing attention and developed rapidly due to its broad applications in digital humans. However, existing methods rely largely on 2D-rendered pose images for motion guidance, which limits generalization and discards essential 3D information for open-world animation. To tackle this problem, we propose MTVCrafter (Motion Tokenization Video Crafter), the first framework that directly models raw 3D motion sequences (i.e., 4D motion) for human image animation. Specifically, we introduce 4DMoT (4D motion tokenizer) to quantize 3D motion sequences into 4D motion tokens. Compared to 2D-rendered pose images, 4D motion tokens offer more robust spatio-temporal cues and avoid strict pixel-level alignment between pose image and character, enabling more flexible and disentangled control. Then, we introduce MV-DiT (Motion-aware Video DiT). By designing unique motion attention with 4D positional encodings, MV-DiT can effectively leverage motion tokens as 4D compact yet expressive context for human image animation in the complex 3D world. Hence, it marks a significant step forward in this field and opens a new direction for pose-guided human video generation. Experiments show that our MTVCrafter achieves state-of-the-art results with an FID-VID of 6.98, surpassing the second-best by 65%. Powered by robust motion tokens, MTVCrafter also generalizes well to diverse open-world characters (single/multiple, full/half-body) across various styles and scenarios. Our video demos and code are on: https://***/DINGYANB/MTVCrafter. Copyright © 2025, The Authors. All rights reserved.

关键词： Three dimensional computer graphics

来源：评论

学校读者我要写书评

暂无评论

Sad: Saliency-based defenses against adversarial examples

arXiv

引用

arXiv 2020年

作者： Tran, Richard Patrick, David Geyer, Michael Fernandez, Amanda S. Vision and Artificial Intelligence Lab Department of Computer Science University of Texas at San Antonio

With the rise in popularity of machine and deep learning models, there is an increased focus on their vulnerability to malicious inputs. These adversarial examples drift model predictions away from the original intent of the network and are a growing concern in practical security. In order to combat these attacks, neural networks can leverage traditional image processing approaches or state-of-the-art defensive models to reduce perturbations in the data. Defensive approaches that take a global approach to noise reduction are effective against adversarial attacks, however their lossy approach often distorts important data within the image. In this work, we propose a visual saliency based approach to cleaning data affected by an adversarial attack. Our model leverages the salient regions of an adversarial image in order to provide a targeted countermeasure while comparatively reducing loss within the cleaned images. We measure the accuracy of our model by evaluating the effectiveness of state-of-the-art saliency methods prior to attack, under attack, and after application of cleaning methods. We demonstrate the effectiveness of our proposed approach in comparison with related defenses and against established adversarial attack methods, across two saliency datasets. Our targeted approach shows significant improvements in a range of standard statistical and distance saliency metrics, in comparison with both traditional and state-of-the-art approaches. Copyright © 2020, The Authors. All rights reserved.

关键词： Noise abatement

来源：评论

学校读者我要写书评

暂无评论

GeneSegNet: a deep learning framework for cell segmentation by integrating gene expression and imaging

引用

Genome Biology 2023年第1期24卷 235-235页

作者： Wang, Yuxing Wang, Wenguan Liu, Dongfang Hou, Wenpin Zhou, Tianfei Ji, Zhicheng Department of Computer Engineering Rochester Institute of Technology Rochester United States Department of Biostatistics and Bioinformatics Duke University School of Medicine Durham United States The ReLER Lab from the Australian Artificial Intelligence Institute (AAII) University of Technology Sydney Sydney Australia Department of Biostatistics Columbia University Mailman School of Public Health New York City United States Computer Vision Lab ETH Zurich Zurich Switzerland

When analyzing data from in situ RNA detection technologies, cell segmentation is an essential step in identifying cell boundaries, assigning RNA reads to cells, and studying the gene expression and morphological features of cells. We developed a deep-learning-based method, GeneSegNet, that integrates both gene expression and imaging information to perform cell segmentation. GeneSegNet also employs a recursive training strategy to deal with noisy training labels. We show that GeneSegNet significantly improves cell segmentation performances over existing methods that either ignore gene expression information or underutilize imaging information. © 2023, BioMed Central Ltd., part of Springer Nature.

关键词： Cell segmentation Deep learning In situ hybridization Spatial transcriptomics

来源：评论

学校读者我要写书评

暂无评论

Interactive Multi-dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration 1

引用

16th European Conference on computer vision, ECCV 2020

作者： He, Jingwen Dong, Chao Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Beijing China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China

ISBN: (数字)9783030585655

ISBN: (纸本)9783030585648

Interactive image restoration aims to generate restored images by adjusting a controlling coefficient which determines the restoration level. Previous works are restricted in modulating image with a single coefficient. However, real images always contain multiple types of degradation, which cannot be well determined by one coefficient. To make a step forward, this paper presents a new problem setup, called multi-dimension (MD) modulation, which aims at modulating output effects across multiple degradation types and levels. Compared with the previous single-dimension (SD) modulation, the MD is setup to handle multiple degradations adaptively and relief unbalanced learning problem in different degradations. We also propose a deep architecture - CResMD with newly introduced controllable residual connections for multi-dimension modulation. Specifically, we add a controlling variable on the conventional residual connection to allow a weighted summation of input and residual. The values of these weights are generated by another condition network. We further propose a new data sampling strategy based on beta distribution to balance different degradation types and levels. With corrupted image and degradation information as inputs, the network can output the corresponding restored image. By tweaking the condition vector, users can control the output effects in MD space at test time. Extensive experiments demonstrate that the proposed CResMD achieve excellent performance on both SD and MD modulation tasks. Code is available at https://***/hejingwenhejingwen/CResMD. © 2020, Springer Nature Switzerland AG.

关键词： Modulation

来源：评论

学校读者我要写书评

暂无评论

LocalViT: Analyzing Locality in vision Transformers

LocalViT: Analyzing Locality in Vision Transformers

引用

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Yawei Li Kai Zhang Jiezhang Cao Radu Timofte Michele Magno Luca Benini Luc Van Goo Computer Vision Lab D-ITET ETH Zurich Switzerland Center for Artificial Intelligence and Data Science (CAIDAS) University of Wurzburg Germany Center for Project-Based Learning D-ITET ETH Zurich Switzerland Integrated Systems Laboratory D-ITET ETH Zurich Switzerland Department of Electrical Electronic and Information Engineering University of Bologna Italy Processing Speech and Images (PSI) KU Leuven Belgium

The aim of this paper is to study the influence of locality mechanisms in vision transformers. Transformers originated from machine translation and are particularly good at modelling long-range dependencies within a long sequence. Although the global interaction between the token embeddings could be well modelled by the self-attention mechanism of transformers, what is lacking is a locality mechanism for infor-mation exchange within a local region. In this paper, locality mechanism is systematically investigated by carefully designed controlled experiments. We add locality to vision transformers into the feed-forward network. This seemingly simple solution is inspired by the comparison between feed-forward networks and inverted residual blocks. The importance of locality mechanisms is validated in two ways: 1) A wide range of design choices (activation function, layer placement, expansion ratio) are available for incorporating locality mechanisms and proper choices can lead to a performance gain over the baseline, and 2) The same locality mechanism is successfully applied to vision transformers with different architecture designs, which shows the generalization of the locality concept. For ImageNet2012 classification, the locality-enhanced transformers outperform the baselines Swin-T [1], DeiT-T [2] and PVT-T [3] by 1.0%, 2.6 % and 3.1 % with a negligible increase in the number of parameters and computational effort. Code is available at https://***/ofsoundof/LocalViT.

关键词：

来源：评论

学校读者我要写书评

暂无评论

NTIRE 2023 Image Shadow Removal Challenge Report

NTIRE 2023 Image Shadow Removal Challenge Report

引用

2023 IEEE/CVF Conference on computer vision and Pattern Recognition Workshops, CVPRW 2023

作者： Vasluianu, Florin-Alexandru Seizinger, Tim Timofte, Radu Cui, Shuhao Huang, Junshi Tian, Shuman Fan, Mingyuan Zhang, Jiaqi Zhu, Li Wei, Xiaoming Wei, Xiaolin Luo, Ziwei Gustafsson, Fredrik K. Zhao, Zheng Sjölund, Jens Schön, Thomas B. Dong, Xiaoyi Zhang, Xi Sheryl Li, Chenghua Leng, Cong Yeo, Woon-Ha Oh, Wang-Taek Lee, Yeo-Reum Ryu, Han-Cheol Luo, Jinting Jiang, Chengzhi Han, Mingyan Wu, Qi Lin, Wenjie Yu, Lei Li, Xinpeng Jiang, Ting Fan, Haoqiang Liu, Shuaicheng Xu, Shuning Song, Binbin Chen, Xiangyu Zhang, Shile Zhou, Jiantao Zhang, Zhao Zhao, Suiyi Zheng, Huan Gao, Yangcheng Wei, Yanyan Wang, Bo Ren, Jiahuan Luo, Yan Kondo, Yuki Miyata, Riku Yasue, Fuma Naruki, Taito Ukita, Norimichi Chang, Hua-En Yang, Hao-Hsiang Chen, Yi-Chung Chiang, Yuan-Chun Huang, Zhi-Kai Chen, Wei-Ting Chen, I-Hsiang Hsieh, Chia-Hsuan Kuo, Sy-Yen Xianwei, Li Fu, Huiyuan Liu, Chunlin Ma, Huadong Fu, Binglan He, Huiming Wang, Mengjia She, Wenxuan Liu, Yu Nathan, Sabari Kansal, Priya Zhang, Zhongjian Yang, Huabin Wang, Yan Zhang, Yanru Phutke, Shruti S. Kulkarni, Ashutosh Khan, Md Raqib Murala, Subrahmanyam Vipparthi, Santosh Kumar Ye, Heng Liu, Zixi Yang, Xingyi Liu, Songhua Wu, Yinwei Jing, Yongcheng Yu, Qianhao Zheng, Naishan Huang, Jie Long, Yuhang Yao, Mingde Zhao, Feng Zhao, Bowen Ye, Nan Shen, Ning Cao, Yanpeng Xiong, Tong Xia, Weiran Li, Dingwen Xia, Shuchen Computer Vision Lab Ifi Caidas University of Würzburg Germany Computer Vision Lab Eth Zürich Switzerland Meituan Group China Department of Information Technology Uppsala University Sweden Institute of Automation Chinese Academy of Sciences Beijing China Nanjing China Maicro Nanjing China Department of Artificial Intelligence Convergence Sahmyook University Seoul Korea Republic of Megvii Technology China University of Electronic Science and Technology of China China University of Macau China China Toyota Technological Institute Japan Graduate Institute of Electronics Engineering National Taiwan University Taiwan Department of Electrical Engineering National Taiwan University Taiwan Graduate Institute of Communication Engineering National Taiwan University Taiwan ServiceNow United States Beijing University of Post and Teleconmunication Beijing China Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education China Couger Inc. Computer Vision and Pattern Recognition Lab Indian Institute of Technology Ropar Punjab Rupnagar India Research Institute Singapore National University of Singapore Singapore Research Institute Singapore University of Sydney Australia Brain-Inspired Vision Laboratory Information Science and Technology Institution University of Science and Technology of China China State Key Laboratory of Fluid Power and Mechatronic Systems School of Mechanical Engineering Zhejiang University Hangzhou310027 China Key Laboratory of Advanced Manufacturing Technology of Zhejiang Province School of Mechanical Engineering Zhejiang University Hangzhou310027 China South China University of Technology China

ISBN: (纸本)9798350302493

This work reviews the results of the NTIRE 2023 Challenge on Image Shadow Removal. The described set of solutions were proposed for a novel dataset, which captures a wide range of object-light interactions. It consists of 1200 roughly pixel aligned pairs of real shadow free and shadow affected images, captured in a controlled environment. The data was captured in a white-box setup, using professional equipment for lights and data acquisition sensors. The challenge had a number of 144 participants registered, out of which 19 teams were compared in the final ranking. The proposed solutions extend the work on shadow removal, improving over the performance level describing state-of-the-art methods. © 2023 IEEE.

关键词： Data acquisition

来源：评论

学校读者我要写书评

暂无评论

A Model for Real-Time Hand Gesture Recognition Using Electromyography (EMG), Covariances and Feed-Forward artificial Neural Networks

A Model for Real-Time Hand Gesture Recognition Using Electro...

引用

IEEE ANDESCON

作者： Marco E. Benalcázar José González Andrés Jaramillo-Yánez Carlos E. Anchundia Patricio Zambrano Marco Segura Artificial Intelligence and Computer Vision Research Lab Escuela Politécnica Nacional Quito Ecuador

ISBN: (数字)9781728193656

ISBN: (纸本)9781728193663

Hand gesture recognition has many applications that require models to work in real time and with high recognition accuracy. The problem of hand gesture recognition involves identifying the time, duration and the class of a given movement of the hand. In this paper, a user-specific hand gesture recognition model is proposed. This model is based on electromyographies (EMGs) measured with the Myo Armband, covariances together with a bag of functions for feature extraction, and a shallow feed-forward neural network for classification. This model recognizes 5 gestures: fist, finger spread, wave in, wave out and double tap. The model is trained per user with 25 repetitions for each gesture to recognize. The model was designed, trained and tested using the data of 120 users. The recognition accuracy of this model is 92.45%, with a standard deviation of 11.00%, and an average time of processing of (40.58 ± 1.62)ms, which is less than the permitted delay 300ms for real-time gesture recognition models. Finally, for reproducing the results, the code and the database used for this paper are publicly available online.

关键词： Electromyography Gesture recognition Computational modeling Muscles Real-time systems Feature extraction Training

来源：评论

学校读者我要写书评

暂无评论

Cross Domain Object Detection by Target-Perceived Dual Branch Distillation

arXiv

引用

arXiv 2022年

作者： He, Mengzhe Wang, Yali Wu, Jiaxi Wang, Yiru Li, Hanqing Li, Bo Gan, Weihao Wu, Wei Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China SenseTime Research University of Chinese Academy of Science China Shanghai AI Laboratory Shanghai China Beihang University China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society China

Cross domain object detection is a realistic and challenging task in the wild. It suffers from performance degradation due to large shift of data distributions and lack of instance-level annotations in the target domain. Existing approaches mainly focus on either of these two difficulties, even though they are closely coupled in cross domain object detection. To solve this problem, we propose a novel Target-perceived Dual-branch Distillation (TDD) framework. By integrating detection branches of both source and target domains in a unified teacher-student learning scheme, it can reduce domain shift and generate reliable supervision effectively. In particular, we first introduce a distinct Target Proposal Perceiver between two domains. It can adaptively enhance source detector to perceive objects in a target image, by leveraging target proposal contexts from iterative cross-attention. Afterwards, we design a concise Dual Branch Self Distillation strategy for model training, which can progressively integrate complementary object knowledge from different domains via self-distillation in two branches. Finally, we conduct extensive experiments on a number of widely-used scenarios in cross domain object detection. The results show that our TDD significantly outperforms the state-of-the-art methods on all the benchmarks. Our code and model will be available at here. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

Mitigating Artifacts in Real-World Video Super-Resolution Models

arXiv

引用

arXiv 2022年

作者： Xie, Liangbin Wang, Xintao Shi, Shuwei Gu, Jinjin Dong, Chao Shan, Ying The Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China University of Macau China ARC Lab Tencent PCG China Shenzhen International Graduate School Tsinghua University China The University of Sydney Australia Shanghai Artificial Intelligence Laboratory China

The recurrent structure is a prevalent framework for the task of video super-resolution, which models the temporal dependency between frames via hidden states. When applied to real-world scenarios with unknown and complex degradations, hidden states tend to contain unpleasant artifacts and propagate them to restored frames. In this circumstance, our analyses show that such artifacts can be largely alleviated when the hidden state is replaced with a cleaner counterpart. Based on the observations, we propose a Hidden State Attention (HSA) module to mitigate artifacts in real-world video super-resolution. Specifically, we first adopt various cheap filters to produce a hidden state pool. For example, Gaussian blur filters are for smoothing artifacts while sharpening filters are for enhancing details. To aggregate a new hidden state that contains fewer artifacts from the hidden state pool, we devise a Selective Cross Attention (SCA) module, in which the attention between input features and each hidden state is calculated. Equipped with HSA, our proposed method, namely FastRealVSR, is able to achieve 2× speedup while obtaining better performance than Real-BasicVSR. Codes will be available at https://***/TencentARC/FastRealVSR. Copyright © 2022, The Authors. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：