检索结果-内蒙古大学图书馆

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yong Li Yuanzhi Wang Zhen Cui PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing China

Human multimodal emotion recognition (MER) aims to perceive human emotions via language, visual and acoustic modalities. Despite the impressive performance of previous MER approaches, the inherent multimodal heterogeneities still haunt and the contribution of different modalities varies significantly. In this work, we mitigate this issue by proposing a decoupled multimodal distillation (DMD) approach that facilitates flexible and adaptive crossmodal knowledge distillation, aiming to enhance the discriminative features of each modality. Specially, the representation of each modality is decoupled into two parts, i.e., modality-irrelevant/-exclusive spaces, in a self-regression manner. DMD utilizes a graph distillation unit (GD-Unit) for each decoupled part so that each GD can be performed in a more specialized and effective manner. A GD-Unit consists of a dynamic graph where each vertice represents a modality and each edge indicates a dynamic knowledge distillation. Such GD paradigm provides a flexible knowledge transfer manner where the distillation weights can be automatically learned, thus enabling diverse crossmodal knowledge transfer patterns. Experimental results show DMD consistently obtains superior performance than state-of-the-art MER methods. Visualization results show the graph edges in DMD exhibit meaningful distributional patterns w.r.t. the modality-irrelevant/-exclusive feature spaces. Codes are re leased at https://***/mdswyz/DMD.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Understanding How Pretraining Regularizes Deep Learning Algorithms

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023年第9期34卷 5828-5840页

作者： Yao, Yu Yu, Baosheng Gong, Chen Liu, Tongliang Univ Sydney Sch Comp Sci Camperdown NSW 2006 Australia Nanjing Univ Sci & Technol Sch Comp Sci & Engn Minist Educ PCA Lab Nanjing 210094 Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Minist Educ Key Lab Intelligent Percept & Syst High Dimens In Nanjing 210094 Peoples R China

Deep learning algorithms have led to a series of breakthroughs in computer vision, acoustical signal processing, and others. However, they have only been popularized recently due to the groundbreaking techniques developed for training deep architectures. Understanding the training techniques is important if we want to further improve them. Through extensive experimentation, Erhan et al. (2010) empirically illustrated that unsupervised pretraining has an effect of regularization for deep learning algorithms. However, theoretical justifications for the observation remain elusive. In this article, we provide theoretical supports by analyzing how unsupervised pretraining regularizes deep learning algorithms. Specifically, we interpret deep learning algorithms as the traditional Tikhonov-regularized batch learning algorithms that simultaneously learn predictors in the input feature spaces and the parameters of the neural networks to produce the Tikhonov matrices. We prove that unsupervised pretraining helps in learning meaningful Tikhonov matrices, which will make the deep learning algorithms uniformly stable and the learned predictor will generalize fast w.r.t. the sample size. Unsupervised pretraining, therefore, can be interpreted as to have the function of regularization.

关键词： Deep learning Training Neural networks Prediction algorithms Signal processing algorithms Decoding Approximation algorithms Classification deep learning regularization representation learning unsupervised pretrain

来源：评论

学校读者我要写书评

暂无评论

Far-field super-resolution chemical microscopy

引用

Light(Science & Applications) 2023年第7期12卷 1246-1267页

作者： Mingwei Tang Yubing Han Danchen Jia Qing Yang Ji-Xin Cheng State Key Laboratory of Modern Optical Instrumentation College of Optical Science and EngineeringZhejiang UniversityHangzhou 310027China Intelligent Perception Research Institute Zhejiang LabHangzhou 311100China Department of Biomedical Engineering Department of Electrical and Computer EngineeringPhotonics CenterBoston UniversityBostonMA 02459USA

Far-feld chemical microscopy providing molecular electronic or vibrational fingerprint information opens a new window for the study of three-dimensional biological,material,and chemical *** microscopy provides a nondestructive way of chemical identification without exterior ***,the diffraction limit of optics hindered it from discovering more details under the resolution *** development of super-resolution techniques gives enlightenment to open this door behind far-field chemical ***,we review recent advances that have pushed the boundary of far-field chemical microscopy in terms of spatial *** further highlight applications in biomedical research,material characterization,environmental study,cultural heritage conservation,and integrated chip inspection.

关键词： chemical details optics

来源：评论

学校读者我要写书评

暂无评论

Decoupled Multimodal Distilling for Emotion Recognition

arXiv

引用

arXiv 2023年

作者： Li, Yong Wang, Yuanzhi Cui, Zhen PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information Ministry of Education School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing China

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

Divide and Conquer: Hybrid Pre-training for Person Search 38

Divide and Conquer: Hybrid Pre-training for Person Search

引用

38th AAAI Conference on Artificial Intelligence (AAAI) / 36th Conference on Innovative Applications of Artificial Intelligence / 14th Symposium on Educational Advances in Artificial Intelligence

作者： Tian, Yanling Chen, Di Liu, Yunan Yang, Jian Zhang, Shanshan Nanjing Univ Sci & Technol Sch Comp Sci & Engn Key Lab Intelligent Percept & Syst High Dimens In PCA LabMinist Educ Nanjing Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Socia Nanjing Peoples R China Dalian Maritime Univ Sch Artificial Intelligence Dalian Peoples R China

ISBN: (纸本)1577358872

Large-scale pre-training has proven to be an effective method for improving performance across different tasks. Current person search methods use ImageNet pre-trained models for feature extraction, yet it is not an optimal solution due to the gap between the pre-training task and person search task (as a downstream task). Therefore, in this paper, we focus on pretraining for person search, which involves detecting and reidentifying individuals simultaneously. Although labeled data for person search is scarce, datasets for two sub-tasks person detection and re-identification are relatively abundant. To this end, we propose a hybrid pre-training framework specifically designed for person search using sub-task data only. It consists of a hybrid learning paradigm that handles data with different kinds of supervisions, and an intra-task alignment module that alleviates domain discrepancy under limited resources. To the best of our knowledge, this is the first work that investigates how to support full-task pre-training using sub-task data. Extensive experiments demonstrate that our pre-trained model can achieve significant improvements across diverse protocols, such as person search method, fine-tuning data, pre-training data and model backbone. For example, our model improves ResNet50 based NAE by 10.3% relative improvement w.r.t. mAP. Our code and pre-trained models are released for plug-and-play usage to the person search community (https://***/personsearch/PretrainPS).

关键词： Artificial intelligence

来源：评论

学校读者我要写书评

暂无评论

Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking

arXiv

引用

arXiv 2024年

作者： Hu, Xiantao Tai, Ying Zhao, Xu Zhao, Chen Zhang, Zhenyu Li, Jun Zhong, Bineng Yang, Jian PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information Ministry of Education School of Computer Science and Engineering Nanjing University of Science and Technology China Nanjing University China Guangxi Normal University China

Multimodal tracking has garnered widespread attention as a result of its ability to effectively address the inherent limitations of traditional RGB tracking. However, existing multimodal trackers mainly focus on the fusion and enhancement of spatial features or merely leverage the sparse temporal relationships between video frames. These approaches do not fully exploit the temporal correlations in multimodal videos, making it difficult to capture the dynamic changes and motion information of targets in complex scenarios. To alleviate this problem, we propose a unified multimodal spatial-temporal tracking approach named STTrack. In contrast to previous paradigms that solely relied on updating reference information, we introduced a temporal state generator (TSG) that continuously generates a sequence of tokens containing multimodal temporal information. These temporal information tokens are used to guide the localization of the target in the next time state, establish long-range contextual relationships between video frames, and capture the temporal trajectory of the target. Furthermore, at the spatial level, we introduced the mamba fusion and background suppression interactive (BSI) modules. These modules establish a dual-stage mechanism for coordinating information interaction and fusion between modalities. Extensive comparisons on five benchmark datasets illustrate that STTrack achieves state-of-the-art performance across various multimodal tracking scenarios. Code is available at: https://***/NJU-pcalab/STTrack. © 2024, CC BY.

关键词： Video analysis

来源：评论

学校读者我要写书评

暂无评论

A Cylindrical Convolution Network for Dense Top-View Semantic Segmentation with LiDAR Point Clouds 1

引用

16th Asian Conference on Computer Vision (ACCV)

作者： Lu, Jiacheng Gu, Shuo Xu, Cheng-Zhong Kong, Hui Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Socia Nanjing Peoples R China Nanjing Univ Sci & Technol PCA Lab Key Lab Intelligent Percept & Syst High Dimens Minist Educ Nanjing Peoples R China Univ Macau Taipa Macao Peoples R China

ISBN: (数字)9783031262937

ISBN: (纸本)9783031262920;9783031262937

Accurate semantic scene understanding of the surrounding environment is a challenge for autonomous driving systems. Recent LiDAR-based semantic segmentation methods mainly focus on predicting point-wise semantic classes, which cannot be directly used before the further densification process. In this paper, we propose a cylindrical convolution network for dense semantic understanding in the top-view LiDAR data representation. 3D LiDAR point clouds are divided into cylindrical partitions before feeding to the network, where semantic segmentation is conducted in the cylindrical representation. Then a cylinder-to-BEV transformation module is introduced to obtain sparse semantic feature maps in the top view. In the end, we propose a modified encoder-decoder network to get the dense semantic estimations. Experimental results on the SemanticKITTI and nuScenes-LidarSeg datasets show that ourmethod outperforms the state-of-the-artmethods with a large margin.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

Semantic Contrastive Embedding for Generalized Zero-Shot Learning

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2022年第11期130卷 2606-2622页

作者： Han, Zongyan Fu, Zhenyong Chen, Shuo Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn Key Lab Intelligent Percept & Syst High Dimens In PCA LabMinist Educ Nanjing Peoples R China Nanjing Univ Sci & Technol Sch Comp Sci & Engn Jiangsu Key Lab Image & Video Understanding Socia Nanjing Peoples R China RIKEN Ctr Adv Intelligence Project Tokyo Japan

Generalized zero-shot learning (GZSL) aims to recognize objects from both seen and unseen classes when only the labeled examples from seen classes are provided. Recent feature generation methods learn a generative model that can synthesize the missing visual features of unseen classes to mitigate the data-imbalance problem in GZSL. However, the original visual feature space is suboptimal for GZSL recognition since it lacks semantic information, which is vital for recognizing the unseen classes. To tackle this issue, we propose to integrate the feature generation model with an embedding model. Our GZSL framework maps both the real and the synthetic samples produced by the generation model into an embedding space, where we perform the final GZSL classification. Specifically, we propose a semantic contrastive embedding (SCE) for our GZSL framework. Our SCE consists of attribute-level contrastive embedding and class-level contrastive embedding. They aim to obtain the transferable and discriminative information, respectively, in the embedding space. We evaluate our GZSL method with semantic contrastive embedding, named SCE-GZSL, on four benchmark datasets. The results show that our SCE-GZSL method can achieve the state-of-the-art or the second-best on these datasets.

关键词： Generalized zero-shot learning Attribute Semantic embedding Contrastive learning

来源：评论

学校读者我要写书评

暂无评论

RETHINKING BRANCHING ON EXACT COMBINATORIAL OPTIMIZATION SOLVER: THE FIRST DEEP SYMBOLIC DISCOVERY FRAMEWORK 12

RETHINKING BRANCHING ON EXACT COMBINATORIAL OPTIMIZATION SOL...

引用

12th International Conference on Learning Representations, ICLR 2024

作者： Kuang, Yufei Wang, Jie Liu, Haoyang Zhu, Fangzhou Li, Xijun Zeng, Jia Hao, Jianye Li, Bin Wu, Feng CAS Key Laboratory of Technology in GIPAS University of Science and Technology of China China MoE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition University of Science and Technology of China China Noah's Ark Lab Huawei Technologies China

Machine learning (ML) has been shown to successfully accelerate solving NP-hard combinatorial optimization (CO) problems under the branch and bound framework. However, the high training and inference cost and limited interpretability of ML approaches severely limit their wide application to modern exact CO solvers. In contrast, human-designed policies-though widely integrated in modern CO solvers due to their compactness and reliability-can not capture data-driven patterns for higher performance. To combine the advantages of the two paradigms, we propose the first symbolic discovery framework-namely, deep symbolic discovery for exact combinatorial optimization solver (Symb4CO)-to learn high-performance symbolic policies on the branching task. Specifically, we show the potential existence of small symbolic policies empirically, employ a large neural network to search in the high-dimensional discrete space, and compile the learned symbolic policies directly for fast deployment. Experiments show that the Symb4CO learned purely CPU-based policies consistently achieve comparable performance to previous GPU-based state-of-the-art approaches. Furthermore, the appealing features of Symb4CO include its high training (ten training instances) and inference (one CPU core) efficiency and good interpretability (one-line expressions), making it simple and reliable for deployment. The results show encouraging potential for the wide deployment of ML to modern solvers. Codes are available at https://***/MIRAlab-USTC/L2O-Symb4CO. © 2024 12th International Conference on Learning Representations, ICLR 2024. All rights reserved.

关键词： Combinatorial optimization

来源：评论

学校读者我要写书评

暂无评论

Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework 38

Towards Next-Generation Logic Synthesis: A Scalable Neural C...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Wang, Zhihai Wang, Jie Yang, Qingyue Bai, Yinqi Li, Xing Chen, Lei Hao, Jianye Yuan, Mingxuan Li, Bin Zhang, Yongdong Wu, Feng MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition University of Science and Technology of China China Noah's Ark Lab Huawei Technologies Canada College of Intelligence and Computing Tianjin University China

Logic Synthesis (LS) aims to generate an optimized logic circuit satisfying a given functionality, which generally consists of circuit translation and optimization. It is a challenging and fundamental combinatorial optimization problem in integrated circuit design. Traditional LS approaches rely on manually designed heuristics to tackle the LS task, while machine learning recently offers a promising approach towards next-generation logic synthesis by neural circuit generation and optimization. In this paper, we first revisit the application of differentiable neural architecture search (DNAS) methods to circuit generation and found from extensive experiments that existing DNAS methods struggle to exactly generate circuits, scale poorly to large circuits, and exhibit high sensitivity to hyper-parameters. Then we provide three major insights for these challenges from extensive empirical analysis: 1) DNAS tends to overfit to too many skip-connections, consequently wasting a significant portion of the network's expressive capabilities;2) DNAS suffers from the structure bias between the network architecture and the circuit inherent structure, leading to inefficient search;3) the learning difficulty of different input-output examples varies significantly, leading to severely imbalanced learning. To address these challenges in a systematic way, we propose a novel regularized triangle-shaped circuit network generation framework, which leverages our key insights for completely accurate and scalable circuit generation. Furthermore, we propose an evolutionary algorithm assisted by reinforcement learning agent restarting technique for efficient and effective neural circuit optimization. Extensive experiments on four different circuit benchmarks demonstrate that our method can precisely generate circuits with up to 1200 nodes. Moreover, our synthesized circuits significantly outperform the state-of-the-art results from several competitive winners in IWLS 2022 and 2023 competition

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：