检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Zhang, Yiming Xu, Xuenan Du, Ruoyi Liu, Haohe Dong, Yuan Tan, Zheng-Hua Wang, Wenwu Ma, Zhanyu The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China The Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai200240 China The Department of Electronic Systems Aalborg University Aalborg9220 Denmark The Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

In traditional audio captioning methods, a model is usually trained in a fully supervised manner using a human-annotated dataset containing audio-text pairs and then evaluated on the test sets from the same dataset. Such methods have two limitations. First, these methods are often data-hungry and require time-consuming and expensive human annotations to obtain audio-text pairs. Second, these models often suffer from performance degradation in cross-domain scenarios, i.e., when the input audio comes from a different domain than the training set, which, however, has received little attention. We propose an effective audio captioning method based on the contrastive language-audio pre-training (CLAP) model to address these issues. Our proposed method requires only textual data for training, enabling the model to generate text from the textual feature in the cross-modal semantic space. In the inference stage, the model generates the descriptive text for the given audio from the audio feature by leveraging the audio-text alignment from CLAP. We devise two strategies to mitigate the discrepancy between text and audio embeddings: a mixed-augmentation-based soft prompt and a retrieval-based acoustic-aware hard prompt. These approaches are designed to enhance the generalization performance of our proposed model, facilitating the model to generate captions more robustly and accurately. Extensive experiments on AudioCaps and Clotho benchmarks show the effectiveness of our proposed method, which outperforms other zero-shot audio captioning approaches for in-domain scenarios and outperforms the compared methods for cross-domain scenarios, underscoring the generalization ability of our method. © 2024, CC BY.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Stable and Feature Decoupled of Deep Mutual Information Maximization Based on Wasserstein Distance

SSRN

引用

SSRN 2023年

作者： He, Xing Peng, Changgen Wang, Lin Tan, Weijie State Key Laboratory of Public Big Data College of Computer Science and Technology Guizhou University PR Guiyang550025 China Guizhou Key Laboratory of Pattern Recognition and Intelligent System Guizhou Minzu University PR Guiyang550025 China Guizhou Big Data Academy Guizhou University PR Guiyang550025 China Key Laboratory of Advanced Manufacturing Technology Ministry of Education Guizhou University PR Guiyang550025 China

Deep learning techniques usually require large amounts of expensive labeled data to train networks, and the extracted deep representations are usually mixed with multiple attributes having uninterpretability, which limits the application and development of deep learning techniques. Therefore, it is crucial to investigate the stability of training models by unsupervised learning methods and to learn feature decoupled representations. Although the method based on depth mutual information maximization(Deep InfoMax or DIM) has achieved good results in the field of computer vision, it is extremely prone to "gradient explosion" or "gradient disappearance", which makes the training model unstable. At the same time, it is difficult to decouple the features learned by the encoder. In order to solve these problems, we stable and feature decoupled of deep mutual information maximization based on Wasserstein distance, which we call the method called WDIM. The method first addresses the instability of the model trained by the DIM method and proposes to measure the distance between the encoder output and the prior distribution based on the Wasserstein distance, which is a good guide to the stability of the model training. Second, to address the problem that the features learned by the encoder are difficult to be decoupled, the mutual information between the features learned by the hidden layer of the encoder and the intermediate layer is required to be minimized during the training of the encoder so that the features learned by each filter are as uncorrelated as possible to achieve feature decoupling. Finally, the WDIM method is validated on the CIFAR-10, CIFAR-100, STL-10, and FashionMNIST datasets. Experiments show that our proposed WDIM method is more stable for model training, faster model convergence, and more significant feature decoupling. © 2023, The Authors. All rights reserved.

关键词： Network coding

来源：评论

学校读者我要写书评

暂无评论

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

arXiv

引用

arXiv 2023年

作者： Li, Wenjie Wang, Mei Zhang, Kai Li, Juncheng Li, Xiaoming Zhang, Yuhang Gao, Guangwei Deng, Weihong Lin, Chia-Wen The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing China The Computer Vision Lab ETH Zürich Zürich Switzerland The School of Communication and Information Engineering Shanghai University Shanghai China The Nanyang Technological University Singapore The Intelligent Visual Information Perception Laboratory Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Department of Electrical Engineering National Tsing Hua University Hsinchu Taiwan

Face restoration (FR) is a specialized field within image restoration that aims to recover low-quality (LQ) face images into high-quality (HQ) face images. Recent advances in deep learning technology have led to significant progress in FR methods. In this paper, we begin by examining the prevalent factors responsible for real-world LQ images and introduce degradation techniques used to synthesize LQ images. We also discuss notable benchmarks commonly utilized in the field. Next, we categorize FR methods based on different tasks and explain their evolution over time. Furthermore, we explore the various facial priors commonly utilized in the restoration process and discuss strategies to enhance their effectiveness. In the experimental section, we thoroughly evaluate the performance of state-of-the-art FR methods across various tasks using a unified benchmark. We analyze their performance from different perspectives. Finally, we discuss the challenges faced in the field of FR and propose potential directions for future advancements. The open-source repository corresponding to this work can be found at https://***/24wenjie-li/Awesome-Face-Restoration. Copyright © 2023, The Authors. All rights reserved.

关键词： Restoration

来源：评论

学校读者我要写书评

暂无评论

Clue me in: Semi-supervised FGVC with out-of-distribution data

arXiv

引用

arXiv 2021年

作者： Du, Ruoyi Chang, Dongliang Ma, Zhanyu Song, Yi-Zhe Guo, Jun The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China SketchX CVSSP University of Surrey London United Kingdom

Despite great strides made on fine-grained visual classification (FGVC), current methods are still heavily reliant on fully-supervised paradigms where ample expert labels are called for. Semi-supervised learning (SSL) techniques, acquiring knowledge from unlabeled data, provide a considerable means forward and have shown great promise for coarse-grained problems. However, exiting SSL paradigms mostly assume in-distribution (i.e., category-aligned) unlabeled data, which hinders their effectiveness when re-proposed on FGVC. In this paper, we put forward a novel design specifically aimed at making out-of-distribution data work for semi-supervised FGVC, i.e., to "clue them in". We work off an important assumption that all fine-grained categories naturally follow a hierarchical structure (e.g., the phylogenetic tree of "Aves" that covers all bird species). It follows that, instead of operating on individual samples, we can instead predict sample relations within this tree structure as the optimization goal of SSL. Beyond this, we further introduced two strategies uniquely brought by these tree structures to achieve inter-sample consistency regularization and reliable pseudo-relation. Our experimental results reveal that (i) the proposed method yields good robustness against out-of-distribution data, and (ii) it can be equipped with prior arts, boosting their performance thus yielding state-of-the-art results. Code is available at https://***/PRIS-CV/RelMatch. © 2021, CC BY.

关键词： Trees (mathematics)

来源：评论

学校读者我要写书评

暂无评论

Isoform Function Prediction Based on Heterogeneous Graph Attention Networks

Isoform Function Prediction Based on Heterogeneous Graph Att...

引用

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Kuo Guo Yifan Li Hao Chen Hong-Bin Shen Yang Yang Department of Computer Science and Engineering Shanghai Jiao Tong University and Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University and Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai China Computational Biology Department School of Computer Science Carnegie Mellon University Pittsburgh PA USA

Isoforms refer to different mRNA molecules transcribed from the same gene, which can be translated into proteins with varying structures and functions. Predicting the functions of isoforms is an essential topic in bioinformatics as it can provide valuable insights into the intricate mechanisms of gene regulation and biological processes. Conventionally, gene function labels are standardized in Gene Ontology (GO) terms. However, traditional methods for predicting isoform function are largely limited by the absence of isoform-specific labels, sparse annotations, and the vast number of GO terms. To address these issues, we propose HANIso, a deep learning-based method for isoform function prediction. HANIso leverages a pretrained protein language model to extract features from protein sequences. It also integrates heterogeneous information, such as isoform sequence features, GO annotations, and isoform interaction data, using a Heterogeneous Graph Attention Network (HAN). This allows the model to learn the importance of different sources of information and their semantic relationships through the attention mechanism. Our method can predict function labels at both the gene level and isoform level. We conduct experiments on two species datasets, and the results demonstrate that our method outperforms existing methods on both AUROC and AUPRC. HANIso has the potential to overcome the limitations of traditional methods and provide a more accurate and comprehensive understanding of isoform function.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Finger in Camera Speaks Everything: Unconstrained Air-Writing for Real-World

arXiv

引用

arXiv 2024年

作者： Wu, Meiqi Huang, Kaiqi Cai, Yuanqiang Hu, Shiyu Zhao, Yuzhong Wang, Weiqiang School of Computer Science and Technology University of Chinese Academy of Sciences Beijing100049 China Beijing University of Posts and Telecommunications Beijing100876 China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing100049 China Center for Research on Intelligent System and Engineering Institute of Automation Chinese Academy of Sciences Beijing100190 China Center for Research on Intelligent System and Engineering National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China CAS Center for Excellence in Brain Science and Intelligence Technology Shanghai200031 China

Air-writing is a challenging task that combines the fields of computer vision and natural language processing, offering an intuitive and natural approach for human-computer interaction. However, current air-writing solutions face two primary challenges: (1) their dependency on complex sensors (e.g., Radar, EEGs and others) for capturing precise handwritten trajectories, and (2) the absence of a video-based air-writing dataset that covers a comprehensive vocabulary range. These limitations impede their practicality in various real-world scenarios, including the use on devices like iPhones and laptops. To tackle these challenges, we present the groundbreaking air-writing Chinese character video dataset (AWCV-100K-UCAS2024), serving as a pioneering benchmark for video-based air-writing. This dataset captures handwritten trajectories in various real-world scenarios using commonly accessible RGB cameras, eliminating the need for complex sensors. AWCV-100K-UCAS2024 includes 8.8 million video frames, encompassing the complete set of 3,755 characters from the GB2312-80 level-1 set (GB1). Furthermore, we introduce our baseline approach, the video-based character recognizer (VCRec). VCRec adeptly extracts fingertip features from sparse visual cues and employs a spatio-temporal sequence module for analysis. Experimental results showcase the superior performance of VCRec compared to existing models in recognizing air-written characters, both quantitatively and qualitatively. This breakthrough paves the way for enhanced human-computer interaction in real-world contexts. Moreover, our approach leverages affordable RGB cameras, enabling its applicability in a diverse range of scenarios. The code and data examples will be made public at https://***/wmeiqi/AWCV. Copyright © 2024, The Authors. All rights reserved.

关键词： Natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

Graph Convolution Based Cross-Network Multi-Scale Feature Fusion for Deep Vessel Segmentation

arXiv

引用

arXiv 2023年

作者： Zhao, Gangming Liang, Kongming Pan, Chengwei Zhang, Fandong Wu, Xianpeng Hu, Xinyang Yu, Yizhou The Department of Computer Science The University of Hong Kong Hong Kong Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing China Institute of Artificial Intelligence Beihang University Beijing China The AI Lab Deepwise Healthcare Beijing China Department of Cardiology of the Second Affiliated Hospital School of Medicine Zhejiang University Hangzhou China Key Laboratory of Cardiovascular of Zhejiang Province Hangzhou China

Vessel segmentation is widely used to help with vascular disease diagnosis. Vessels reconstructed using existing methods are often not sufficiently accurate to meet clinical use standards. This is because 3D vessel structures are highly complicated and exhibit unique characteristics, including sparsity and anisotropy. In this paper, we propose a novel hybrid deep neural network for vessel segmentation. Our network consists of two cascaded subnetworks performing initial and refined segmentation respectively. The second subnetwork further has two tightly coupled components, a traditional CNN-based U-Net and a graph U-Net. Cross-network multi-scale feature fusion is performed between these two U-shaped networks to effectively support high-quality vessel segmentation. The entire cascaded network can be trained from end to end. The graph in the second subnetwork is constructed according to a vessel probability map as well as appearance and semantic similarities in the original CT volume. To tackle the challenges caused by the sparsity and anisotropy of vessels, a higher percentage of graph nodes are distributed in areas that potentially contain vessels while a higher percentage of edges follow the orientation of potential nearby vessels. Extensive experiments demonstrate our deep network achieves state-of-the-art 3D vessel segmentation performance on multiple public and in-house datasets. Copyright © 2023, The Authors. All rights reserved.

关键词： Anisotropy

来源：评论

学校读者我要写书评

暂无评论

Natural Image Matting Based on Surrogate Model

SSRN

引用

SSRN 2022年

作者： Liang, Yihui Gou, Hongshan Feng, Fujian Liu, Guisong Huang, Han School of Computer Science Zhongshan Institute University of Electronic Science and Technology of China Zhongshan528400 China Guizhou Key Laboratory of Pattern Recognition and Intelligent System Guizhou Minzu University Guiyang550025 China School of Economic Information Engineering Southwestern University of Finance and Economics Chengdu610000 China Laboratory of Intelligent Algorithms and Intelligent Software School of Software Engineering South China University of Technology Guangzhou510006 China

Image matting is important in digital image processing. The pixel pair optimization-based methods are some of the image matting methods that have distinct advantages in spatially disconnected foreground. However, they cannot provide high-quality alpha mattes with limited computing resources which limits their computing resource-sensitive applications. To address this problem, we present a natural image matting method based on a surrogate model. The surrogate model for pixel pair optimization is established in the presented method to approximate pixel pair fitness, and its analytical optimal solution is used as the approximate optimal solution of pixel pair optimization problem, which saves a lot of computing resources. Experimental results show that the image matting method based on a surrogate model provides high-quality matting mattes with low computing resources and its image matting performance is competitive against state-of-the-art pixel pair optimization-based methods which require a lot of computational resources. © 2022, The Authors. All rights reserved.

关键词： Pixels

来源：评论

学校读者我要写书评

暂无评论

SliceProp: A Slice-Wise Bidirectional Propagation Model for Interactive 3D Medical Image Segmentation

SliceProp: A Slice-Wise Bidirectional Propagation Model for ...

引用

Medical Artificial Intelligence (MedAI), IEEE International Conference on

作者： Xin Xu Wenjing Lu Jiahao Lei Peng Qiu Hong-Bin Shen Yang Yang Department of Computer Science and Engineering Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China Department of Vascular Surgery Shanghai Ninth People's Hospital Shanghai Jiao Tong University School of Medicine Institute of Image Processing and Pattern Recognition and Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai Jiao Tong University Shanghai China

Interactive medical image segmentation methods have become increasingly popular in recent years. These methods combine manual labeling and automatic segmentation, reducing the workload of annotation while maintaining high accuracy. However, most current interactive segmentation frameworks are limited to 2D image data, and are not suitable for 3D image data due to the large size and high complexity of 3D data, as well as the challenges posed by information asymmetry and sparse annotation. In this paper, we propose SliceProp, an interactive segmentation framework that implements slice-wise Label Bidirectional Propagation (LBP) for 3D medical image segmentation. SliceProp extends the interactive 2D image segmentation algorithm to 3D image segmentation, and can handle 3D data with large size and high complexity. Moreover, equipped with a Backtracking Feedback Check (BFC) module, SliceProp effectively addresses the issues of information asymmetry and spatial sparse annotation in 3D medical image segmentation. Additionally, we adopt an uncertainty-based criterion to pri-oritize the slices to be refined interactively, which enhances the efficiency of the interaction process by enabling the model to focus on the regions with the most unreliable predictions. SliceProp is evaluated on two datasets and achieves promising results compared to state-of-the-art methods.

关键词：

来源：评论

学校读者我要写书评

暂无评论

IU-Module: Intersection and Union Module for Fine-Grained Visual Classification

IU-Module: Intersection and Union Module for Fine-Grained Vi...

引用

IEEE International Conference on Multimedia and Expo (ICME)

作者： Yixiao Zheng Dongliang Chang Jiyang Xie Zhanyu Ma Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications

ISBN: (数字)9781728113319

ISBN: (纸本)9781728113326

A predominant viewpoint in previous works of fine-grained visual classification (FGVC) is to the localize discriminative parts by auxiliary networks and extract the part-based finegrained features for classification. In this paper, we propose a simple yet effective approach by introducing an intersection and union module (IU-Module). The IU-Module aims to capture more discriminative features by 1) dividing features into distinct groups, 2) sharing parts of interests within each group, and 3) adding a differentiation loss to reduce the similarity among those grouped feature channels. Without adding any new learnable parameters, the proposed approach imposes two straightforward operations, namely channel intersection (CI) and channel union (CU) operations, on the convolutional features and achieves competitive results compared with the state-of-the-art methods. Experimental results on three publicly available FGVC datasets show the effectiveness of the IU-Module. Ablation studies and visualizations are also provided to make further demonstrations.

关键词： Feature extraction Visualization Task analysis Automobiles Semantics Toy manufacturing industry Atmospheric modeling

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：