检索结果-内蒙古大学图书馆

AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Han, Boyu Xu, Qianqian Yang, Zhiyong Bao, Shilong Wen, Peisong Jiang, Yangbangyan Huang, Qingming Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China School of Computer Science and Tech. University of Chinese Academy of Sciences China Peng Cheng Laboratory China Key Laboratory of Big Data Mining and Knowledge Management CAS China

The Area Under the ROC Curve (AUC) is a well-known metric for evaluating instance-level long-tail learning problems. In the past two decades, many AUC optimization methods have been proposed to improve model performance under long-tail distributions. In this paper, we explore AUC optimization methods in the context of pixel-level long-tail semantic segmentation, a much more complicated scenario. This task introduces two major challenges for AUC optimization techniques. On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis. On the other hand, we find that mini-batch estimation of AUC loss in this case requires a larger batch size, resulting in an unaffordable space complexity. To address these issues, we develop a pixel-level AUC loss function and conduct a dependency-graph-based theoretical analysis of the algorithm’s generalization ability. Additionally, we design a Tail-Classes Memory Bank (T-Memory Bank) to manage the significant memory demand. Finally, comprehensive experiments across various benchmarks confirm the effectiveness of our proposed AUCSeg method. The code is available at https://***/boyuh/AUCSeg. © 2024, CC BY.

关键词： Semantic Segmentation

CoSign: Exploring Co-occurrence Signals in Skeleton-based Continuous Sign Language Recognition

学校读者我要写书评

暂无评论

CoSign: Exploring Co-occurrence Signals in Skeleton-based Co...

International Conference on Computer Vision (ICCV)

作者： Peiqi Jiao Yuecong Min Yanan Li Xiaotao Wang Lei Lei Xilin Chen Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China Xiaomi Inc. China

The co-occurrence signals (e.g., hand shape, facial expression, and lip pattern) play a critical role in Continuous Sign Language Recognition (CSLR). Compared to RGB data, skeleton data provide a more efficient and concise option, and lay a good foundation for the co-occurrence exploration in CSLR. However, skeleton data are often used as a tool to assist visual grounding and have not attracted sufficient attention. In this paper, we propose a simple yet effective GCN-based approach, named CoSign, to incorporate Co-occurrence Signals and explore the potential of skeleton data in CSLR. Specifically, we propose a group-specific GCN to better exploit the knowledge of each signal and a complementary regularization to prevent complex co-adaptation across signals. Furthermore, we propose a two-stream framework that gradually fuses both static and dynamic information in skeleton data. Experimental results on three public CSLR datasets (PHOENIX14, PHOENIX14-T and CSL-Daily) show that the proposed CoSign achieves competitive performance with recent video-based approaches while reducing the computation cost during training.

关键词：

Stable Attribute Group Editing for Reliable Few-shot Image Generation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ding, Guanqi Han, Xinzhe Wang, Shuhui Wu, Shuzhe Jin, Xin Tu, Dandan Huang, Qingming The School of Computer Science and Technology University of Chinese Academy of Sciences Beijing101408 China The Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China Peng Cheng Laboratory Shenzhen518066 China Huawei Cloud EI Innovation Lab China

Few-shot image generation aims to generate data of an unseen category based on only a few samples. Apart from basic content generation, a bunch of downstream applications hopefully benefit from this task, such as low-data detection and few-shot classification. To achieve this goal, the generated images should guarantee category retention for classification beyond the visual quality and diversity. In our preliminary work, we present an "editing-based" framework Attribute Group Editing (AGE) for reliable few-shot image generation, which largely improves the performance compared with existing methods that require re-training a GAN with limited data. Nevertheless, AGE's performance on downstream classification is not as satisfactory as expected. This paper investigates the class inconsistency problem and proposes Stable Attribute Group Editing (SAGE) for more stable class-relevant image generation. Different from AGE which directly edits from a one-shot image, SAGE takes use of all given few-shot images and estimates a class center embedding based on the category-relevant attribute dictionary. Meanwhile, according to the projection weights on the category-relevant attribute dictionary, we can select category-irrelevant attributes from the similar seen categories. Consequently, SAGE injects the whole distribution of the novel class into StyleGAN's latent space, thus largely remains the category retention and stability of the generated images. Going one step further, we find that class inconsistency is a common problem in GAN-generated images for downstream classification. Even though the generated images look photo-realistic and requires no category-relevant editing, they are usually of limited help for downstream classification. We systematically discuss this issue from both the generative model and classification model perspectives, and propose to boost the downstream classification performance of SAGE by enhancing the pixel and frequency components. Extensive experime

关键词： Generative adversarial networks

Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutual information

学校读者我要写书评

暂无评论

Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutua...

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Jicun Li Xingjian Li Tianyang Wang Shi Wang Yanan Cao Chengzhong Xu Dejing Dou Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Big Data Lab Baidu Research State Key Lab of IOTSC University of Macau University of Alabama at Birmingham Institute of Information Engineering Chinese Academy of Sciences

Fine-tuning pre-trained language models, such as BERT, has shown enormous success among various NLP tasks. Though simple and effective, the process of fine-tuning has been found unstable, which often leads to unexpected poor performance. To increase stability and generalizability, most existing works resort to maintaining the parameters or representations of pre-trained models during fine-tuning. Nevertheless, very little work explores mining the reliable part of pre-learned information that can help to stabilize fine-tuning. To address this challenge, we introduce a novel solution in which we fine-tune BERT with stabilized cross-layer mutual information. Our method aims to preserve the reliable behaviors of cross-layer information propagation, instead of preserving the information itself, of the pre-trained model. Therefore, our method circumvents the domain conflicts between pre-trained and target tasks. We conduct extensive experiments with popular pre-trained BERT variants on NLP datasets, demonstrating the universal effectiveness and robustness of our method.

关键词： Cross layer design Bit error rate Signal processing Stability analysis Robustness Acoustics Behavioral sciences

Article reranking by memory-enhanced key sentence matching for detecting previously fact-checked claims 59

学校读者我要写书评

暂无评论

Article reranking by memory-enhanced key sentence matching f...

Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language processing, ACL-IJCNLP 2021

作者： Sheng, Qiang Cao, Juan Zhang, Xueyao Li, Xirong Zhong, Lei Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Key Lab of Data Engineering and Knowledge Engineering Renmin University of China China

ISBN: (纸本)9781954085527

False claims that have been previously fact-checked can still spread on social media. To mitigate their continual spread, detecting previously fact-checked claims is indispensable. Given a claim, existing works retrieve fact-checking articles (FC-articles) for detection and focus on reranking candidate articles in the typical two-stage retrieval framework. However, their performance may be limited as they ignore the following characteristics of FC-articles: (1) claims are often quoted to describe the checked events, providing lexical information besides semantics;and (2) sentence templates to introduce or debunk claims are common across articles, providing pattern information. In this paper, we propose a novel reranker, MTM (Memoryenhanced Transformers for Matching), to rank FC-articles using key sentences selected using event (lexical and semantic) and pattern information. For event information, we propose to finetune the Transformer with regression of ROUGE. For pattern information, we generate pattern vectors as a memory bank to match with the parts containing patterns. By fusing event and pattern information, we select key sentences to represent an article and then predict if the article fact-checks the given claim using the claim, key sentences, and patterns. Experiments on two real-world datasets show that MTM outperforms existing methods. Human evaluation proves that MTM can capture key sentences for explanations. The code and the dataset are at https://***/ICTMCG/MTM. © 2021 Association for Computational Linguistics

关键词： Semantics

Video Mobile-Former: Video Recognition with Efficient Global Spatial-temporal Modeling

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Rui Wu, Zuxuan Chen, Dongdong Chen, Yinpeng Dai, Xiyang Liu, Mengchen Zhou, Luowei Yuan, Lu Jiang, Yu-Gang Shanghai Key Lab of Intelligent Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Microsoft Cloud&AI China

Transformer-based models have achieved top performance on major video recognition benchmarks. Benefiting from the self-attention mechanism, these models show stronger ability of modeling long-range dependencies compared to CNN-based models. However, significant computation overheads, resulted from the quadratic complexity of self-attention on top of a tremendous number of tokens, limit the use of existing video transformers in applications with limited resources like mobile devices. In this paper, we extend Mobile-Former to Video Mobile-Former, which decouples the video architecture into a lightweight 3D-CNNs for local context modeling and a Transformer modules for global interaction modeling in a parallel fashion. To avoid significant computational cost incurred by computing self-attention between the large number of local patches in videos, we propose to use very few global tokens (e.g., 6) for a whole video in Transformers to exchange information with 3D-CNNs with a cross-attention mechanism. Through efficient global spatial-temporal modeling, Video Mobile-Former significantly improves the video recognition performance of alternative lightweight baselines, and outperforms other efficient CNN-based models at the low FLOP regime from 500M to 6G total FLOPs on various video recognition tasks. It is worth noting that Video Mobile-Former is the first Transformer-based video model which constrains the computational budget within 1G FLOPs. Copyright © 2022, The Authors. All rights reserved.

关键词： Benchmarking

KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation

学校读者我要写书评

暂无评论

KERM: Knowledge Enhanced Reasoning for Vision-and-Language N...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Xiangyang Li Zihan Wang Jiahao Yang Yaowei Wang Shuqiang Jiang Key Lab of Intelligent Information Processing Laboratory of the Chinese Academy of Sciences (CAS) Institute of Computing Technology Beijing China University of Chinese Academy of Sciences Beijing China Peng Cheng Laboratory Shenzhen China

Vision-and-language navigation (VLN) is the task to enable an embodied agent to navigate to a remote location following the natural language instruction in real scenes. Most of the previous approaches utilize the entire features or object-centric features to represent navigable candidates. However, these representations are not efficient enough for an agent to perform actions to arrive the target location. As knowledge provides crucial information which is complementary to visible content, in this paper, we propose a Knowledge Enhanced Reasoning Model (KERM) to leverage knowledge to improve agent navigation ability. Specifically, we first retrieve facts (i.e., knowledge described by language descriptions) for the navigation views based on local regions from the constructed knowledge base. The re-trieved facts range from properties of a single object (e.g., color, shape) to relationships between objects (e.g., action, spatial position), providing crucial information for VLN. We further present the KERM which contains the purification, fact-aware interaction, and instruction-guided aggregation modules to integrate visual, history, instruction, and fact features. The proposed KERM can automatically select and gather crucial and relevant cues, obtaining more accurate action prediction. Experimental results on the REVERIE, R2R, and SOON datasets demonstrate the effectiveness of the proposed method. The source code is available at https://***/XiangyangLi20/KERM.

关键词：

Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hu, Beizhe Sheng, Qiang Cao, Juan Zhu, Yongchun Wang, Danding Wang, Zhengjia Jin, Zhiwei Key Lab of Intelligent Information Processing of Chinese Academy of Sciences Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China ZhongKeRuijian Technology Co. Ltd China

Fake news detection has been a critical task for maintaining the health of the online news ecosystem. However, very few existing works consider the temporal shift issue caused by the rapidly-evolving nature of news data in practice, resulting in significant performance degradation when training on past data and testing on future data. In this paper, we observe that the appearances of news events on the same topic may display discernible patterns over time, and posit that such patterns can assist in selecting training instances that could make the model adapt better to future data. Specifically, we design an effective framework FTT (Forecasting Temporal Trends), which could forecast the temporal distribution patterns of news data and then guide the detector to fast adapt to future distribution. Experiments on the real-world temporally split dataset demonstrate the superiority of our proposed framework. The code is available at https://***/ICTMCG/FTTACL23. © 2023, CC BY-NC-SA.

关键词： Forecasting

Patch Is Not All You Need

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Changzhen Zhang, Jie Wei, Yang Ji, Zhilong Bai, Jinfeng Shan, Shiguang Key Lab of Intelligent Information Processing Chinese Academy of Sciences Institute of Computing Technology China University of Chinese Academy of Sciences China Hangzhou lnstitute for Advanced Study UCAS School of Intelligent Scienceand Technology China Tomorrow Advancing Life China

Vision Transformers have achieved great success in computer visions, delivering exceptional performance across various tasks. However, their inherent reliance on sequential input enforces the manual partitioning of images into patch sequences, which disrupts the image’s inherent structural and semantic continuity. To handle this, we propose a novel Pattern Transformer (Patternformer) to adaptively convert images to pattern sequences for Transformer input. Specifically, we employ the Convolutional Neural Network to extract various patterns from the input image, with each channel representing a unique pattern that is fed into the succeeding Transformer as a visual token. By enabling the network to optimize these patterns, each pattern concentrates on its local region of interest, thereby preserving its intrinsic structural and semantic information. Only employing the vanilla ResNet and Transformer, we have accomplished state-of-the-art performance on CIFAR-10 and CIFAR-100, and have achieved competitive results on ImageNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics