检索结果-内蒙古大学图书馆

CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Xiaonan Gong, Yeyun Shen, Yelong Qiu, Xipeng Zhang, Hang Yao, Bolun Qi, Weizhen Jiang, Daxin Chen, Weizhu Duan, Nan Shanghai Key Laboratory of Intelligent Information Processing Fudan University China School of Computer Science Fudan University China Microsoft United States

In this paper, we propose the CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training. We adopt two contrastive learning schemes in CodeRetriever: unimodal contrastive learning and bimodal contrastive learning. For unimodal contrastive learning, we design an unsupervised learning approach to build semantic-related code pairs based on the documentation and function name. For bimodal contrastive learning, we leverage the documentation and in-line comments of code to build code-text pairs. Both contrastive objectives can fully leverage large-scale code corpus for pre-training. Extensive experimental results show that CodeRetriever achieves new state-of-the-art with significant improvement over existing code pre-trained models, on eleven domain/language-specific code search tasks with six programming languages in different code granularity (function-level, snippet-level and statement-level). These results demonstrate the effectiveness and robustness of CodeRetriever. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantics

Two-stage visual cues enhancement network for referring image segmentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jiao, Yang Jie, Zequn Luo, Weixin Chen, Jingjing Jiang, Yu-Gang Wei, Xiaolin Ma, Lin Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Meituan

Referring Image Segmentation (RIS) aims at segmenting the target object from an image referred by one given natural language expression. The diverse and flexible expressions as well as complex visual contents in the images raise the RIS model with higher demands for investigating fine-grained matching behaviors between words in expressions and objects presented in images. However, such matching behaviors are hard to be learned and captured when the visual cues of referents (i.e. referred objects) are insufficient, as the referents with weak visual cues tend to be easily confused by cluttered background at boundary or even overwhelmed by salient objects in the image. And the insufficient visual cues issue can not be handled by the cross-modal fusion mechanisms as done in previous work. In this paper, we tackle this problem from a novel perspective of enhancing the visual information for the referents by devising a Two-stage Visual cues enhancement Network (TV-Net), where a novel Retrieval and Enrichment Scheme (RES) and an Adaptive Multi-resolution feature Fusion (AMF) module are proposed. Specifically, RES retrieves the most relevant image from an external data pool with regard to both the visual and textual similarities, and then enriches the visual information of the referent with the retrieved image for better multimodal feature learning. AMF further enhances the visual detailed information by incorporating the high-resolution feature maps from lower convolution layers of the image. Through the two-stage enhancement, our proposed TV-Net enjoys better performances in learning fine-grained matching behaviors between the natural language expression and image, especially when the visual information of the referent is inadequate, thus produces better segmentation results. Extensive experiments are conducted to validate the effectiveness of the proposed method on the RIS task, with our proposed TV-Net surpassing the state-of-theart approaches on four benchmark datasets

关键词： Image segmentation

Isoform Function Prediction Based on Heterogeneous Graph Attention Networks

学校读者我要写书评

暂无评论

Isoform Function Prediction Based on Heterogeneous Graph Att...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Kuo Guo Yifan Li Hao Chen Hong-Bin Shen Yang Yang Department of Computer Science and Engineering Shanghai Jiao Tong University and Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University and Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai China Computational Biology Department School of Computer Science Carnegie Mellon University Pittsburgh PA USA

Isoforms refer to different mRNA molecules transcribed from the same gene, which can be translated into proteins with varying structures and functions. Predicting the functions of isoforms is an essential topic in bioinformatics as it can provide valuable insights into the intricate mechanisms of gene regulation and biological processes. Conventionally, gene function labels are standardized in Gene Ontology (GO) terms. However, traditional methods for predicting isoform function are largely limited by the absence of isoform-specific labels, sparse annotations, and the vast number of GO terms. To address these issues, we propose HANIso, a deep learning-based method for isoform function prediction. HANIso leverages a pretrained protein language model to extract features from protein sequences. It also integrates heterogeneous information, such as isoform sequence features, GO annotations, and isoform interaction data, using a Heterogeneous Graph Attention Network (HAN). This allows the model to learn the importance of different sources of information and their semantic relationships through the attention mechanism. Our method can predict function labels at both the gene level and isoform level. We conduct experiments on two species datasets, and the results demonstrate that our method outperforms existing methods on both AUROC and AUPRC. HANIso has the potential to overcome the limitations of traditional methods and provide a more accurate and comprehensive understanding of isoform function.

关键词：

OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hong, Lingyi Yan, Shilin Zhang, Renrui Li, Wanyun Zhou, Xinyu Guo, Pinxue Jiang, Kaixun Chen, Yiting Li, Jinglun Chen, Zhaoyu Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China The Chinese University of Hong Kong Hong Kong Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China

Visual object tracking aims to localize the target object of each frame based on its initial appearance in the first frame. Depending on the input modility, tracking tasks can be divided into RGB tracking and RGB+X (e.g. RGB+N, and RGB+D) tracking. Despite the different input modalities, the core aspect of tracking is the temporal matching. Based on this common ground, we present a general framework to unify various tracking tasks, termed as OneTracker. OneTracker first performs a large-scale pre-training on a RGB tracker called Foundation Tracker. This pretraining phase equips the Foundation Tracker with a stable ability to estimate the location of the target object. Then we regard other modality information as prompt and build Prompt Tracker upon Foundation Tracker. Through freezing the Foundation Tracker and only adjusting some additional trainable parameters, Prompt Tracker inhibits the strong localization ability from Foundation Tracker and achieves parameter-efficient finetuning on downstream RGB+X tracking tasks. To evaluate the effectiveness of our general framework OneTracker, which is consisted of Foundation Tracker and Prompt Tracker, we conduct extensive experiments on 6 popular tracking tasks across 11 benchmarks and our OneTracker outperforms other models and achieves state-ofthe-art performance. Copyright © 2024, The Authors. All rights reserved.

关键词： Foundations

Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Huang, Bingchen Chen, Zhineng Zhou, Peng Chen, Jiayin Wu, Zuxuan Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China University of Maryland College ParkMD United States

The dynamic expansion architecture is becoming popular in class incremental learning, mainly due to its advantages in alleviating catastrophic forgetting. However, task confusion is not well assessed within this framework, e.g., the discrepancy between classes of different tasks is not well learned (i.e., inter-task confusion, ITC), and certain priority is still given to the latest class batch (i.e., old-new confusion, ONC). We empirically validate the side effects of the two types of confusion. Meanwhile, a novel solution called Task Correlated Incremental Learning (TCIL) is proposed to encourage discriminative and fair feature utilization across tasks. TCIL performs a multi-level knowledge distillation to propagate knowledge learned from old tasks to the new one. It establishes information flow paths at both feature and logit levels, enabling the learning to be aware of old classes. Besides, attention mechanism and classifier re-scoring are applied to generate more fair classification scores. We conduct extensive experiments on CIFAR100 and ImageNet100 datasets. The results demonstrate that TCIL consistently achieves state-of-the-art accuracy. It mitigates both ITC and ONC, while showing advantages in battle with catastrophic forgetting even no rehearsal memory is reserved. 1 Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

Adaptive Pedestrian Trajectory Prediction via Target-Directed Angle Augmentation

学校读者我要写书评

暂无评论

Adaptive Pedestrian Trajectory Prediction via Target-Directe...

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Hao Kong Jie Xu Shenjian Gong Jian Yang Shanshan Zhang PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education and Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China

Pedestrian trajectory prediction is an important task for many applications such as autonomous driving and surveillance systems. Yet the prediction performance drops dramatically when applying a model trained on the source domain to a new target domain. Therefore, it is of great importance to adapt a predictor to a new domain. Previous works mainly focus on feature-level alignment to solve this problem. In contrast, we solve it from a new perspective of instance-level alignment. Specifically, we first point out one key factor of the domain gaps, i.e., trajectory angles, and then augment the source training data by target-directed orientation augmentation so that its distribution matches with that of the target data. In this way, the trajectory predictor trained on the aligned source data performs better on the target domain. Experiments on standard baselines show that our method improves the state of the art by a large margin. The source code is available at https://***/NeoKH/PTP-DA.

关键词：

BBTv2: Towards a Gradient-Free Future with Large Language Models

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Sun, Tianxiang He, Zhengfu Qian, Hong Zhou, Yunhua Huang, Xuanjing Qiu, Xipeng School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China School of Computer Science and Technology East China Normal University China

Most downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growth of the model size. By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment. Though, past work on gradient-free tuning often introduces gradient descent to seek a good initialization of prompt and lacks versatility across tasks and PTMs. In this paper, we present BBTv2, an improved version of Black-Box Tuning (Sun et al., 2022b), to drive PTMs for few-shot learning. We prepend continuous prompts to every layer of the PTM and propose a divide-and-conquer gradient-free algorithm to optimize the prompts at different layers alternately. Extensive experiments across various tasks and PTMs show that BBTv2 can achieve comparable performance to full model tuning and state-of-the-art parameter-efficient methods (e.g., Adapter, LoRA, BitFit, etc.) under few-shot settings while maintaining much fewer tunable parameters. Copyright © 2022, The Authors. All rights reserved.

关键词： Gradient methods

Large Language Models are Complex Table Parsers

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhao, Bowen Ji, Changkai Zhang, Yuejie He, Wen Wang, Yingwen Wang, Qing Feng, Rui Zhang, Xiaobo School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai200433 China Academy for Engineering and Technology Fudan University Shanghai China Children’s Hospital of Fudan University National Children’s Medical Center Shanghai China

With the Generative Pre-trained Transformer 3.5 (GPT-3.5) exhibiting remarkable reasoning and comprehension abilities in Natural Language processing (NLP), most Question Answering (QA) research has primarily centered around general QA tasks based on GPT, neglecting the specific challenges posed by Complex Table QA. In this paper, we propose to incorporate GPT-3.5 to address such challenges, in which complex tables are reconstructed into tuples and specific prompt designs are employed for dialogues. Specifically, we encode each cell’s hierarchical structure, position information, and content as a tuple. By enhancing the prompt template with an explanatory description of the meaning of each tuple and the logical reasoning process of the task, we effectively improve the hierarchical structure awareness capability of GPT-3.5 to better parse the complex tables. Extensive experiments and results on Complex Table QA datasets, i.e., the open-domain dataset HiTAB and the aviation domain dataset AIT-QA show that our approach significantly outperforms previous work on both datasets, leading to state-of-the-art (SOTA) performance. Copyright © 2023, The Authors. All rights reserved.

关键词： Natural language processing systems

Flexible and highly-sensitive pressure sensor based on controllably oxidized MXene

学校读者我要写书评

暂无评论

InfoMat 2022年第9期4卷 60-71页

作者： Yanan Ma Yongfa Cheng Jian Wang Si Fu Mengjun Zhou Yue Yang Baowen Li Xin Zhang Ce-Wen Nan State Key Laboratory of Advanced Technology for Materials Synthesis and Processing Center of Smart Materials and Devices&International School of Materials Science and EngineeringWuhan University of TechnologyWuhanChina School of Mathematics Physics and Optoelectronic EngineeringHubei University of Automotive TechnologyShiyanChina Center for Nanoscale Characterization&Devices(CNCD) Wuhan National Laboratory for Optoelectronics(WNLO)Huazhong University of Science and Technology(HUST)WuhanChina Information Materials and Intelligent Sensing Laboratory of Anhui Province Key Laboratory of Structure and Functional Regulation of Hybrid Materials of Ministry of EducationInstitutes of Physical Science and Information TechnologyAnhui UniversityHefeiChina State Key Lab of New Ceramics and Fine Processing School of Materials Science and EngineeringTsinghua UniversityBeijingChina

Conductive Ti_(3)C_(2)T_(x)MXenes have been widely investigated for the construction of flexible and highly-sensitive pressure *** the inevitable oxidation of solution-processed MXene has been recognized,the effect of the irreversible oxidation of MXene on its electrical conductivity and sensing properties is yet to be ***,we construct a highly-sensitive and degradable piezoresistive pressure sensor by coating Ti_(3)C_(2)T_(x)MXene flakes with different degrees of in situ oxidation onto paper substrates using the dipping-drying *** situ oxidation can tune the intrinsic resistance and expand the interlayer distance of MXene *** partially oxidized MXene-based piezoresistive pressure sensor exhibits a high sensitivity of 28.43 kPa^(-1),which is greater than those of pristine MXene,over-oxidized MXene,and state-of-the-art paper-based pressure ***,these sensors exhibit a short response time of 98.3 ms,good durability over 5000 measurement cycles,and a low force detection limit of 0.8 ***,MXene-based sensing elements are easily degraded and environmentally *** MXene-based pressure sensor shows promise for practical applications in tracking body movements,sports coaching,remote health monitoring,and human–computer interactions.

关键词： adjustable sensitivity in situ oxidation interlayer distance MXene pressure sensor