检索结果-内蒙古大学图书馆

Alignment efficient image-sentence retrieval considering transferable cross-modal representation learning

Frontiers of computer science 2024年第1期18卷 77-91页

作者： Yang YANG Jinyi GUO Guangyu LI Lanyu LI Wenjie LI Jian YANG School of Computer Science and Engineering Nanjing University of Science and TechnologyNanjing 210094China Department of Computing Hong Kong Polytechnic UniversityHong Kong 100872China State Key Lab.for Novel Software Technology Nanjing UniversityNanjing 210094China 14th Research Institute of China Electronics Technology Group Corporation Nanjing 210094China guangyu.li2017@***

Traditional image-sentence cross-modal retrieval methods usually aim to learn consistent representations of heterogeneous modalities,thereby to search similar instances in one modality according to the query from another modality in *** basic assumption behind these methods is that parallel multi-modal data(i.e.,different modalities of the same example are aligned)can be obtained in *** other words,the image-sentence cross-modal retrieval task is a supervised task with the alignments as ***,in many real-world applications,it is difficult to realign a large amount of parallel data for new scenarios due to the substantial labor costs,leading the non-parallel multi-modal data and existing methods cannot be used *** the other hand,there actually exists auxiliary parallel multi-modal data with similar semantics,which can assist the non-parallel data to learn the consistent ***,in this paper,we aim at“Alignment Efficient Image-Sentence Retrieval”(AEIR),which recurs to the auxiliary parallel image-sentence data as the source domain data,and takes the non-parallel data as the target domain *** single-modal transfer learning,AEIR learns consistent image-sentence cross-modal representations of target domain by transferring the alignments of existing parallel ***,AEIR learns the image-sentence consistent representations in source domain with parallel data,while transferring the alignment knowledge across domains by jointly optimizing a novel designed cross-domain cross-modal metric learning based constraint with intra-modal domain adversarial ***,we can effectively learn the consistent representations for target domain considering both the structure and semantic ***,extensive experiments on different transfer scenarios validate that AEIR can achieve better retrieval results comparing with the baselines.

关键词： image-sentence retrieval transfer learning semantic transfer structure transfer

来源：评论

学校读者我要写书评

暂无评论

Time and Cost-Efficient Cloud Data Transmission based on Serverless Computing Compression 42

Time and Cost-Efficient Cloud Data Transmission based on Ser...

引用

42nd IEEE International Conference on computer Communications, INFOCOM 2023

作者： Gu, Rong Chen, Xiaofei Dai, Haipeng Wang, Shulin Wang, Zhaokang Tu, Yaofeng Huang, Yihua Chen, Guihai Nanjing University State Key Laboratory for Novel Software Technology China Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology China Zte Cooperation China

ISBN: (纸本)9798350334142

Nowadays, there exists a lot of cross-region data transmission demand on cloud. It is promising to use serverless computing for compressing data to save the transmission data amount. However, it is challenging to estimate the data transmission time and monetary cost with serverless compression. In addition, minimizing the data transmission cost is non-trivial due to enormous parameter space and joint optimization. This paper focuses on this problem and makes the following contributions: (1) We propose empirical data transmission time and monetary cost models based on serverless compression. (2) For single-task cloud data transmission, we propose two efficient parameter search methods based on Sequential Quadratic Programming (SQP ) and Eliminate then Divide and Conquer (EDC), which are theoretically proven with error upper bounds. (3) Furthermore, for multi-task cloud data transmission, a parameter search method based on dynamic programming and numerical computation is proposed to reduce the algorithm complexity from exponential to linear complexity. We have implemented the entire actual system and evaluated it with various workloads and application cases on the real-world AWS serverless computing platform. Experimental results on cross-region public cloud show that the proposed approach can improve the parameter search efficiency by more than 3× compared with the state-of-art parameter search methods and achieves better parameter quality. Compared with other competing cloud data transmission approaches, our approach is able to achieve higher time efficiency and lower monetary cost. © 2023 IEEE.

关键词： Efficiency

来源：评论

学校读者我要写书评

暂无评论

Detecting Double Mixed Compressed Images Based on Quaternion Convolutional Neural Network

引用

Chinese Journal of Electronics 2024年第3期33卷 657-671页

作者： Hao WANG Jinwei WANG Xuelong HU Bingtao HU Qilin YIN Xiangyang LUO Bin MA Jinsheng SUN Department of Automation Nanjing University of Science and Technology Engineering Research Center of Digital Forensics Ministry of Education Department of Computer and Software Nanjing University of Information Science and Technology School of Management Nanjing University of Posts and Telecommunications Department of Computer Science and Engineering Guangdong Province Key Laboratory of Information Security Technology Sun Yat-sen University State Key Laboratory of Mathematical Engineering and Advanced Computing Qilu University of Technology

Detection of color images that have undergone double compression is a critical aspect of digital image *** the existence of various methods capable of detecting double Joint Photographic Experts Group(JPEG) compression,they are unable to address the issue of mixed double compression resulting from the use of different compression *** particular,the implementation of Joint Photographic Experts Group 2000(JPEG2000)as the secondary compression standard can result in a decline or complete loss of performance in existing *** tackle this challenge of JPEG+JPEG2000 compression,a detection method based on quaternion convolutional neural networks(QCNN) is *** QCNN processes the data as a quaternion,transforming the components of a traditional convolutional neural network(CNN) into a quaternion *** relationships between the color channels of the image are preserved,and the utilization of color information is ***,the method includes a feature conversion module that converts the extracted features into quaternion statistical features,thereby amplifying the evidence of double *** results indicate that the proposed QCNN-based method improves,on average,by 27% compared to existing methods in the detection of JPEG+JPEG2000 compression.

关键词： Color image forensics JPEG JPEG2000 Mixed double compression Quaternion convolutional neural network

来源：评论

学校读者我要写书评

暂无评论

Improving actionable warning identification via the refined warning-inducing context representation

引用

science China(Information sciences) 2024年第5期67卷 321-322页

作者： Xiuting GE Chunrong FANG Xuanye LI Quanjun ZHANG Jia LIU Zhihong ZHAO Zhenyu CHEN State Key Laboratory for Novel Software Technology Nanjing University

Static analysis tools(SATs) have shown potential ability in software quality assurance [1]. However, SATs report an overwhelming number of warnings, where most warnings are unactionable ones(i.e., warnings that are not acted on or fixed by developers). It is not a trivial task for developers to find actionable warnings(i.e., warnings that are acted on and fixed by developers) in all reported warnings. Hence,the excessive unactionable warnings seriously impede the usability of SATs [2].

关键词： representation identification context actionable improving inducing refined warning

来源：评论

学校读者我要写书评

暂无评论

Efficient Nearest Neighbor Emotion Classification with BERT-whitening

Efficient Nearest Neighbor Emotion Classification with BERT-...

引用

2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022

作者： Yin, Wenbiao Shang, Lin State Key Laboratory for Novel Software Technology Department of Computer Science and Technology Nanjing University Nanjing China

Retrieval-based methods have been proven effective in many NLP tasks. Previous methods use representations from the pre-trained model for similarity search directly. However, the sentence representations from the pre-trained model like BERT perform poorly in retrieving semantically similar sentences, resulting in poor performance of the retrieval-based methods. In this paper, we propose KNN-EC, a simple and efficient non-parametric emotion classification (EC) method using nearest neighbor retrieval. We use BERT-whitening to get better sentence semantics, ensuring that nearest neighbor retrieval works. Meanwhile, BERT-whitening can also reduce memory storage of datastore and accelerate retrieval speed, solving the efficiency problem of the previous methods. KNN-EC average improves the pre-trained model by 1.17 F1-macro on two emotion classification datasets. © 2022 Association for Computational Linguistics.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Complementary Learning System Theory-based Active Learning for Audio Classification

Complementary Learning System Theory-based Active Learning f...

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Geng, Hui Gao, Zijian Wan, Tianjiao Feng, Dawei Wang, Changjian Xu, Kele College of Computer Science and Technology National University of Defense Technology Changsha China State Key Laboratory of Complex & Critical Software Environment Changsha China

ISBN: (纸本)9798350368741

Deep learning has significantly advanced the audio classification, achieving remarkable results. However, these successes often rely on extensive manual annotation of audio, a labor-intensive and costly process. Active Learning (AL) presents a promising solution by minimizing the required amount of annotation through the iterative selection of the most informative audio samples. Current AL methods for audio classification typically depend solely on the latest model checkpoint, overlooking the dynamics of the entire training process. The Complementary Learning Systems (CLS) theory posits that the interplay between short-term and long-term memory systems can effectively measure sample uncertainty, offering a means to capture training dynamics. In this work, we introduce a novel AL framework for audio classification, termed CLS-AL, which addresses the limitations of existing methods by simultaneously maintaining both short-term and long-term memory models. This dual-memory approach allows for a more comprehensive consideration of training dynamics. The divergence in predictions between these memory models provides a new metric for evaluating the uncertainty of unlabeled samples, enhancing the effectiveness of the AL sample selection process. We demonstrate the effectiveness and generalizability of CLS-AL through extensive experiments on a diverse set of audio datasets, showing that CLS-AL obviously outperforms existing state-of-the-art methods. © 2025 IEEE.

关键词： Active Learning Audio Classification Complementary Learning System Uncertainty Estimation

来源：评论

学校读者我要写书评

暂无评论

Multiple Domain-Adversarial Ensemble Learning for Domain Generalization 48

Multiple Domain-Adversarial Ensemble Learning for Domain Gen...

引用

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Mi, Ze-Yu Long, Kun Yang, Yu-Bin Nanjing University State Key Laboratory for Novel Software Technology Nanjing210023 China

ISBN: (纸本)9781728163277

Domain generalization (DG) aims to train a model on multiple source domains which can be well generalized to the unseen target domains. Currently, most DG techniques in vision tasks mainly focus on one of the three research lines (i.e., data manipulation, representation learning, and learning strategy). The direction of combining multiple DG research lines still remains to be studied and explored. In this paper, we propose a unified framework for DG that combines multiple research lines to enhance the generalization ability. To be specific, we combine representative strategies of three research lines: ensemble learning, domain alignment, and data augmentation. A basic framework is built with an ensemble learning strategy to train an expert model in a domain unit. Besides, the ability to learn domain-independent features is enhanced with an adversarial-based domain alignment strategy. Furthermore, a domain transformation network (DoTNet) is introduced, which is expected to generate additional deviated images and further enhance the generalization capability. Extensive experiments on DG datasets demonstrate the effectiveness of our approach. © 2023 IEEE.

关键词： Learning systems

来源：评论

学校读者我要写书评

暂无评论

Understanding Advertisements 39

Understanding Advertisements

引用

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Feng, Yi Li, Chuanyi Ng, Vincent State Key Laboratory for Novel Software Technology Nanjing University China Human Language Technology Research Institute University of Texas Dallas United States

ISBN: (纸本)157735897X

While AI systems are capable of reading texts and seeing images, they typically perceive surface information explicitly conveyed with limited abilities to comprehend hidden messages (e.g., a double-edged remark). We propose the novel task of advertisement understanding: given an advertisement, which can be a text, an image, or a video, the goal is to identify the persuasion strategies used and determine the (possibly hidden) messages conveyed. Efforts on this task could enhance machine comprehension capabilities, and provide users with increased situation awareness w.r.t. the advertised message and thus possibly enable mindful decision making. We believe that this task presents long-term challenges to AI researchers and that successful understanding of ads could bring machine understanding one important step closer to human understanding. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Sales

来源：评论

学校读者我要写书评

暂无评论

Prioritizing Testing Instances to Enhance the Robustness of Object Detection Systems 23

Prioritizing Testing Instances to Enhance the Robustness of ...

引用

14th Asia-Pacific Symposium on Internetware, Internetware 2023

作者： Weng, Shihao Feng, Yang Yin, Yining Liu, Jia State Key Laboratory for Novel Software Technology Nanjing University Nanjing210023 China

ISBN: (纸本)9798400708947

Object detection models have been widely deployed in military and life-related intelligent software systems. However, along with the outstanding success of object detection, it may exhibit abnormal behavior and lead to severe accidents and losses. During the development and evaluation process, training and evaluating an object detection model are computationally intensive, while preparing annotated tests requires extremely heavy manual labor. Therefore, reducing the annotation budget of test data collection becomes a challenging and necessary task. Although many test prioritization approaches for DNN-based systems have been proposed, the large differences between classification and object detection make them difficult to apply to testing object detection models. In this paper, we propose DeepView, a novel instance-level test prioritization tool for object detection models to reduce data annotation costs. DeepView first splits the object detection results into instances, and then computes the localization and classification capabilities of the instances, respectively. Next, we design a test prioritization tool that enables testers to improve model performance by focusing on instances that may cause model errors from a large unlabeled dataset. To evaluate DeepView, we conduct extensive experiments on two kinds of object detection model architectures and two commonly used datasets. The experimental results show that DeepView outperforms existing test prioritization approaches regarding effectiveness and diversity. Also, we observe that using DeepView can effectively improve the accuracy and robustness of object detection models. © 2023 ACM.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

Skinned Motion Retargeting with Preservation of Body Part Relationships

引用

IEEE Transactions on Visualization and computer Graphics 2024年 PP卷 1-13页

作者： Zhang, Jia-Qi Wang, Miao Zhang, Fu-Cheng Zhang, Fang-Lue State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China State Key Laboratory of Virtual Reality Technology and Systems School of Computer Science and Engineering Beihang University Beijing China School of Engineering and Computer Science Victoria University of Wellington New Zealand

Motion retargeting is an active research area in computer graphics and animation, allowing for the transfer of motion from one character to another, thereby creating diverse animated character data. While this technology has numerous applications in animation, games, and movies, current methods often produce unnatural or semantically inconsistent motion when applied to characters with different shapes or joint counts. This is primarily due to a lack of consideration for the geometric and spatial relationships between the body parts of the source and target characters. To tackle this challenge, we introduce a novel spatially-preserving Skinned Motion Retargeting Network (SMRNet) capable of handling motion retargeting for characters with varying shapes and skeletal structures while maintaining semantic consistency. By learning a hybrid representation of the character's skeleton and shape in a rest pose, SMRNet transfers the rotation and root joint position of the source character's motion to the target character through embedded rest pose feature alignment. Additionally, it incorporates a differentiable loss function to further preserve the spatial consistency of body parts between the source and target. Comprehensive quantitative and qualitative evaluations demonstrate the superiority of our approach over existing alternatives, particularly in preserving spatial relationships more effectively IEEE

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：