检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Yang, Yang The Nanjing University of Science and Technology Nanjing210094 China PCA Lab Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Lab of Image and Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology China Ministry of Education State Key Lab. for Novel Software Technology Nanjing University China

Image captioning can automatically generate captions for the given images, and the key challenge is to learn a mapping function from visual features to natural language features. Existing approaches are mostly supervised ones, i.e., each image has a corresponding sentence in the training set. However, considering that describing images always requires a huge of manpower, we usually have limited amount of described images (i.e., image-text pairs) and a large number of undescribed images in real-world applications. Thereby, a dilemma is the "Semi-Supervised Image Captioning". To solve this problem, we propose a novel Semi-Supervised Image Captioning method considering Wasserstein Graph Matching (SSIC-WGM), which turns to adopt the raw image inputs to supervise the generated sentences. Different from traditional single modal semi-supervised methods, the difficulty of semi-supervised cross-modal learning lies in constructing intermediately comparable information among heterogeneous modalities. In this paper, SSIC-WGM adopts the successful scene graphs as intermediate information, and constrains the generated sentences from two aspects: 1) inter-modal consistency. SSIC-WGM constructs the scene graphs of the raw image and generated sentence respectively, then employs the wasserstein distance to better measure the similarity between region embeddings of different graphs. 2) intra-modal consistency. SSIC-WGM takes the data augmentation techniques for the raw images, then constrains the consistency among augmented images and generated sentences. Consequently, SSIC-WGM combines the cross-modal pseudo supervision and structure invariant measure for efficiently using the undescribed images, and learns more reasonable mapping function. Experiments show that our method can outperform state-of-the-art comparison methods on the MS-COCO "Karpathy" offline test split, under different complex semi-supervised scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

A comprehensive survey on shadow removal from document images: datasets, methods, and opportunities

引用

Vicinagearth 2025年第1期2卷 1-18页

作者： Wang, Bingshu Li, Changping Zou, Wenbin Zhang, Yongjun Chen, Xuhang Chen, C.L. Philip School of Software Northwestern Polytechnical University Xi’an China Guangdong Provincial Key Laboratory of Intelligent Information Processing & Shenzhen Key Laboratory of Media Security Shenzhen University Shenzhen China Guangdong Key Laboratory of Intelligent Information Processing College of Electronics and Information Engineering Shenzhen University Shenzhen China Yongjun Zhang is with the State Key Laboratory of Public Big Data College of Computer Science and Technology Guizhou University Guiyang China School of Computer Science and Engineering Huizhou University Huizhou China School of Computer Science and Engineering South China University of Technology and Pazhou Lab Guangzhou China

With the rapid development of document digitization, people have become accustomed to capturing and processing documents using electronic devices such as smartphones. However, the captured document images often suffer from issues like shadows and noise due to environmental factors, which can affect their readability. To improve the quality of captured document images, researchers have proposed a series of models or frameworks and applied them in distinct scenarios such as image enhancement, and document information extraction. In this paper, we primarily focus on shadow removal methods and open-source datasets. We concentrate on recent advancements in this area, first organizing and analyzing nine available datasets. Then, the methods are categorized into conventional methods and neural network-based methods. Conventional methods use manually designed features and include shadow map-based approaches and illumination-based approaches. Neural network-based methods automatically generate features from data and are divided into single-stage approaches and multi-stage approaches. We detail representative algorithms and briefly describe some typical techniques. Finally, we analyze and discuss experimental results, identifying the limitations of datasets and methods. Future research directions are discussed, and nine suggestions for shadow removal from document images are proposed. To our knowledge, this is the first survey of shadow removal methods and related datasets from document images.

关键词：

来源：评论

学校读者我要写书评

暂无评论

An Innovative Approach To Software Modularization Based On The Artificial Fish Swarm Algorithm 22

An Innovative Approach To Software Modularization Based On T...

引用

Proceedings of the 2022 6th International Conference on Electronic Information Technology and computer Engineering

作者： Jianqiang Pan Cheng Zhang Huihui Jia Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Computer Science and Technology State Key Lab for Novel Software Technology Anhui University China

ISBN: (纸本)9781450397148

It gets more and more expensive to maintain the complete software system as time goes on since the software architecture grows more complicated and the software code is more difficult to understand. This issue may be solved by taking the essential parts out of the source code and organizing them into the appropriate subsystems. Hierarchy-based partitioning is more complex and less successful in solving issues with huge software modules. In order to do this, this work suggests an innovative artificial fish swarm method as a meta-heuristic to solve the software module clustering problems (SMCPs) and presents a mutation operator and a local search strategy to handle the premature situation. The performance of the suggested method is assessed on a range of real-world software systems by comparison to the most recent meta-heuristic methods. The outcomes of the experiments demonstrate that the suggested methodology works better than those of other methods.

关键词： Software refactoring Artificial fish swarm algorithm Software module clustering

来源：评论

学校读者我要写书评

暂无评论

Attention Bootstrapping for Multi-Modal Test-Time Adaptation

arXiv

引用

arXiv 2025年

作者： Zhao, Yusheng Luo, Junyu Luo, Xiao Huang, Jinsheng Yuan, Jingyang Xiao, Zhiping Zhang, Ming State Key Laboratory for Multimedia Information Processing School of Computer Science PKU-Anker LLM Lab Peking University Beijing China Department of Computer Science University of California Los AngelesCA United States Paul G. Allen School of Computer Science and Engineering University of Washington SeattleWA United States

Test-time adaptation aims to adapt a well-trained model to potential distribution shifts at test time using only unlabeled test data, without access to the original training data. While previous efforts mainly focus on a single modality, test-time distribution shift in the multi-modal setting is more complex and calls for new solutions. This paper tackles the problem of multi-modal test-time adaptation by proposing a novel method named Attention Bootstrapping with Principal Entropy Minimization (ABPEM). We observe that test-time distribution shift causes misalignment across modalities, leading to a large gap between intra-modality discrepancies (measured by self-attention) and inter-modality discrepancies (measured by cross-attention). We name this the attention gap. This attention gap widens with more severe distribution shifts, hindering effective modality fusion. To mitigate this attention gap and encourage better modality fusion, we propose attention bootstrapping that promotes cross-attention with the guidance of self-attention. Moreover, to reduce the gradient noise in the commonly-used entropy minimization, we adopt principal entropy minimization, a refinement of entropy minimization that reduces gradient noise by focusing on the principal parts of entropy, excluding less reliable gradient information. Extensive experiments on the benchmarks validate the effectiveness of the proposed ABPEM in comparison with competing baselines. © 2025, CC BY-NC-ND.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

arXiv

引用

arXiv 2024年

作者： Zhang, Zeyi Ao, Tenglong Zhang, Yuyao Gao, Qingzhe Lin, Chuan Chen, Baoquan Liu, Libin School of Electronics Engineering and Computer Science Peking University China School of Computer Science Peking University China Renmin University of China China Shandong University China Peking University China State Key Lab of General AI China

In this work, we present Semantic Gesticulator, a novel framework designed to synthesize realistic gestures accompanying speech with strong semantic correspondence. Semantically meaningful gestures are crucial for effective non-verbal communication, but such gestures often fall within the long tail of the distribution of natural human motion. The sparsity of these movements makes it challenging for deep learning-based systems, trained on moderately sized datasets, to capture the relationship between the movements and the corresponding speech semantics. To address this challenge, we develop a generative retrieval framework based on a large language model. This framework efficiently retrieves suitable semantic gesture candidates from a motion library in response to the input speech. To construct this motion library, we summarize a comprehensive list of commonly used semantic gestures based on findings in linguistics, and we collect a high-quality motion dataset encompassing both body and hand movements. We also design a novel GPT-based model with strong generalization capabilities to audio, capable of generating high-quality gestures that match the rhythm of speech. Furthermore, we propose a semantic alignment mechanism to efficiently align the retrieved semantic gestures with the GPT’s output, ensuring the naturalness of the final animation. Our system demonstrates robustness in generating gestures that are rhythmically coherent and semantically explicit, as evidenced by a comprehensive collection of examples. User studies confirm the quality and human-likeness of our results, and show that our system outperforms state-of-the-art systems in terms of semantic appropriateness by a clear margin. We will release the code and dataset for academic research. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

CorrMAE: Pre-training Correspondence Transformers with Masked Autoencoder

arXiv

引用

arXiv 2024年

作者： Liao, Tangfei Zhang, Xiaoqin Xiao, Guobao Li, Min Wang, Tao Ye, Mang School of Computer Science Wuhan University China College of Computer Science and Artificial Intelligence Wenzhou University China School of Electronics and Information Engineering Tongji University China State Key Lab for Novel Software Technology Nanjing University China

Pre-training has emerged as a simple yet powerful methodology for representation learning across various domains. However, due to the expensive training cost and limited data, pre-training has not yet been extensively studied in correspondence pruning. To tackle these challenges, we propose a pre-training method to acquire a generic inliers-consistent representation by reconstructing masked correspondences, providing a strong initial representation for downstream tasks. Toward this objective, a modicum of true correspondences naturally serve as input, thus significantly reducing pre-training overhead. In practice, we introduce CorrMAE, an extension of the mask autoencoder framework tailored for the pre-training of correspondence pruning. CorrMAE involves two main phases, i.e., correspondence learning and matching point reconstruction, guiding the reconstruction of masked correspondences through learning visible correspondence consistency. Herein, we employ a dual-branch structure with an ingenious positional encoding to reconstruct unordered and irregular correspondences. Also, a bi-level designed encoder is proposed for correspondence learning, which offers enhanced consistency learning capability and transferability. Extensive experiments have shown that the model pre-trained with our CorrMAE outperforms prior work on multiple challenging benchmarks. Meanwhile, our CorrMAE is primarily a task-driven pre-training method, and can achieve notable improvements for downstream tasks by pre-training on the targeted dataset. We hope this work can provide a starting point for correspondence pruning pre-training. © 2024, CC BY-NC-SA.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

TextToucher: Fine-Grained Text-to-Touch Generation

arXiv

引用

arXiv 2024年

作者： Tu, Jiahang Fu, Hao Yang, Fengyu Zhao, Hanbin Zhang, Chao Qian, Hui College of Computer Science and Technology Zhejiang University China Advanced Technology Institute Zhejiang University China State Key Lab of CAD&CG Zhejiang University China College of Computer Science and Technology Yale University United States

Tactile sensation plays a crucial role in the development of multi-modal large models and embodied intelligence. To collect tactile data with minimal cost as possible, a series of studies have attempted to generate tactile images by vision-to-touch image translation. However, compared to text modality, visual modality-driven tactile generation cannot accurately depict human tactile sensation. In this work, we analyze the characteristics of tactile images in detail from two granularities: object-level (tactile texture, tactile shape), and sensor-level (gel status). We model these granularities of information through text descriptions and propose a fine-grained Text-to-Touch generation method (TextToucher) to generate high-quality tactile samples. Specifically, we introduce a multimodal large language model to build the text sentences about object-level tactile information and employ a set of learnable text prompts to represent the sensor-level tactile information. To better guide the tactile generation process with the built text information, we fuse the dual grains of text information and explore various dual-grain text conditioning methods within the diffusion transformer architecture. Furthermore, we propose a Contrastive Text-Touch Pre-training (CTTP) metric to precisely evaluate the quality of text-driven generated tactile data. Extensive experiments demonstrate the superiority of our TextToucher method. Copyright © 2024, The Authors. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

On the Approximation of the Quotient of Two Gaussian Densities for Multiple-Model Smoothing 26

On the Approximation of the Quotient of Two Gaussian Densiti...

引用

26th International Conference on Information Fusion, FUSION 2023

作者： Liu, Yi Li, Xi Yang, Le Mihaylova, Lyudmila Xue, Yanbo University of Canterbury Department of Electrical and Computer Engineering Christchurch New Zealand National University of Defense Technology State Key Lab of Complex Electromagnetic Environment Effects on Electronics and Information Systems Changsha China University of Sheffield Department of Automatic Control and System Engineering Sheffield United Kingdom Career Science Lab Beijing China

ISBN: (纸本)9798890344854

The quotient of two multivariate Gaussian densities can be written as an unnormalized Gaussian density, which has been applied in some recently developed multiple-model fixed-interval smoothing algorithms. However, this expression is invalid if instead of being positive definite, the covariance of the unnormalized Gaussian density is indefinite (i.e., it has both positive and negative eigenvalues) or undefined (i.e., computing it requires inverting a singular matrix). This paper considers approximating the quotient of two Gaussian densities in this case using two different approaches to mitigate the caused numerical problems. The first approach directly replaces the indefinite covariance of the unnormalized Gaussian density with a positive definite matrix nearest to it. The second approach computes the approximation through solving, using the natural gradient, an optimization problem with a Kullback-Leibler divergence-based cost function. This paper illustrates the application of the theoretical results by incorporating them into an existing smoothing method for jump Markov systems and utilizing the obtained smoothers to track a maneuvering target. © 2023 International Society of Information Fusion.

关键词： Cost functions

来源：评论

学校读者我要写书评

暂无评论

SOUND ADVERSARIAL AUDIO-VISUAL NAVIGATION 10

SOUND ADVERSARIAL AUDIO-VISUAL NAVIGATION

引用

10th International Conference on Learning Representations, ICLR 2022

作者： Yu, Yinfeng Huang, Wenbing Sun, Fuchun Chen, Changan Wang, Yikai Liu, Xiaohong State Key Lab on Intelligent Technology and Systems Department of Computer Science and Technology Tsinghua University China Tsinghua University China College of Information Science and Engineering Xinjiang University China UT Austin United States JD Explore Academy ***

Audio-visual navigation task requires an agent to find a sound source in a realistic, unmapped 3D environment by utilizing egocentric audio-visual observations. Existing audio-visual navigation works assume a clean environment that solely contains the target sound, which, however, would not be suitable in most real-world applications due to the unexpected sound noise or intentional interference. In this work, we design an acoustically complex environment in which, besides the target sound, there exists a sound attacker playing a zero-sum game with the agent. More specifically, the attacker can move and change the volume and category of the sound to make the agent suffer from finding the sounding object, while the agent tries to dodge the attack and navigate to the goal under the intervention. Under certain constraints to the attacker, we can improve the robustness of the agent towards unexpected sound attacks in audio-visual navigation. For better convergence, we develop a joint training mechanism by employing the property of a centralized critic with decentralized actors. Experiments on two real-world 3D scan datasets (Replica and Matterport3D) verify the effectiveness and the robustness of the agent trained under our designed environment when transferred to the clean environment or the one containing sound attackers with random policy. Project: https://***/SAAVN. © 2022 ICLR 2022 - 10th International Conference on Learning Representationss. All rights reserved.

关键词： Navigation

来源：评论

学校读者我要写书评

暂无评论

Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization

Learning Instructions with Unlabeled Data for Zero-Shot Cros...

引用

2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022

作者： Gu, Yuxian Ke, Pei Zhu, Xiaoyan Huang, Minlie The CoAI group Tsinghua University Beijing China Institute for Artificial Intelligence State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Department of Computer Science and Technology Tsinghua University Beijing China

Training language models to learn from human instructions for zero-shot cross-task generalization has attracted much attention in NLP communities. Recently, instruction tuning (IT), which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions, has been shown effective in instruction learning for unseen tasks. However, IT relies on a large amount of human-annotated samples, which restricts its generalization. Unlike labeled data, unlabeled data are often massive and cheap to obtain. In this work, we study how IT can be improved with unlabeled data. We first empirically explore the IT performance trends versus the number of labeled data, instructions, and training tasks. We find it critical to enlarge the number of training instructions, and the instructions can be underutilized due to the scarcity of labeled data. Then, we propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during IT by constructing pseudo-labeled data from unlabeled plain texts. We conduct extensive experiments to show UDIT's effectiveness in various scenarios of tasks and datasets. We also comprehensively analyze the key factors of UDIT to investigate how to better improve IT with unlabeled data. The code is publicly available at https://***/thu-coai/UDIT. © 2022 Association for Computational Linguistics.

关键词： Zero-shot learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：