检索结果-内蒙古大学图书馆

BUFF: Bayesian Uncertainty Guided Diffusion Probabilistic Model for Single Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2025年

作者： He, Zihao Zhang, Shengchuan Hu, Runze Shen, Yunhang Zhang, Yan Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University 361005 China Department of Information Science Tsinghua university China Tencent Youtu Lab Shanghai200233 China

Super-resolution (SR) techniques are critical for enhancing image quality, particularly in scenarios where high-resolution imagery is essential yet limited by hardware constraints. Existing diffusion models for SR have relied predominantly on Gaussian models for noise generation, which often fall short when dealing with the complex and variable texture inherent in natural scenes. To address these deficiencies, we introduce the Bayesian Uncertainty Guided Diffusion Probabilistic Model (BUFF). BUFF distinguishes itself by incorporating a Bayesian network to generate high-resolution uncertainty masks. These masks guide the diffusion process, allowing for the adjustment of noise intensity in a manner that is both context-aware and adaptive. This novel approach not only enhances the fidelity of super-resolved images to their original high-resolution counterparts but also significantly mitigates artifacts and blurring in areas characterized by complex textures and fine details. The model demonstrates exceptional robustness against complex noise patterns and showcases superior adaptability in handling textures and edges within images. Empirical evidence, supported by visual results, illustrates the model’s robustness, especially in challenging scenarios, and its effectiveness in addressing common SR issues such as blurring. Experimental evaluations conducted on the DIV2K dataset reveal that BUFF achieves a notable improvement, with a +0.61 increase compared to baseline in SSIM on BSD100, surpassing traditional diffusion approaches by an average additional +0.20dB PSNR gain. These findings underscore the potential of Bayesian methods in enhancing diffusion processes for SR, paving the way for future advancements in the *** Codes 68T45 © 2025, CC BY.

关键词： Image enhancement

Cheap and quick: efficient vision-language instruction tuning for large language models 23

学校读者我要写书评

暂无评论

Cheap and quick: efficient vision-language instruction tunin...

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Gen Luo Yiyi Zhou Tianhe Ren Shengxin Chen Xiaoshuai Sun Rongrong Ji Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University P.R. China and Peng Cheng Laboratory Shenzhen China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University P.R. China and Institute of Artificial Intelligence Xiamen University P.R. China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University P.R. China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University P.R. China and Institute of Artificial Intelligence Xiamen University P.R. China and Peng Cheng Laboratory Shenzhen China

Recently, growing interest has been aroused in extending the multimodal capability of large language models (LLMs), e.g., vision-language (VL) learning, which is regarded as the next milestone of artificial general intelligence. However, existing solutions are prohibitively expensive, which not only need to optimize excessive parameters, but also require another large-scale pre-training before VL instruction tuning. In this paper, we propose a novel and affordable solution for the effective VL adaption of LLMs, called Mixture-of-Modality Adaptation (MMA). Instead of using large neural networks to connect the image encoder and LLM, MMA adopts lightweight modules, i.e., adapters, to bridge the gap between LLMs and VL tasks, which also enables the joint optimization of the image and language models. Meanwhile, MMA is also equipped with a routing algorithm to help LLMs achieve an automatic shift between single- and multi-modal instructions without compromising their ability of natural language understanding. To validate MMA, we apply it to a recent LLM called LLaMA and term this formed large vision-language instructed model as LaVIN. To validate MMA and LaVIN, we conduct extensive experiments under two setups, namely multimodal science question answering and multimodal dialogue. The experimental results not only demonstrate the competitive performance and the superior training efficiency of LaVIN than existing multimodal LLMs, but also confirm its great potential as a general-purpose chatbot. More importantly, the actual expenditure of LaVIN is extremely cheap, e.g., only 1.4 training hours with 3.8M trainable parameters, greatly confirming the effectiveness of MMA. Our project is released at https://***/lavin.

关键词：

EALink: An efficient and Accurate Pre-trained Framework for Issue-Commit Link Recovery 23

学校读者我要写书评

暂无评论

EALink: An Efficient and Accurate Pre-trained Framework for ...

Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering

作者： Chenyuan Zhang Yanlin Wang Zhao Wei Yong Xu Juhong Wang Hui Li Rongrong Ji Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University China School of Software Engineering Sun Yat-sen University China Tencent China

ISBN: (纸本)9798350329964

Issue-commit links, as a type of software traceability links, play a vital role in various software development and maintenance tasks. However, they are typically deficient, as developers often forget or fail to create tags when making commits. Existing studies have deployed deep learning techniques, including pre-trained models, to improve automatic issue-commit link recovery. Despite their promising performance, we argue that previous approaches have four main problems, hindering them from recovering links in large software projects. To overcome these problems, we propose an efficient and accurate pre-trained framework called EALink for issue-commit link recovery. EALink requires much fewer model parameters than existing pre-trained methods, bringing efficient training and recovery. Moreover, we design various techniques to improve the recovery accuracy of EALink. We construct a large-scale dataset and conduct extensive experiments to demonstrate the power of EALink. Results show that EALink outperforms the state-of-the-art methods by a large margin (15.23%-408.65%) on various evaluation metrics. Meanwhile, its training and inference overhead is orders of magnitude lower than existing methods. We provide our implementation and data at https://***/KDEGroup/EALink.

关键词： issue-commit link recovery

CamoTeacher: Dual-Rotation Consistency Learning for Semi-Supervised Camouflaged Object Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lai, Xunfa Yang, Zhiyu Hu, Jie Zhang, Shengchuan Cao, Liujuan Jiang, Guannan Wang, Zhiyu Zhang, Songan Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Contemporary Amperex Technology Co. Limited China Shanghai Jiao Tong University China

Existing camouflaged object detection (COD) methods depend heavily on large-scale pixel-level annotations. However, acquiring such annotations is laborious due to the inherent camouflage characteristics of the objects. Semi-supervised learning offers a promising solution to this challenge. Yet, its application in COD is hindered by significant pseudo-label noise, both pixel-level and instance-level. We introduce CamoTeacher, a novel semi-supervised COD framework, utilizing Dual-Rotation Consistency Learning (DRCL) to effectively address these noise issues. Specifically, DRCL minimizes pseudo-label noise by leveraging rotation views’ consistency in pixel-level and instance-level. First, it employs Pixel-wise Consistency Learning (PCL) to deal with pixel-level noise by reweighting the different parts within the pseudo-label. Second, Instance-wise Consistency Learning (ICL) is used to adjust weights for pseudo-labels, which handles instance-level noise. Extensive experiments on four COD benchmark datasets demonstrate that the proposed CamoTeacher not only achieves state-of-the-art compared with semi-supervised learning methods, but also rivals established fully-supervised learning methods. Our code will be available soon. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Mi, Peng Shen, Li Ren, Tianhe Zhou, Yiyi Xu, Tianshuo Sun, Xiaoshuai Liu, Tongliang Ji, Rongrong Tao, Dacheng The Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China JD Explire Adacemy Beijing China The University of Sydney Australia

Deep neural networks often suffer from poor generalization due to complex and non-convex loss landscapes. Sharpness-Aware Minimization (SAM) is a popular solution that smooths the loss landscape by minimizing the maximized change of training loss when adding a perturbation to the weight. However, indiscriminate perturbation of SAM on all parameters is suboptimal and results in excessive computation, double the overhead of common optimizers like Stochastic Gradient Descent (SGD). In this paper, we propose Sparse SAM (SSAM), an efficient and effective training scheme that achieves sparse perturbation by a binary mask. To obtain the sparse mask, we provide two solutions based on Fisher information and dynamic sparse training, respectively. We investigate the impact of different masks, including unstructured, structured, and N:M structured patterns, as well as explicit and implicit forms of implementing sparse perturbation. We theoretically prove that SSAM can converge at the same rate as SAM, i.e., O(log T/√T). Sparse SAM has the potential to accelerate training and smooth the loss landscape effectively. Extensive experimental results on CIFAR and ImageNet-1K confirm that our method is superior to SAM in terms of efficiency, and the performance is preserved or even improved with a perturbation of merely 50% sparsity. Code is available at https://***/Mi-Peng/Systematic-Investigation-of-Sparse-Perturbed-Sharpness-Aware-Minimization-Optimizer. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep neural networks

STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Zhenglin Li, Huaxia Liu, Hong Wang, Nanyang Yu, Gang Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Tencent PCG China National Institute of Informatics Japan

Recently, deep learning-based facial landmark detection has achieved significant improvement. However, the semantic ambiguity problem degrades detection performance. Specifically, the semantic ambiguity causes inconsistent annotation and negatively affects the model’s convergence, leading to worse accuracy and instability prediction. To solve this problem, we propose a Self-adapTive Ambiguity Reduction (STAR) loss by exploiting the properties of semantic ambiguity. We find that semantic ambiguity results in the anisotropic predicted distribution, which inspires us to use predicted distribution to represent semantic ambiguity. Based on this, we design the STAR loss that measures the anisotropism of the predicted distribution. Compared with the standard regression loss, STAR loss is encouraged to be small when the predicted distribution is anisotropic and thus adaptively mitigates the impact of semantic ambiguity. Moreover, we propose two kinds of eigenvalue restriction methods that could avoid both distribution’s abnormal change and the model’s premature convergence. Finally, the comprehensive experiments demonstrate that STAR loss outperforms the state-of-the-art methods on three benchmarks, i.e., COFW, 300W, and WFLW, with negligible computation overhead. Code is at https://***/ZhenglinZhou/STAR © 2023, CC BY.

关键词： Eigenvalues and eigenfunctions

Graph Contrastive Learning with Decoupled Augmentation

学校读者我要写书评

暂无评论

Graph Contrastive Learning with Decoupled Augmentation

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Gao, Shihao Li, Caoshuo Mao, Cunli Zhang, Xulong Qu, Xiaoyang Jin, Taisong Wang, Jianzong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University China Faculty of Information Engineering and Automation Kunming University of Science and Technology China Yunnan Key Laboratory of Artificial Intelligence Kunming University of Science and Technology China Key Laboratory of Oracle Bone Inscriptions Information Processing Ministry of Education of China Anyang Normal University China Ping An Technology Co. Ltd. Shenzhen China

ISBN: (纸本)9798350368741

Graph contrastive learning based on augmentation strategies has recently demonstrated remarkable performance. Existing methods typically jointly leverage attribute and structural augmentations to generate graph views, learning data invariance information through contrasting sample pairs. However, this joint approach may deviate from the expectation of semantically similar before and after augmentation. The propagation of attribute information in graphs usually occurs through their structure, meaning that structural and attribute augmentations can interfere with each other and potentially distort the graph's semantics. To address this, we propose a decoupled augmentation framework for graph contrastive learning, which eliminates the mutual interference between the two levels of augmentation while fully exploring graph information. Specifically, our framework employs separate encoders to learn data invariance under different augmentation levels, and it considers the positive gains generated between these levels. Experimental results on five public datasets show that the proposed method is more competitive than state-of-the-art approaches. © 2025 IEEE.

关键词： Contrastive learning Graph representation Self-supervised learning

Memory-Reduced Meta-Learning with Guaranteed Convergence

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Honglin Ma, Ji Yu, Xiao Department of Automation Xiamen University Xiamen China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen China Institute of Artificial Intelligence Xiamen University Xiamen China

The optimization-based meta-learning approach is gaining increased traction because of its unique ability to quickly adapt to a new task using only small amounts of data. However, existing optimization-based meta-learning approaches, such as MAML, ANIL and their variants, generally employ backpropagation for upper-level gradient estimation, which requires using historical lower-level parameters/gradients and thus increases computational and memory overhead in each iteration. In this paper, we propose a meta-learning algorithm that can avoid using historical parameters/gradients and significantly reduce memory costs in each iteration compared to existing optimization-based meta-learning approaches. In addition to memory reduction, we prove that our proposed algorithm converges sublinearly with the iteration number of upper-level optimization, and the convergence error decays sublinearly with the batch size of sampled tasks. In the specific case in terms of deterministic meta-learning, we also prove that our proposed algorithm converges to an exact solution. Moreover, we quantify that the computational complexity of the algorithm is on the order of O(ϵ−1), which matches existing convergence results on meta-learning even without using any historical parameters/gradients. Experimental results on meta-learning benchmarks confirm the efficacy of our proposed algorithm. Copyright © 2024, The Authors. All rights reserved.

关键词：

efficient Dataset Distillation through Low-Rank Space Sampling

学校读者我要写书评

暂无评论

Efficient Dataset Distillation through Low-Rank Space Sampli...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Hangyang Kong Wenbo Zhou Xuxiang He Xiaotong Tu Xinghao Ding Institute of Artificial Intelligence Xiamen University Xiamen China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University Xiamen China School of Informatics Xiamen University Xiamen China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Huge amount of data is the key of the success of deep learning, however, redundant information impairs the generalization ability of the model and increases the burden of calculation. Dataset Distillation (DD) compresses the original dataset into a smaller but representative subset for high-quality data and efficient training strategies. Existing works for DD generate synthetic images by treating each image as an independent entity, thereby overlooking the common features among data. This paper proposes a dataset distillation method based on Matching Training Trajectories with Low-rank Space Sampling(MTT-LSS), which uses low-rank approximations to capture multiple low-dimensional manifold subspaces of the original data. The synthetic data is represented by basis vectors and shared dimension mappers from these subspaces, reducing the cost of generating individual data points while effectively minimizing information redundancy. The proposed method is tested on CIFAR-10, CIFAR-100, and SVHN datasets, and outperforms the baseline methods by an average of 9.9%.

关键词： Training Manifolds Costs Redundancy Noise Signal processing algorithms Vectors Trajectory Speech processing Synthetic data