检索结果-内蒙古大学图书馆

Scaling Laws for Data-Efficient Visual Transfer Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yang, Wenxuan Wei, Qingqu Ma, Chenxi Tan, Weimin Yan, Bo Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

Current scaling laws for visual AI models focus predominantly on large-scale pretraining, leaving a critical gap in understanding how performance scales for data-constrained downstream tasks. To address this limitation, this paper establishes the first practical framework for data-efficient scaling laws in visual transfer learning, addressing two fundamental questions: 1) How do scaling behaviors shift when downstream tasks operate with limited data? 2) What governs the efficacy of knowledge distillation under such constraints? Through systematic analysis of vision tasks across data regimes (1K–1M samples), we propose the distillation boundary theory, revealing a critical turning point in distillation efficiency: 1) Distillation superiority: In data-scarce conditions, distilled models significantly outperform their non-distillation counterparts, efficiently leveraging inherited knowledge to compensate for limited training samples. 2) Pre-training dominance: As pretraining data increases beyond a critical threshold, non-distilled models gradually surpass distilled versions, suggesting diminishing returns from knowledge inheritance when sufficient task-specific data becomes available. Empirical validation across various model scales (2.5M to 38M parameters) and data volumes demonstrate these performance inflection points, with error difference curves transitioning from positive to negative values at critical data thresholds, confirming our theoretical predictions. This work redefines scaling laws for data-limited regimes, bridging the knowledge gap between large-scale pretraining and practical downstream adaptation, addressing a critical barrier to understanding vision model scaling behaviors and optimizing computational resource allocation. Copyright © 2025, The Authors. All rights reserved.

关键词： Resource allocation

Explainable Synthetic Image Detection through Diffusion Timestep Ensembling

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wu, Yixin Zhang, Feiran Shi, Tianyuan Yin, Ruicheng Wang, Zhenghua Gan, Zhenliang Wang, Xiaohua Lv, Changze Zheng, Xiaoqing Huang, Xuanjing School of Computer Science Fudan University Shanghai Key Laboratory of Intelligent Information Processing Shanghai China

Recent advances in diffusion models have enabled the creation of deceptively real images, posing significant security risks when misused. In this study, we reveal that natural and synthetic images exhibit distinct differences in the high-frequency domains of their Fourier power spectra after undergoing iterative noise perturbations through an inverse multi-step denoising process, suggesting that such noise can provide additional discriminative information for identifying synthetic images. Based on this observation, we propose a novel detection method that amplifies these differences by progressively adding noise to the original images across multiple timesteps, and train an ensemble of classifiers on these noised images. To enhance human comprehension, we introduce an explanation generation and refinement module to identify flaws located in AI-generated images. Additionally, we construct two new datasets, GenHard and GenExplain, derived from the GenImage benchmark, providing detection samples of greater difficulty and high-quality rationales for fake images. Extensive experiments show that our method achieves state-of-the-art performance with 98.91% and 95.89% detection accuracy on regular and harder samples, increasing a minimal of 2.51% and 3.46% compared to baselines. Furthermore, our method also generalizes effectively to images generated by other diffusion models. Our code and datasets will be made publicly available. Copyright © 2025, The Authors. All rights reserved.

关键词： Inverse problems

Principled Design of Translation, Scale, and Rotation Invariant Variation Operators for Metaheuristics

学校读者我要写书评

暂无评论

Chinese Journal of Electronics 2023年第1期32卷 111-129页

作者： TIAN Ye ZHANG Xingyi HE Cheng TAN Kay Chen JIN Yaochu Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education Institutes of Physical Science and Information Technology Anhui University School of Artificial Intelligence Anhui University School of Electrical and Electronic Engineering Huazhong University of Science and Technology Department of Computing The Hong Kong Polytechnic University Faculty of Technology Bielefeld University

A large number of metaheuristics have been proposed and shown high performance in solving complex optimization problems. While most variation operators in existing metaheuristics are empirically designed,new operators are automatically designed in this work,which are expected to be search space independent and thus exhibit robust performance on different *** work first investigates the influence of translation invariance, scale invariance, and rotation invariance on the search behavior and performance of some representative operators. This work then deduces the generic form of translation, scale, and rotation invariant operators, and proposes a principled approach for the automated design of operators, which searches for high-performance operators based on the deduced generic form. The experimental results demonstrate that the operators generated by the proposed approach outperform state-of-the-art ones on a variety of problems with complex landscapes and up to 1000decision variables.

关键词： Deep learning Metaheuristics Reinforcement learning Benchmark testing Search problems Robustness Behavioral sciences

Visualization facilitates uncertainty evaluation of multiple-point geostatistical stochastic simulation

学校读者我要写书评

暂无评论

Visual Intelligence 2023年第1期1卷 1-18页

作者： Huang, Qianhong Chen, Qiyu Liu, Gang Cui, Zhesi School of Computer Science China University of Geosciences Wuhan 430074 China State Key Laboratory of Biogeology and Environmental Geology China University of Geosciences Wuhan 430074 China Hubei Key Laboratory of Intelligent Geo-Information Processing China University of Geosciences Wuhan 430074 China

Stochastic simulation is an essential method for modeling complex geological structures in geosciences. Evaluating the uncertainty of the realizations of stochastic simulations can better describe real phenomena. However, uncertainty evaluation of stochastic simulation methods remains a challenge due to the limited data from geological surveys and the uncertainty in reliability estimation with stochastic simulation models. In addition, understanding the sensitivity of the parameters in stochastic simulation models is invaluable when exploring the parameters with a higher influence on the uncertainty associated with predictions generated from stochastic simulation. To facilitate uncertainty evaluation in stochastic simulation methods, we use the circular treemap as an interactive workflow to explore prediction uncertainty in and the parameter sensitivity of multiple-point geostatistical (MPS) stochastic simulation methods. In this work, we present a novel visualization framework for assessing the uncertainty in MPS stochastic simulation methods and exploring the parameter sensitivity of the MPS methods. We present a new indicator to integrate multiple metrics that characterize geospatial features and visualize these metrics to assist domain experts in making decisions. Parallel coordinates-scatter matrix plots and multi-dimensional scaling (MDS) plots are used to analyze the parametric sensitivity of MPS stochastic simulation methods. The realizations and parameters of two MPS stochastic simulation methods are used to test the applicability of the proposed visualization workflow and the visualization methods. The results demonstrate that our workflow and the visualization methods can assist experts in finding the model with less uncertainty and improve the efficiency of parameter adjustment using different MPS stochastic simulation methods. © The Author(s) 2023.

关键词： Circular treemap Multiple-point geostatistics Parallel coordinates-scatter matrix plots Stochastic simulations Uncertainty evaluation

TOWARDS GENERATIVE ABSTRACT REASONING: COMPLETING RAVEN’S PROGRESSIVE MATRIX VIA RULE ABSTRACTION AND SELECTION

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Shi, Fan Li, Bin Xue, Xiangyang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University China

Endowing machines with abstract reasoning ability has been a long-term research topic in artificial intelligence. Raven’s Progressive Matrix (RPM) is widely used to probe abstract visual reasoning in machine intelligence, where models will analyze the underlying rules and select one image from candidates to complete the image matrix. Participators of RPM tests can show powerful reasoning ability by inferring and combining attribute-changing rules and imagining the missing images at arbitrary positions of a matrix. However, existing solvers can hardly manifest such an ability in realistic RPM tests. In this paper, we propose a deep latent variable model for answer generation problems through Rule AbstractIon and SElection (RAISE). RAISE can encode image attributes into latent concepts and abstract atomic rules that act on the latent concepts. When generating answers, RAISE selects one atomic rule out of the global knowledge set for each latent concept to constitute the underlying rule of an RPM. In the experiments of bottom-right and arbitrary-position answer generation, RAISE outperforms the compared solvers in most configurations of realistic RPM datasets. In the odd-one-out task and two held-out configurations, RAISE can leverage acquired latent concepts and atomic rules to find the rule-breaking image in a matrix and handle problems with unseen combinations of rules and attributes. Copyright © 2024, The Authors. All rights reserved.

关键词： Abstracting

RGB-guided hyperspectral image super-resolution with deep progressive learning

学校读者我要写书评

暂无评论

CAAI Transactions on Intelligence Technology 2024年第3期9卷 679-694页

作者： Tao Zhang Ying Fu Liwei Huang Siyuan Li Shaodi You Chenggang Yan School of Computer Science and Technology Beijing Institute of TechnologyBeijingChina Satellite Information Intelligent Processing and Application Research Laboratory Beijing Institute of Remote SensingBeijingChina Key Laboratory of Spectral Imaging Technology Xi'an Institute of Optics and Precision Mechanics of Chinese Academy of SciencesXi'anChina Institute of Informatics University of AmsterdamAmsterdamNetherlands School of Communication Engineering Hangzhou Dianzi UniversityHangzhouChina

Due to hardware limitations,existing hyperspectral(HS)camera often suffer from low spatial/temporal ***,it has been prevalent to super-resolve a low reso-lution(LR)HS image into a high resolution(HR)HS image with a HR RGB(or mul-tispectral)image *** approaches for this guided super-resolution task often model the intrinsic characteristic of the desired HR HS image using hand-crafted ***,researchers pay more attention to deep learning methods with direct supervised or unsupervised learning,which exploit deep prior only from training dataset or testing *** this article,an efficient convolutional neural network-based method is presented to progressively super-resolve HS image with RGB image ***-ically,a progressive HS image super-resolution network is proposed,which progressively super-resolve the LR HS image with pixel shuffled HR RGB image ***,the super-resolution network is progressively trained with supervised pre-training and un-supervised adaption,where supervised pre-training learns the general prior on training data and unsupervised adaptation generalises the general prior to specific prior for variant testing *** proposed method can effectively exploit prior from training dataset and testing HS and RGB images with spectral-spatial *** has a good general-isation capability,especially for blind HS image *** experimental results show that the proposed deep progressive learning method out-performs the existing state-of-the-art methods for HS image super-resolution in non-blind and blind cases.

关键词： computer vision deep neural networks image processing image resolution unsupervised learning

Optimizing V-information for Self-Supervised Pre-training Data-Effective Medical Foundation Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Wenxuan Zhang, Hanyu Tan, Weimin Sun, Yuqi Yan, Bo Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University China

Self-supervised pre-training medical foundation models on large-scale datasets demonstrate exceptional performance. However, recent research questions this traditional notion, exploring whether an increase in pre-training data always leads to enhanced model performance. To address this issue, data-effective learning approaches have been introduced to select valuable samples for foundation model pretraining. Notably, current methods in this area lack a clear standard for sample selection, and the underlying theoretical foundation remains unknown. As the first attempt to address this limitation, we leverage V-information in self-supervised pre-training of foundation models. Our theoretical derivation confirms that by optimizing V-information, sample selection can be framed as an optimization problem where choosing diverse and challenging samples enhances model performance even under limited training data. Under this guidance, we develop an optimal data-effective learning method (OptiDEL) to optimize V-information in real-world medical domains. The OptiDEL method generates more diverse and harder samples to achieve or even exceed the performance of models trained on the full dataset while using substantially less data. We compare the OptiDEL method with state-of-the-art approaches finding that OptiDEL consistently outperforms existing approaches across eight different datasets, with foundation models trained on only 5% of the pre-training data surpassing the performance of those trained on the full dataset. The code can be accessed at GitHub Repository. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

Specific Emitter Identification Based on ACGAN and STFT 7

学校读者我要写书评

暂无评论

Specific Emitter Identification Based on ACGAN and STFT

7th International Conference on Advanced Algorithms and Control Engineering, ICAACE 2024

作者： Wang, Bin Xie, Jun Wang, Feng The 38th Research Institute Of China Electronics Technology Group Corporation China Key Laboratory Of Aperture Array And Space Application China Key Laboratory Of Intelligent Information Processing Anhui Hefei230088 China School Of Computer And Information Hefei University Of Technology Hefei230601 China

ISBN: (纸本)9798350361445

Deep learning methodologies are gaining popularity for enhancing the efficiency of Specific Emitter Identification (SEI). Nevertheless, the lack of labeled datasets continues to pose a challenge in most non-cooperative scenarios for SEI. Addressing this challenge, a framework for SEI is proposed, leveraging Short-Time Fourier Transform (STFT) in conjunction with auxiliary classifier GAN(ACGAN). The Short-Time Fourier Transform of emitters is utilized as radio-frequency fingerprint for the emitters. The input latent layer is infused with label information to steer the training process of the Generative Adversarial Network. Through semi-supervised training, shared weight updates are implemented for the classification network. Experimental results demonstrate that our model can achieve superior or competitive performances in several datasets compared with the former models. © 2024 IEEE.

关键词： Deep learning

Broadband High Gain Composite VHF Monopole Antenna Design 6

学校读者我要写书评

暂无评论

Broadband High Gain Composite VHF Monopole Antenna Design

6th IEEE International Conference on Electronic Information and Communication Technology, ICEICT 2023

作者： Wang, Lihua Cao, Baofeng Li, Peng Li, Zongxiang Li, Yingsong College of Information and Communication Engineering Harbin Engineering University Harbin150001 China State Key Laboratory of Nuclear and Biochemical Calamity Protection for Civilian Beijing102205 China Anhui University Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education Hefei203601 China

ISBN: (纸本)9798350399059

A novel wideband miniaturized composite monopole antenna is proposed, analysed and investigated in detail, where the antenna consists of a conical monopole structure with a loaded capacitive disc. The antenna is composed of multiple nested section, aiming to reduce the profile of the antenna. The loading disc at its top is equivalent to a capacitive structure to improve the antenna matching and the stability of the pattern. According to the simulation results, the impedance bandwidth (S11 © 2023 IEEE.

关键词： Monopole antennas