检索结果-内蒙古大学图书馆

DuMo: Dual Encoder Modulation Network for Precise Concept Erasure

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Han, Feng Chen, Kai Gong, Chao Wei, Zhipeng Chen, Jingjing Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China

The exceptional generative capability of text-to-image models has raised substantial safety concerns regarding the generation of Not-Safe-For-Work (NSFW) content and potential copyright infringement. To address these concerns, previous methods safeguard the models by eliminating inappropriate concepts. Nonetheless, these models alter the parameters of the backbone network and exert considerable influences on the structural (low-frequency) components of the image, which undermines the model’s ability to retain non-target concepts. In this work, we propose our Dual encoder Modulation network (DuMo), which achieves precise erasure of inappropriate target concepts with minimum impairment to non-target concepts. In contrast to previous methods, DuMo employs the Eraser with PRior Knowledge (EPR) module which modifies the skip connection features of the U-NET and primarily achieves concept erasure on details (high-frequency) components of the image. To minimize the damage to non-target concepts during erasure, the parameters of the backbone U-NET are frozen and the prior knowledge from the original skip connection features is introduced to the erasure process. Meanwhile, the phenomenon is observed that distinct erasing preferences for the image structure and details are demonstrated by the EPR at different timesteps and layers. Therefore, we adopt a novel Time-Layer MOdulation process (TLMO) that adjusts the erasure scale of EPR module’s outputs across different layers and timesteps, automatically balancing the erasure effects and model’s generative ability. Our method achieves state-of-the-art performance on Explicit Content Erasure (detecting only 34 nude parts), Cartoon Concept Removal (with an average LPIPSda of 0.428, 0.113 higher than SOTA at 0.315), and Artistic Style Erasure (with an average LPIPSda of 0.387, 0.088 higher than SOTA at 0.299), clearly outperforming alternative methods. Code is availab.e at https://***/Maplebb/DuMo Copyright © 2025, The Author

关键词： HTTP

BlockDance: Reuse Structurally Similar Spatio-Temporal Features to Accelerate Diffusion Transformers

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Hui Gao, Tingwei Shao, Jie Wu, Zuxuan Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China ByteDance Intelligent Creation China

Diffusion models have demonstrated impressive generation capabilities, particularly with recent advancements leveraging transformer architectures to improve both visual and artistic quality. However, Diffusion Transformers (DiTs) continue to encounter challenges related to low inference speed, primarily due to the iterative denoising process. To address this issue, we propose BlockDance, a training-free approach that explores feature similarities at adjacent time steps to accelerate DiTs. Unlike previous feature-reuse methods that lack tailored reuse strategies for features at different scales, BlockDance prioritizes the identification of the most structurally similar features, referred to as Structurally Similar Spatio-Temporal (STSS) features. These features are primarily located within the structure-focused blocks of the transformer during the later stages of denoising. BlockDance caches and reuses these highly similar features to mitigate redundant computation, thereby accelerating DiTs while maximizing consistency with the generated results of the original model. Furthermore, considering the diversity of generated content and the varying distributions of redundant features, we introduce BlockDance-Ada, a lightweight decision-making network tailored for instance-specific acceleration. BlockDance-Ada dynamically allocates resources and provides superior content quality. Both BlockDance and BlockDance-Ada have proven effective across various generation tasks and models, achieving accelerations between 25% and 50% while maintaining generation quality. Copyright © 2025, The Authors. All rights reserved.

关键词： Decision making

Achieving More with Less: Additive Prompt Tuning for Rehearsal-Free Class-Incremental Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Haoran Wang, Ping Zhou, Zihan Zhang, Xu Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China APUS AI Lab China

Class-incremental learning (CIL) enables models to learn new classes progressively while preserving knowledge of previously learned ones. Recent advances in this field have shifted towards parameter-efficient fine-tuning techniques, with many approaches building upon the framework that maintains a pool of learnable prompts. Although effective, these methods introduce substantial computational overhead, primarily due to prompt pool querying and increased input sequence lengths from prompt concatenation. In this work, we present a novel prompt-based approach that addresses this limitation. Our method trains a single set of shared prompts across all tasks and, rather than concatenating prompts to the input, directly modifies the CLS token’s attention computation by adding the prompts to it. This simple and lightweight design not only significantly reduces computational complexity—both in terms of inference costs and the number of trainable parameters—but also eliminates the need to optimize prompt lengths for different downstream tasks, offering a more efficient yet powerful solution for rehearsal-free class-incremental learning. Extensive experiments across a diverse range of CIL benchmarks demonstrate the effectiveness of our approach, highlighting its potential to establish a new prompt-based CIL paradigm. Furthermore, experiments on general recognition benchmarks beyond the CIL setting also show strong performance, positioning our method as a promising candidate for a general parameter-efficient fine-tuning approach. Copyright © 2025, The Authors. All rights reserved.

关键词： Contrastive Learning

EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Zihao Chen, Haoran Zhao, Haoyu Lu, Guansong Fu, Yanwei Xu, Hang Wu, Zuxuan Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Noah’s Ark Lab Huawei Canada

Handling complex or nonlinear motion patterns has long posed challenges for video frame interpolation. Although recent advances in diffusion-based methods offer improvements over traditional optical flow-based approaches, they still struggle to generate sharp, temporally consistent frames in scenarios with large motion. To address this limitation, we introduce EDEN, an Enhanced Diffusion for high-quality large-motion vidEo frame iNterpolation. Our approach first utilizes a transformer-based tokenizer to produce refined latent representations of the intermediate frames for diffusion models. We then enhance the diffusion transformer with temporal attention across the process and incorporate a start-end frame difference embedding to guide the generation of dynamic motion. Extensive experiments demonstrate that EDEN achieves state-of-the-art results across popular benchmarks, including nearly a 10% LPIPS reduction on DAVIS and SNU-FILM, and an 8% improvement on DAIN-HD. Copyright © 2025, The Authors. All rights reserved.

关键词： Optical flows

Pix2Cap-COCO: Advancing Visual Comprehension via Pixel-Level Captioning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： You, Zuyao Wang, Junke Kong, Lingyu He, Bo Wu, Zuxuan Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China University of Maryland College Park United States

(Image Present) We present Pix2Cap-COCO, the first panoptic pixel-level caption dataset designed to advance fine-grained visual understanding. To achieve this, we carefully design an automated annotation pipeline that prompts GPT-4V to generate pixel-aligned, instance-specific captions for individual objects within images, enabling models to learn more granular relationships between objects and their contexts. This approach results in 167,254 detailed captions, with an average of 22.94 words per caption. Building on Pix2Cap-COCO, we introduce a novel task, panoptic segmentation-captioning, which challenges models to recognize instances in an image and provide detailed descriptions for each simultaneously. To benchmark this task, we design a robust baseline based on X-Decoder. The experimental results demonstrate that Pix2Cap-COCO is a particularly challenging dataset, as it requires models to excel in both fine-grained visual understanding and detailed language generation. Furthermore, we leverage Pix2Cap-COCO for Supervised Fine-Tuning (SFT) on large multimodal models (LMMs) to enhance their performance. For example, training with Pix2Cap-COCO significantly improves the performance of GPT4RoI, yielding gains in CIDEr (+1.4%), ROUGE (+0.4%), and SPICE (+0.5%) on Visual Genome dataset, and strengthens its region understanding ability on the ViP-Bench, with an overall improvement of +5.1%, including notable increases in recognition accuracy (+11.2%) and language generation quality (+22.2%). Code is availab.e at https://***/geshang777/pix2cap. © 2025, CC BY-NC-SA.

关键词： Pixels

UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encoding

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Jiao, Yang Qiu, Haibo Jie, Zequn Chen, Shaoxiang Chen, Jingjing Ma, Lin Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Meituan China

We introduce UniToken, an auto-regressive generation model that encodes visual inputs through a combination of discrete and continuous representations, enabling seamless integration of unified visual understanding and image generation tasks. Unlike previous approaches that rely on unilateral visual representations, our unified visual encoding framework captures both high-level semantics and low-level details, delivering multidimensional info.mation that empowers heterogeneous tasks to selectively assimilate domain-specific knowledge based on their inherent characteristics. Through in-depth experiments, we uncover key principles for developing a unified model capable of both visual understanding and image generation. Extensive evaluations across a diverse range of prominent benchmarks demonstrate that UniToken achieves state-of-the-art performance, surpassing existing approaches. These results establish UniToken as a robust foundation for future research in this domain. The code and models are availab.e at https://***/SxJyJay/UniToken. © 2025, CC BY-NC-SA.

关键词： Knowledge acquisition

An inheritable clustering algorithm suited for parameter changing

学校读者我要写书评

暂无评论

An inheritable clustering algorithm suited for parameter cha...

Proceedings of 2004 International Conference on Machine Learning and Cybernetics

作者： Li, Fei Liu, Shang Dou, Zhi-Tong Huang, Ya-Lou Lab. of Intelligent Info. Processing Coll. of Info. Sci. and Technology NanKai University Tianjin 300071 China

ISBN: (纸本)0780384032

DBSCAN is a classic density based algorithm and it clusters the data set according to the user input parameters. This paper investigates how to inherit the mining results of last time when parameters change. A new incremental clustering algorithm IPC_DBSCAN is proposed, which gets the same result as that of rerunning DBSCAN yet high efficiency is achieved. Theoretical analysis and experimental results show that the proposed method reduces search space greatly and has novel efficiency. By interaction, IPC_DBSCAN gets the most satisfying result quickly and especially suits large volume data set.

关键词： Algorithms

A new ant colony clustering algorithm based on DBSCAN

学校读者我要写书评

暂无评论

A new ant colony clustering algorithm based on DBSCAN

Proceedings of 2004 International Conference on Machine Learning and Cybernetics

作者： Liu, Shang Dou, Zhi-Tong Li, Fei Huang, Ya-Lou Lab. of Intelligent Info. Processing Coll. of Info. Sci. and Technology NanKai University Tianjin 300071 China

ISBN: (纸本)0780384032

The AntClass algorithm is a new algorithm applying ant colony clustering algorithm to cluster analysis, and the result is satisfying. To attack the slow speed of the AntClass algorithm, a new algorithm named DBAntCluster is proposed. First the high density clusters are got in the dataset by using DBSCAN algorithm, and then these high density clusters are scattered in the grid board as a special kind of data object with other single data objects in the dataset in DBAntCluster algorithm, the ants can avoid many unnecessary movements by using the data attribute of density and distribution well, and the speed is greatly accelerated. This improvement is validated in our experiments.

关键词： Algorithms

Abstractive multi-document summarization with semantic info.mation extraction

学校读者我要写书评

暂无评论

Abstractive multi-document summarization with semantic infor...

Conference on Empirical Methods in Natural Language processing, EMNLP 2015

作者： Li, Wei Key Lab of Intelligent Info. Processing Institute of Computing Technology CAS Beijing100190 China

ISBN: (纸本)9781941643327

This paper proposes a novel approach to generate abstractive summary for multiple documents by extracting semantic info.mation from texts. The concept of Basic Semantic Unit (BSU) is defined to describe the semantics of an event or action. A semantic link network on BSUs is constructed to capture the semantic info.mation of texts. Summary structure is planned with sentences generated based on the semantic link network. Experiments demonstrate that the approach is effective in generating info.mative, coherent and compact summary. © 2015 Association for Computational Linguistics.

关键词： Semantics