检索结果-内蒙古大学图书馆

ControlVideo: conditional control for one-shot text-driven video editing and beyond

Science China(Information Sciences) 2025年第3期68卷 150-162页

作者： Min ZHAO Rongzhen WANG Fan BAO Chongxuan LI Jun ZHU Department of Computer Science and Technology Institute for AI Tsinghua-Bosch Joint ML CenterTsinghua Laboratory of Brain and Intelligence Lab Tsinghua University ShengShu Technology Gaoling School of Artificial Intelligence Renmin University of China Beijing Key Laboratory of Big Data Management and Analysis Methods Pazhou Laboratory (Huangpu)

This paper presents ControlVideo for text-driven video editing — generating a video that aligns with a given text while preserving the structure of the source video. Building on a pre-trained text-to-image diffusion model, ControlVideo enhances the fidelity and temporal consistency by incorporating additional conditions(such as edge maps), and fine-tuning the key-frame and temporal attention on the source video-text pair via an in-depth exploration of the design space. Extensive experimental results demonstrate that ControlVideo outperforms various competitive baselines by delivering videos that exhibit high fidelity w.r.t. the source content, and temporal consistency, all while aligning with the text. By incorporating low-rank adaptation layers into the model before training, ControlVideo is further empowered to generate videos that align seamlessly with reference images. More importantly, ControlVideo can be readily extended to the more challenging task of long video editing(e.g., with hundreds of frames), where maintaining long-range temporal consistency is crucial. To achieve this, we propose to construct a fused ControlVideo by applying basic ControlVideo to overlapping short video segments and key frame videos and then merging them by pre-defined weight functions. Empirical results validate its capability to create videos across 140 frames, which is approximately 5.83 to 17.5 times more than what previous studies achieved. The code is available at https://***/thu-ml/controlvideo.

关键词： diffusion models controllable generation text-driven editing video editing long video editing

来源：评论

学校读者我要写书评

暂无评论

Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing

Uniform Attention Maps: Boosting Image Fidelity in Reconstru...

引用

2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025

作者： Mo, Wenyi Zhang, Tianyu Bai, Yalong Su, Bing Wen, Ji-Rong Gaoling School of Artificial Intelligence Renmin University of China China Beijing Key Laboratory of Big Data Management and Analysis Methods China Du Xiaoman Technology China

ISBN: (纸本)9798331510831

Text-guided image generation and editing using diffusion models have achieved remarkable advancements. Among these, tuning-free methods have gained attention for their ability to perform edits without extensive model adjustments, offering simplicity and efficiency. However, existing tuning-free approaches often struggle with balancing fidelity and editing precision. Reconstruction errors in DDIM Inversion are partly attributed to the cross-attention mechanism in U-Net, which introduces misalignments during the inversion and reconstruction process. To address this, we analyze reconstruction from a structural perspective and propose a novel approach that replaces traditional cross-attention with uniform attention maps, significantly enhancing image reconstruction fidelity. Our method effectively minimizes distortions caused by varying text conditions during noise prediction. To complement this improvement, we introduce an adaptive mask-guided editing technique that integrates seamlessly with our reconstruction approach, ensuring consistency and accuracy in editing tasks. Experimental results demonstrate that our approach not only excels in achieving high-fidelity image reconstruction but also performs robustly in real image composition and editing scenarios. This study underscores the potential of uniform attention maps to enhance the fidelity and versatility of diffusion-based image processing methods. Code is available at https://***/Mowenyii/Uniform-Attention-Maps. © 2025 IEEE.

关键词： Photointerpretation

来源：评论

学校读者我要写书评

暂无评论

ICLEval: Evaluating In-Context Learning Ability of Large Language Models 31

ICLEval: Evaluating In-Context Learning Ability of Large Lan...

引用

31st International Conference on Computational Linguistics, COLING 2025

作者： Chen, Wentong Lin, Yankai Zhou, ZhenHao Huang, HongYun Jia, Yantao Cao, Zhao Wen, Ji-Rong Gaoling School of Artificial Intelligence Renmin University of China China Huawei Poisson Lab China Beijing Key Laboratory of Big Data Management and Analysis Methods China School of Information Renmin University of China China

ISBN: (纸本)9798891761964

In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs. Evaluating the ICL ability of LLMs can enhance their utilization and deepen our understanding of how this ability is acquired at the training stage. However, existing evaluation frameworks primarily focus on language abilities and knowledge, often overlooking the assessment of ICL ability. In this work, we introduce the ICLEval benchmark to evaluate the ICL abilities of LLMs, which encompasses two key sub-abilities: exact copying and rule learning. Through the ICLEval benchmark, we demonstrate that ICL ability is universally present in different LLMs, and model size is not the sole determinant of ICL efficacy. Surprisingly, we observe that ICL abilities, particularly copying, develop early in the pretraining process and stabilize afterward. Our source codes and benchmark are released at https://***/RUCBM/ICLEval. © 2025 Association for Computational Linguistics.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

Effective and Efficient Masked Image Generation Models

arXiv

引用

arXiv 2025年

作者： You, Zebin Ou, Jingyang Zhang, Xiaolu Hu, Jun Zhou, Jun Li, Chongxuan Gaoling School of AI Renmin University of China Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China Ant Group China

Although masked image generation models and masked diffusion models are designed with different motivations and objectives, we observe that they can be unified within a single framework. Building upon this insight, we carefully explore the design space of training and sampling, identifying key factors that contribute to both performance and efficiency. Based on the improvements observed during this exploration, we develop our model, referred to as eMIGM. Empirically, eMIGM demonstrates strong performance on ImageNet generation, as measured by Fréchet Inception Distance (FID). In particular, on ImageNet 256×256, with similar number of function evaluations (NFEs) and model parameters, eMIGM outperforms the seminal VAR. Moreover, as NFE and model parameters increase, eMIGM achieves performance comparable to the state-of-the-art continuous diffusion models while requiring less than 40% of the NFE. Additionally, on ImageNet 512 × 512, with only about 60% of the NFE, eMIGM outperforms the state-of-the-art continuous diffusion models. © 2025, CC BY.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing

Uniform Attention Maps: Boosting Image Fidelity in Reconstru...

引用

IEEE Workshop on Applications of Computer Vision (WACV)

作者： Wenyi Mo Tianyu Zhang Yalong Bai Bing Su Ji-Rong Wen Gaoling School of Artificial Intelligence Renmin University of China Beijing Key Laboratory of Big Data Management and Analysis Methods Du Xiaoman Technology

ISBN: (数字)9798331510831

ISBN: (纸本)9798331510848

关键词： Computer vision Accuracy Codes Image synthesis Noise Reconstruction algorithms Distortion Diffusion models Robustness Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Large Language Diffusion Models

arXiv

引用

arXiv 2025年

作者： Nie, Shen Zhu, Fengqi You, Zebin Zhang, Xiaolu Ou, Jingyang Hu, Jun Zhou, Jun Lin, Yankai Wen, Ji-Rong Li, Chongxuan Gaoling School of Artificial Intelligence Renmin University of China Beijing Key Laboratory of Big Data Management and Analysis Methods China Ant Group China

Autoregressive models (ARMs) are widely regarded as the cornerstone of large language models (LLMs). We challenge this notion by introducing LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning (SFT) paradigm. LLaDA models distributions through a forward data masking process and a reverse process, parameterized by a vanilla Transformer to predict masked tokens. By optimizing a likelihood bound, it provides a principled generative approach for probabilistic inference. Across extensive benchmarks, LLaDA demonstrates strong scalability, outperforming our self-constructed ARM baselines. Remarkably, LLaDA 8B is competitive with strong LLMs like LLaMA3 8B in in-context learning and, after SFT, exhibits impressive instruction-following abilities in case studies such as multi-turn dialogue. Moreover, LLaDA addresses the reversal curse, surpassing GPT-4o in a reversal poem completion task. Our findings establish diffusion models as a viable and promising alternative to ARMs, challenging the assumption that key LLM capabilities discussed above are inherently tied to ARMs. Project page and codes: https://***/LLaDA-demo/. Copyright © 2025, The Authors. All rights reserved.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

A Theory for Conditional Generative Modeling on Multiple data Sources

arXiv

引用

arXiv 2025年

作者： Wang, Rongzhen Zhang, Yan Zheng, Chenyu Li, Chongxuan Wu, Guoqiang Gaoling School of AI Renmin University of China China Beijing Key Laboratory of Big Data Management and Analysis Methods Beijing China School of Software Shandong University Shandong China

The success of large generative models has driven a paradigm shift, leveraging massive multi-source data to enhance model capabilities. However, the interaction among these sources remains theoretically underexplored. This paper takes the first step toward a rigorous analysis of multi-source training in conditional generative modeling, where each condition represents a distinct data source. Specifically, we establish a general distribution estimation error bound in average total variation distance for conditional maximum likelihood estimation based on the bracketing number. Our result shows that when source distributions share certain similarities and the model is expressive enough, multi-source training guarantees a sharper bound than single-source training. We further instantiate the general theory on conditional Gaussian estimation and deep generative models including autoregressive and flexible energy-based models, by characterizing their bracketing numbers. The results highlight that the number of sources and similarity among source distributions improve the advantage of multi-source training. Simulations and real-world experiments validate our theory. Code is available at: https://***/ML-GSAI/Multi-Source-GM. Copyright © 2025, The Authors. All rights reserved.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

ReQFlow: Rectified Quaternion Flow for Efficient and High-Quality Protein Backbone Generation

arXiv

引用

arXiv 2025年

作者： Yue, Angxiao Wang, Zichong Xu, Hongteng Gaoling School of Artificial Intelligence Renmin University of China China School of Statistics Renmin University of China China Beijing Key Laboratory of Big Data Management and Analysis Methods China

Protein backbone generation plays a central role in de novo protein design and is significant for many biological and medical applications. Although diffusion and flow-based generative models provide potential solutions to this challenging task, they often generate proteins with undesired designability and suffer computational inefficiency. In this study, we propose a novel rectified quaternion flow (ReQFlow) matching method for fast and high-quality protein backbone generation. In particular, our method generates a local translation and a 3D rotation from random noise for each residue in a protein chain, which represents each 3D rotation as a unit quaternion and constructs its flow by spherical linear interpolation (SLERP) in an exponential format. We train the model by quaternion flow (QFlow) matching with guaranteed numerical stability and rectify the QFlow model to accelerate its inference and improve the designability of generated protein backbones, leading to the proposed ReQFlow model. Experiments show that ReQFlow achieves state-of-the-art performance in protein backbone generation while requiring much fewer sampling steps and significantly less inference time (e.g., being 37× faster than RFDiffusion and 62× faster than Genie2 when generating a backbone of length 300), demonstrating its effectiveness and efficiency. The code is available at https://***/AngxiaoYue/ReQFlow. Copyright © 2025, The Authors. All rights reserved.

关键词： Medical applications

来源：评论

学校读者我要写书评

暂无评论

A Plug-and-Play Bregman ADMM Module for Inferring Event Branches in Temporal Point Processes

arXiv

引用

arXiv 2025年

作者： Wang, Qingmei Wu, Yuxin Long, Yujie Huang, Jing Ran, Fengyuan Su, Bing Xu, Hongteng Gaoling School of Artificial Intelligence Renmin University of China China School of Cyber Science and Engineering Wuhan University China Beijing Key Laboratory of Big Data Management and Analysis Methods China

An event sequence generated by a temporal point process is often associated with a hidden and structured event branching process that captures the triggering relations between its historical and current events. In this study, we design a new plug-and-play module based on the Bregman ADMM (BADMM) algorithm, which infers event branches associated with event sequences in the maximum likelihood estimation framework of temporal point processes (TPPs). Specifically, we formulate the inference of event branches as an optimization problem for the event transition matrix under sparse and low-rank constraints, which is embedded in existing TPP models or their learning paradigms. We can implement this optimization problem based on subspace clustering and sparse group-lasso, respectively, and solve it using the Bregman ADMM algorithm, whose unrolling leads to the proposed BADMM module. When learning a classic TPP (e.g., Hawkes process) by the expectation-maximization algorithm, the BADMM module helps derive structured responsibility matrices in the E-step. Similarly, the BADMM module helps derive low-rank and sparse attention maps for the neural TPPs with self-attention layers. The structured responsibility matrices and attention maps, which work as learned event transition matrices, indicate event branches, e.g., inferring isolated events and those key events triggering many subsequent events. Experiments on both synthetic and real-world data show that plugging our BADMM module into existing TPP models and learning paradigms can improve model performance and provide us with interpretable structured event branches. The code is available at https://***/qingmeiwangdaily/BADMM *** Codes 60G55, 62M10 Copyright © 2025, The Authors. All rights reserved.

关键词： Maximum likelihood estimation

来源：评论

学校读者我要写书评

暂无评论

Efficient Variants of Wasserstein Distance in Hyperbolic Space via Space-Filling Curve Projection

引用

IEEE Transactions on Neural Networks and Learning Systems 2025年 PP卷 PP页

作者： Li, Tao Meng, Cheng Xu, Hongteng Zhu, Jun Renmin University of China Institute of Statistics and Big Data Beijing100872 China Renmin University of China Center for Applied Statistics Institute of Statistics and Big Data Beijing100872 China Renmin University of China Gaoling School of Artificial Intelligence Beijing100872 China Beijing Key Laboratory of Big Data Management and Analysis Beijing100872 China

Hyperbolic spaces have been considered pervasively for embedding hierarchically structured data in the recent decade. However, there is a lack of studies focusing on efficient distance metrics for comparing probability distributions in hyperbolic spaces. To bridge the gap, we propose a novel metric called the hyperbolic space-filling curve projection Wasserstein (SFW) distance. The idea is to first project two probability distributions onto a space-filling curve to obtain a closed-form coupling between them and then calculate the transport distance between these two distributions in the hyperbolic space accordingly. Theoretically, we show the SFW distance is a proper metric and is well-defined for probability measures with bounded supports. Statistical convergence rates for the proposed estimator are provided as well. Moreover, we propose two variants of the SFW distance based on geodesic and horospherical projections, respectively, to combat the curse-of-dimensionality. Empirical results on synthetic and real-world data indicate that the SFW distance can effectively serve as a surrogate of the popular Wasserstein distance with low complexity. © 2012 IEEE.

关键词： Hilbert spaces

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：