检索结果-内蒙古大学图书馆

Dis²Booth: Learning Image Distribution with Disentangled Features for Text-to-Image Diffusion Models 39

学校读者我要写书评

暂无评论

Dis<sup>2</sup>Booth: Learning Image Distribution with Disen...

39th Annual AAAI Conference on Artificial intell.gence, AAAI 2025

作者： Ding, Guanqi Yang, Chengyu Wang, Shuhui Li, Xincheng Zhang, Jinzhe Jin, Xin Huang, Qingming University of Chinese Academy of Sciences Beijing China Beijing Institute of Technology Beijing China Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China Huawei Cloud EI Innovation Lab China Peng Cheng Laboratory Shenzhen China

ISBN: (纸本)157735897X

Personalized image generation enables customized content creation based on the text-to-image diffusion models. However, existing personalization methods focus on fine-tuning generative models to generate a specific instance or concept, like a specific Corgi, but cannot handle multiple instances or concepts with commonalities, such as images of multiple different Corgis. In this work, we focus on personalizing a diffusion model to generated varied data usually containing multiple instances, which has a more diverse and complex data distribution. Our basic assumption is that the varied data distribution is composed of the common features shared among all samples, as well as the reasonable variations within it. Accordingly, we are capable to decompose the learning process.of complex data distributions into two simpler sub-tasks, employing a divide-and-conquer approach. To this end, we propose Dis²Booth, a framework that can learn complex image Distribution by Disentangling data distribution in an unsupervised manner. Specifically, Dis²Booth contains two modules, Anchor LoRA and Delta LoRA, tasked with learning the common features and variational features constrained by Contextual Loss and Delta Loss unsupervisedly. Besides, the Asynchronous Optimization Strategy is proposed to ensure the collaborative training of the two modules. Extensive experiments suggest that Dis²Booth can learn the data distribution with higher diversity and complexity and be flexibly applied across various tasks. © 2025, Association for the Advancement of Artificial intell.gence (***). All rights reserved.

关键词： Unsupervised learning

Context-aware Difference Distilling for Multi-change Captioning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tu, Yunbin Li, Liang Su, Li Zha, Zheng-Jun Yan, Chenggang Huang, Qingming University of Chinese Academy of Sciences Beijing China Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China University of Science and Technology of China Hefei China Hangzhou Dianzi University Hangzhou China Lishui Institute of Hangzhou Dianzi University Lishui China

Multi-change captioning aims to describe complex and coupled changes within an image pair in natural language. Compared with single-change captioning, this task requires the model to have higher-level cognition ability to reason an arbitrary number of changes. In this paper, we propose a novel context-aware difference distilling (CARD) network to capture all genuine changes for yielding sentences. Given an image pair, CARD first decouples context features that aggregate all similar/dissimilar semantics, termed common/difference context features. Then, the consistency and independence constraints are designed to guarantee the alignment/discrepancy of common/difference context features. Further, the common context features guide the model to mine locally unchanged features, which are subtracted from the pair to distill locally difference features. Next, the difference context features augment the locally difference features to ensure that all changes are distilled. In this way, we obtain an omni-representation of all changes, which is translated into linguistic sentences by a transformer decoder. Extensive experiments on three public datasets show CARD performs favourably against state-of-the-art methods. The code is available at https://***/tuyunbin/CARD. © 2024, CC BY.

关键词： Semantics

Weakly Supervised Video Individual Counting

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Xinyan Li, Guorong Qi, Yuankai Yan, Ziheng Han, Zhenjun van den Hengel, Anton Yang, Ming-Hsuan Huang, Qingming University of Chinese Academy of Science Beijing China Australian Institute for Machine Learning The University of Adelaide Australia Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China Peng Cheng Laboratory Shenzhen China University of California Merced United States

Video Individual Counting (VIC) aims to predict the number of unique individuals in a single video. Existing methods learn representations based on trajectory labels for individuals, which are annotation-expensive. To provide a more realistic reflection of the underlying practical challenge, we introduce a weakly supervised VIC task, wherein trajectory labels are not provided. Instead, two types of labels are provided to indicate traffic entering the field of view (inflow) and leaving the field view (outflow). We also propose the first solution as a baseline that formulates the task as a weakly supervised contrastive learning problem under group-level matching. In doing so, we devise an end-to-end trainable soft contrastive loss to drive the network to distinguish inflow, outflow, and the remaining. To facilitate future study in this direction, we generate annotations from the existing VIC datasets SenseCrowd and CroHD and also build a new dataset, UAVVIC. Extensive results show that our baseline weakly supervised method outperforms supervised methods, and thus, little information is lost in the transition to the more practically relevant weakly supervised task. The code and trained model will be public at CGNet. © 2023, CC BY.

关键词： Supervised learning

Automatic Relation-aware Graph Network Proliferation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cai, Shaofei Li, Liang Han, Xinzhe Luo, Jiebo Zha, Zheng-Jun Huang, Qingming Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China University of Chinese Academy of Sciences Beijing China University of Rochester United States University of Science and Technology of China China Peng Cheng Laboratory Shenzhen China

Graph neural architecture search has sparked much attention as Graph Neural Networks (GNNs) have shown powerful reasoning capability in many relational tasks. However, the currently used graph search space overemphasizes learning node features and neglects mining hierarchical relational information. Moreover, due to diverse mechanisms in the message passing, the graph search space is much larger than that of CNNs. This hinders the straightforward application of classical search strategies for exploring complicated graph search space. We propose Automatic Relation-aware Graph Network Proliferation (ARGNP) for efficiently searching GNNs with a relation-guided message passing mechanism. Specifically, we first devise a novel dual relation-aware graph search space that comprises both node and relation learning operations. These operations can extract hierarchical node/relational information and provide anisotropic guidance for message passing on a graph. Second, analogous to cell proliferation, we design a network proliferation search paradigm to progressively determine the GNN architectures by iteratively performing network division and differentiation. The experiments on six datasets for four graph learning tasks demonstrate that GNNs produced by our method are superior to the current state-of-the-art handcrafted and search-based GNNs. Codes are available at https://***/phython96/ARGNP. © 2022, CC BY-NC-ND.

关键词： Graph neural networks

Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xiao, Jiayu Li, Liang Wang, Chaofei Zha, Zheng-Jun Huang, Qingming Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China University of Chinese Academy of Sciences Beijing China Department of Automation Tsinghua University China University of Science and Technology of China China Peng Cheng Laboratory Shenzhen China

Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. However, existing methods are prone to model overfitting and collapse in extremely few shot setting (less than 10). To solve this problem, we propose a relaxed spatial structural alignment method to calibrate the target generative models during the adaption. We design a cross-domain spatial structural consistency loss comprising the self-correlation and disturbance correlation consistency loss. It helps align the spatial structural information between the synthesis image pairs of the source and target domains. To relax the cross-domain alignment, we compress the original latent space of generative models to a subspace. Image pairs generated from the subspace are pulled closer. Qualitative and quantitative experiments show that our method consistently surpasses the state-of-the-art methods in few shot setting. © 2022, CC BY.

关键词： Generative adversarial networks

R3Net:Relation-embedded representation reconstruction network for change captioning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Tu, Yunbin Li, Liang Yan, Chenggang Gao, Shengxiang Yu, Zhengtao Yunnan Key Laboratory of Artificial Intelligence Kunming University of Science and Technology Key Lab of Intell. Info. Process. Inst. of Comput. Tech. Chinese Academy of Sciences Intelligent Information Processing Laboratory Hangzhou Dianzi University

Change captioning is to use a natural language sentence to describe the fine-grained disagreement between two similar images. Viewpoint change is the most typical distractor in this task, because it changes the scale and location of the objects and overwhelms the representation of real change. In this paper, we propose a Relation-embedded Representation Reconstruction Network (R3Net) to explicitly distinguish the real change from the large amount of clutter and irrelevant changes. Specifically, a relation-embedded module is first devised to explore potential changed objects in the large amount of clutter. Then, based on the semantic similarities of corresponding locations in the two images, a representation reconstruction module (RRM) is designed to learn the reconstruction representation and further model the difference representation. Besides, we introduce a syntactic skeleton predictor (SSP) to enhance the semantic interaction between change localization and caption generation. Extensive experiments show that the proposed method achieves the state-of-the-art results on two public datasets 1 © 2021, CC BY.

关键词： Semantics

Progressive Multi-resolution Loss for Crowd Counting

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yan, Ziheng Qi, Yuankai Li, Guorong Liu, Xinyan Zhang, Weigang Huang, Qingming Yang, Ming-Hsuan University of Chinese Academy of Science Beijing China Australian Institute for Machine Learning The University of Adelaide Australia Harbin Institute of Technology Weihai China Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China Peng Cheng Laboratory Shenzhen China University of California MercedCA United States

Crowd counting is usually handled in a density map regression fashion, which is supervised via a L2 loss between the predicted density map and ground truth. To effectively regulate models, various improved L2 loss functions have been proposed to find a better correspondence between predicted density and annotation positions. In this paper, we propose to predict the density map at one resolution but measure the density map at multiple resolutions. By maximizing the posterior probability in such a setting, we obtain a log-formed multi-resolution L2-difference loss, where the traditional single-resolution L2 loss is its particular case. We mathematically prove it is superior to a single-resolution L2 loss. Without bells and whistles, the proposed loss substantially improves several baselines and performs favorably compared to state-of-the-art methods on four crowd counting datasets, ShanghaiTech A & B, UCF-QNRF, and JHU-Crowd++. © 2022, CC BY.

关键词：

Consistency-Aware Anchor Pyramid Network for Crowd Localization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Xinyan Li, Guorong Qi, Yuankai Han, Zhenjun Huang, Qingming Yang, Ming-Hsuan Sebe, Nicu University of Chinese Academy of Science Beijing China Australian Institute for Machine Learning The University of Adelaide Australia Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China Peng Cheng Laboratory Shenzhen China University of California Merced United States University of Trento Italy

Crowd localization aims to predict the spatial position of humans in a crowd scenario. We observe that the performance of existing methods is challenged from two aspects: (i) ranking inconsistency between test and training phases;and (ii) fixed anchor resolution may underfit or overfit crowd densities of local regions. To address these problems, we design a supervision target reassignment strategy for training to reduce ranking inconsistency and propose an anchor pyramid scheme to adaptively determine the anchor density in each image region. Extensive experimental results on three widely adopted datasets (ShanghaiTech A & B, JHU-CROWD++, UCF-QNRF) demonstrate the favorable performance against several state-of-the-art methods. © 2022, CC BY.

关键词：

Edge-featured graph neural architecture search

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Cai, Shaofei Li, Liang Han, Xinzhe Zha, Zheng-Jun Huang, Qingming Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS Beijing China University of Chinese Academy of Sciences Beijing China University of Science and Technology of China China Peng Cheng Laboratory Shenzhen China

Graph neural networks (GNNs) have been successfully applied to learning representation on graphs in many relational tasks. Recently, researchers study neural architecture search (NAS) to reduce the dependence of human expertise and explore better GNN architectures, but they over-emphasize entity features and ignore latent relation information concealed in the edges. To solve this problem, we incorporate edge features into graph search space and propose Edge-featured Graph Neural Architecture Search (EGNAS) to find the optimal GNN architecture. Specifically, we design rich entity and edge updating operations to learn high-order representations, which convey more generic message passing mechanisms. Moreover, the architecture topology in our search space allows to explore complex feature dependence of both entities and edges, which can be efficiently optimized by differentiable search strategy. Experiments at three graph tasks on six datasets show EGNAS can search better GNNs with higher performance than current state-of-the-art human-designed and searched-based GNNs. © 2021, CC BY-SA.

关键词： Graph neural networks