检索结果-内蒙古大学图书馆

Multi2human: Controllable Human Image Generation with Multimodal Controls

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Gu, Xiaoling Xu, Shengwenzhuo Wong, Yongkang Wu, Zizhao Yu, Jun Fan, Jianping Kankanhalli, Mohan S. Key Laboratory of Complex Systems Modeling and Simulation School of Computer Science and Technology Hangzhou Dianzi University China National University of Singapore Singapore School of Digital Media Technology Hangzhou Dianzi University China

Generating high-quality and diverse human images presents a substantial difficulty within the field of computer vision, especially in developing controllable generative models that can utilize input from various modalities. Such models could enable innovative applications like digital human, fashion design, and content creation. In this study, we introduce Multi2Human, a two-stage image synthesis framework for controllable human image generation with multimodal controls. In the first stage, a novel WAvelet-vqVaE (WAVE) architecture is designed to embed human images using a learnable codebook. The WAVE model enhances the conventional Vector Quantized Variational Autoencoder (VQVAE) by integrating wavelets throughout the encoder, thereby enhancing the quality of image reconstruction and synthesis. In the second stage, a new Multimodal Conditioned Diffusion Model (MCDM) is de- signed to estimate the underlying prior distribution within the discrete latent space using a discrete diffusion process, thus allowing for human image generation condi- tioned on multimodal controls. Quantitative and qualitative analysis demonstrates that the proposed method has the ability to create high-quality, lifelike full-body human images while satisfying the specified multimodal controls. Our code is available at https://***/gxl-groups/Multi2Human. © 2023, The Authors. All rights reserved.

关键词： Image reconstruction

Strength of the North African monsoon in the Last Interglacial and under future warming

学校读者我要写书评

暂无评论

Atmospheric and Oceanic science Letters 2023年第3期16卷 20-26页

作者： Jiazhi He Weiyi Sun Jing Wang Bin Wang Jian Liu Key Laboratory for Virtual Geographic Environment Ministry of EducationState Key Laboratory Cultivation Base of Geographical Environment Evolution of Jiangsu ProvinceJiangsu Center for Collaborative Innovation in Geographical Information Resource Development and ApplicationSchool of Geography ScienceNanjing Normal UniversityNanjingChina Department of Atmospheric Sciences and AtmosphereOcean Research Center University of Hawaii at ManoaHonoluluHIUA Jiangsu Provincial Key Laboratory for Numerical Simulation of Large Scale Complex Systems School of Mathematical ScienceNanjing Normal UniversityNanjingChina Open Studioforthe Simulationof Ocean-Climate-Isotope Qingdao National Laboratoryfor Marine Science and TechnologyQingdaoChina

与人为强迫引起的全球变暖相比,末次间冰期是轨道强迫引起的过去80万年来最暖的一个间冰期,但鲜有人研究末次间冰期中北非季风的响应.因此,本文基于CMIP6多模式模拟结果对比研究了末次间冰期和SSP5-8.5情景下北非季风的响应,发现末次间... 详细信息

与人为强迫引起的全球变暖相比,末次间冰期是轨道强迫引起的过去80万年来最暖的一个间冰期,但鲜有人研究末次间冰期中北非季风的响应.因此,本文基于CMIP6多模式模拟结果对比研究了末次间冰期和SSP5-8.5情景下北非季风的响应,发现末次间冰期下北非季风平均降水及其降水变率均远大于SSP5-8.5情景下的结果.轨道强迫导致的北大西洋暖于南大西洋增加了北非季风环流和平均降水,同时,南北大西洋海温梯度变化通过增强热带北大西洋的海气相互作用增大了海温梯度和撒哈拉低压的变率,从而增强了北非季风降水变率.

关键词：北非季风降水变率末次间冰期全球变暖

Mitigating Over-Trust in Chat Companion Robot Interaction Using Augmented Reality 4

学校读者我要写书评

暂无评论

Mitigating Over-Trust in Chat Companion Robot Interaction Us...

4th International Conference on Artificial Intelligence and Computer Engineering, ICAICE 2023

作者： Wang, Guobao Zhao, Changshun Liu, Ying He, Di Wang, Yuqing Zhang, Kaiyue Yanshan University Hebei Qinhuangdao China Jiangsu Xuzhou Construction Machinery Group Co. Ltd. Jiangsu Xuzhou221004 China Officers College of Pap Sichuan Chengdu China National Key Laboratory for Complex Systems Simulation Beijing China University of Electronic Science and Technology of China Sichuan Chengdu China Jiangsu Province Tongshan Secondary Vocational School Jiangsu Tongshan China

ISBN: (纸本)9798400708831

This study aims to explore the effects of different human-machine interfaces, system performance, and robot interaction modalities on over-trust in chat companion robot systems. We designed and implemented three types of interfaces: 2D Virtual Interface + Virtual Robot, 3D Augmented Reality Interface + AR-based Virtual Robot, and 3D Real World + NAO Physical Robot. We also manipulated the system performance (high or low) and the robot interaction modality (language, gesture, or both). We conducted a user study with 30 participants, who interacted with the chat companion robot under different conditions and rated their trust, distrust, and human-robot distance. We also measured the user experience in terms of usefulness and ease of use. The results showed that the type of interface significantly influenced the trust, distrust, and human-robot distance of the users, and that the 3D augmented reality interface was the most effective in enhancing trust and reducing distrust and distance. The performance of the system also affected the trust and distance of the users, and the gesture interface was the most sensitive to the performance variation. The modality of the interaction did not have a significant impact on the dependent variables. The user experience ratings also indicated that the 3D augmented reality interface was the most useful and easy to use. The implications and limitations of these findings are discussed in the paper. © 2023 ACM.

关键词： Augmented reality

Architecture Transformation Research on Joint Border Defense systems-of-systems

学校读者我要写书评

暂无评论

Architecture Transformation Research on Joint Border Defense...

作者： Mengmeng Zhang Shuanghui Yi Menglong Lin Honghui Chen Science and Technology on Information Systems Engineering Laboratory National Key Laboratory for Complex System Simulation

Currently,with the deepening of development and open of China's border,issues of border joint defense become increasingly *** means of cooperation and coordination are far from being able to align requirements;information systems of different joint defense units are difficult to interconnect and interoperate;the previous results of joint defense informatization are difficult to *** main reason is the lack of border joint defense information construction with a top-level design *** the lens of top-level design,this paper establishes a multi-viewpoint architecture framework and transformation process,develops the as-is and to-be joint defense architecture,and finally analyzes transformation principles from the as-is architecture to to-be *** paper mainly explores the architecture transformation mechanisms under the border joint defense scenario.

关键词：

Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yu, Zhou Ouyang, Xuecheng Shao, Zhenwei Wang, Meng Yu, Jun The Key Laboratory of Complex Systems Modeling and Simulation the School of Computer Science Hangzhou Dianzi University China The HDU-ITMO Joint Institute Hangzhou Dianzi University China The School of Computer Science and Information Engineering Hefei University of Technology China

Knowledge-based visual question answering (VQA) requires external knowledge beyond the image to answer the question. Early studies retrieve required knowledge from explicit knowledge bases (KBs), which often introduces irrelevant information to the question, hence restricting the performance of their models. Recent works have resorted to using a powerful large language model (LLM) as an implicit knowledge engine to acquire the necessary knowledge for answering. Despite the encouraging results achieved by these methods, we argue that they have not fully activated the capacity of the blind LLM as the provided textual input is insufficient to depict the required visual information to answer the question. In this paper, we present Prophet—a conceptually simple, flexible, and general framework designed to prompt LLM with answer heuristics for knowledge-based VQA. Specifically, we first train a vanilla VQA model on a specific knowledge-based VQA dataset without external knowledge. After that, we extract two types of complementary answer heuristics from the VQA model: answer candidates and answer-aware examples. Finally, the two types of answer heuristics are jointly encoded into a formatted prompt to facilitate the LLM’s understanding of both the image and question, thus generating a more accurate answer. By incorporating the state-of-the-art LLM GPT-3 [1], Prophet significantly outperforms existing state-of-the-art methods on four challenging knowledge-based VQA datasets. To demonstrate the generality of our approach, we instantiate Prophet with the combinations of different VQA models (i.e., both discriminative and generative ones) and different LLMs (i.e., both commercial and open-source ones). © 2023, CC BY.

关键词： Knowledge based systems

A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Ting Lin, Xiaojun Wang, Shuhui Sheng, Weiguo Huang, Qingming Yu, Jun School of Information Science and Technology Hangzhou Normal University Hangzhou311121 China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing101408 China Key Laboratory of Complex Systems Modeling and Simulation School of Computer Science and Technology Hangzhou Dianzi University Hangzhou310018 China

Three-Dimensional (3D) dense captioning is an emerging vision-language bridging task that aims to generate multiple detailed and accurate descriptions for 3D scenes. It presents significant potential and challenges due to its closer representation of the real world compared to 2D visual captioning, as well as complexities in data collection and processing of 3D point cloud sources. Despite the popularity and success of existing methods, there is a lack of comprehensive surveys summarizing the advancements in this field, which hinders its progress. In this paper, we provide a comprehensive review of 3D dense captioning, covering task definition, architecture classification, dataset analysis, evaluation metrics, and in-depth prosperity discussions. Based on a synthesis of previous literature, we refine a standard pipeline that serves as a common paradigm for existing methods. We also introduce a clear taxonomy of existing models, summarize technologies involved in different modules, and conduct detailed experiment analysis. Instead of a chronological order introduction, we categorize the methods into different classes to facilitate exploration and analysis of the differences and connections among existing techniques. We also provide a reading guideline to assist readers with different backgrounds and purposes in reading efficiently. Furthermore, we propose a series of promising future directions for 3D dense captioning by identifying challenges and aligning them with the development of related tasks, offering valuable insights and inspiring future research in this field. Our aim is to provide a comprehensive understanding of 3D dense captioning, foster further investigations, and contribute to the development of novel applications in multimedia and related domains. Copyright © 2024, The Authors. All rights reserved.

关键词： Data handling

Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Ting Fu, Kunhao Wang, Shuhui Huang, Qingming Yu, Jun School of Information Science and Technology Hangzhou Normal University Hangzhou311121 China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing101408 China Key Laboratory of Complex Systems Modeling and Simulation School of Computer Science and Technology Hangzhou Dianzi University Hangzhou310018 China

Video Question Answering (VideoQA) represents a crucial intersection between video understanding and language processing, requiring both discriminative unimodal comprehension and sophisticated cross-modal interaction for accurate inference. Despite advancements in multi-modal pre-trained models and video-language foundation models, these systems often struggle with domain-specific VideoQA due to their generalized pre-training objectives. Addressing this gap necessitates bridging the divide between broad cross-modal knowledge and the specific inference demands of VideoQA tasks. To this end, we introduce HeurVidQA, a framework that leverages domain-specific entity-action heuristics to refine pre-trained video-language foundation models. Our approach treats these models as implicit knowledge engines, employing domain-specific entity-action prompters to direct the model’s focus toward precise cues that enhance reasoning. By delivering fine-grained heuristics, we improve the model’s ability to identify and interpret key entities and actions, thereby enhancing its reasoning capabilities. Extensive evaluations across multiple VideoQA datasets demonstrate that our method significantly outperforms existing models, underscoring the importance of integrating domain-specific knowledge into video-language models for more accurate and context-aware VideoQA. Copyright © 2024, The Authors. All rights reserved.

关键词： Inference engines

Decentralized Event-Triggered Online Learning for Safe Consensus of Multi-Agent systems with Gaussian Process Regression

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Dai, Xiaobing Yang, Zewen Xu, Mengtian Liu, Fangzhou Hattab, Georges Hirche, Sandra TUM School of Computation Information and Technology Technical University of Munich Munich80333 Germany Robert Koch Institute Berlin Germany National Key Laboratory of Modeling and Simulation for Complex Systems Harbin Institute of Technology 150001 China Department of Mathematics and Computer Science Freie Universität Berlin Berlin Germany

Consensus control in multi-agent systems has received significant attention and practical implementation across various domains. However, managing consensus control under unknown dynamics remains a significant challenge for control design due to system uncertainties and environmental disturbances. This paper presents a novel learning-based distributed control law, augmented by an auxiliary dynamics. Gaussian processes are harnessed to compensate for the unknown components of the multi-agent system. For continuous enhancement in predictive performance of Gaussian process model, a data-efficient online learning strategy with a decentralized event-triggered mechanism is proposed. Furthermore, the control performance of the proposed approach is ensured via the Lyapunov theory, based on a probabilistic guarantee for prediction error bounds. To demonstrate the efficacy of the proposed learning-based controller, a comparative analysis is conducted, contrasting it with both conventional distributed control laws and offline learning methodologies. Copyright © 2024, The Authors. All rights reserved.

关键词： Multi agent systems

A Distributed Approximation Algorithm for the Total Dominating Set Problem 1

学校读者我要写书评

暂无评论

18th International Conference on Algorithmic Aspects in Information and Management, AAIM 2024

作者： Wang, Limin Zhang, Zhao Du, Donglei Mao, Yaping Zhang, Xiaoyan School of Mathematical Sciences Henan Institute of Science and Technology Xinxiang453003 China College of Mathematics and Computer Sciences Zhejiang Normal University Zhejiang321004 China Faculty of Management University of New Brunswick Fredericton E3B 5A3 New Brunswick Canada School of Mathematics and Statistics Qinghai Normal University Qinghai810008 China Academy of Plateau Science and Sustainability Xining Qinghai810008 China School of Mathematical Science and Institute of Mathematics Key Laboratory of Ministry of Education Numerical Simulation of Large Scale Complex Systems Nanjing Normal University Jiangsu210023 China

ISBN: (数字)9789819777983

ISBN: (纸本)9789819777976

Given a network graph G=(V,E), a subset T⊆V is said to be a total dominating set (TDS) if every v∈V is adjacent to at least one node in T. In this paper, we present a distributed algorithm for the TDS problem via the LP relaxation techniques. For a positive integer k and maximum degree Δ, the proposed algorithm outputs a fractional total dominating set of expected size O(kΔ2k)|TDSOPT|. The distributed algorithm runs in O(k2) communication rounds, and the algorithm use messages of size O(logΔ). The fractional solution is then rounded to obtain an integer total dominating set for the original problem. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

关键词： Consensus algorithm