检索结果-内蒙古大学图书馆

Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Xinyang Lai, Zhangyu Xu, Linning Qu, Yansong Cao, Liujuan Zhang, Shengchuan Dai, Bo Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Shanghai Artificial Intelligence Laboratory China The Chinese University of Hong Kong Hong Kong

Recent advancements in 3D generation have leveraged synthetic datasets with ground truth 3D assets and predefined cameras. However, the potential of adopting real-world datasets, which can produce significantly more realistic 3D scenes, remains largely unexplored. In this work, we delve into the key challenge of the complex and scene-specific camera trajectories found in real-world captures. We introduce Director3D, a robust open-world text-to-3D generation framework, designed to generate both real-world 3D scenes and adaptive camera trajectories. To achieve this, (1) we first utilize a Trajectory Diffusion Transformer, acting as the Cinematographer, to model the distribution of camera trajectories based on textual descriptions. (2) Next, a Gaussian-driven Multi-view Latent Diffusion Model serves as the Decorator, modeling the image sequence distribution given the camera trajectories and texts. This model, fine-tuned from a 2D diffusion model, directly generates pixel-aligned 3D Gaussians as an immediate 3D scene representation for consistent denoising. (3) Lastly, the 3D Gaussians are refined by a novel SDS++ loss as the Detailer, which incorporates the prior of the 2D diffusion model. Extensive experiments demonstrate that Director3D outperforms existing methods, offering superior performance in real-world 3D generation. Copyright © 2024, The Authors. All rights reserved.

关键词： Trajectories

Dual3D: efficient and Consistent Text-to-3D Generation with Dual-mode Multi-view Latent Diffusion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Xinyang Lai, Zhangyu Xu, Linning Guo, Jianfei Cao, Liujuan Zhang, Shengchuan Dai, Bo Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China The Chinese University of Hong Kong Hong Kong Shanghai Artificial Intelligence Laboratory China

We present Dual3D, a novel text-to-3D generation framework that generates high-quality 3D assets from texts in only 1 minute. The key component is a dual-mode multi-view latent diffusion model. Given the noisy multi-view latents, the 2D mode can efficiently denoise them with a single latent denoising network, while the 3D mode can generate a tri-plane neural surface for consistent rendering-based denoising. Most modules for both modes are tuned from a pre-trained text-to-image latent diffusion model to circumvent the expensive cost of training from scratch. To overcome the high rendering cost during inference, we propose the dual-mode toggling inference strategy to use only 1/10 denoising steps with 3D mode, successfully generating a 3D asset in just 10 seconds without sacrificing quality. The texture of the 3D asset can be further enhanced by our efficient texture refinement process in a short time. Extensive experiments demonstrate that our method delivers state-of-the-art performance while significantly reducing generation time. Our project page is available at https://***. Copyright © 2024, The Authors. All rights reserved.

关键词： Textures

Crowd OS Powered Active 3D Mapping Leveraging Heterogeneous Crowd Robots

学校读者我要写书评

暂无评论

Crowd OS Powered Active 3D Mapping Leveraging Heterogeneous ...

Artificial Intelligence of Things and Systems (AIoTSys), International Conference on

作者： Yigao Wang Changzhen Liu Yufei Wang Shengjie Wu Cheng Wang Longbiao Chen Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University Xiamen China

This paper presents a method for active 3D mapping leveraging heterogeneous crowd robots. By incorporating a UAV with its aerial perspective and powerful perception capabilities, the blind spots present in ground robot mapping are successfully eliminated, resulting in more comprehensive and accurate mapping outcomes. Furthermore, our heterogeneous crowd robots exhibits high scalability, allowing for the addition of a specified number of robots based on the mapping requirements of different indoor environments, while also supporting dynamic addition and removal. This scalability enables the system to cater to diverse applications and establishes a foundation for a more robust, comprehensive, and sustainable collective robot system. Through this research, we fully demonstrate the advantages of integrating heterogeneous robotic systems and how different types of robots with complementary capabilities collaborate to improve indoor 3D mapping processes. In particular, the introduced CrowdOS plays a significant role in task assignment and coordination, offering powerful functionalities for intelligent task allocation and coordination among different robots.

关键词：

DSMNet: Deep High-precision 3D Surface Modeling from Sparse Point Cloud Frames

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qiu, Changjie Wang, Zhiyong Lin, Xiuhong Zang, Yu Wang, Cheng Liu, Weiquan Fujian Key Laboratory of Sensing and Computing for Smart Cities Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University Xiamen361005 China

Existing point cloud modeling datasets primarily express the modeling precision by pose or trajectory precision rather than the point cloud modeling effect itself. Under this demand, we first independently construct a set of LiDAR system with an optical stage, and then we build a HPMB dataset based on the constructed LiDAR system, a High-Precision, Multi-Beam, real-world dataset. Second, we propose an modeling evaluation method based on HPMB for object-level modeling to overcome this limitation. In addition, the existing point cloud modeling methods tend to generate continuous skeletons of the global environment, hence lacking attention to the shape of complex objects. To tackle this challenge, we propose a novel learning-based joint framework, DSMNet, for high-precision 3D surface modeling from sparse point cloud frames. DSMNet comprises density-aware Point Cloud Registration (PCR) and geometry-aware Point Cloud Sampling (PCS) to effectively learn the implicit structure feature of sparse point clouds. Extensive experiments demonstrate that DSMNet outperforms the state-of-the-art methods in PCS and PCR on Multi-View Partial Point Cloud (MVP) database. Furthermore, the experiments on the open source KITTI and our proposed HPMB datasets show that DSMNet can be generalized as a post-processing of Simultaneous Localization And Mapping (SLAM), thereby improving modeling precision in environments with sparse point clouds. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical radar

Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labels

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Xia, Qiming Lin, Wenkai Xiang, Haoen Huang, Xun Chen, Siheng Dong, Zhen Wang, Cheng Wen, Chenglu Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Shanghai Jiao Tong University China Wuhan University China

Unsupervised 3D object detection serves as an important solution for offline 3D object annotation. However, due to the data sparsity and limited views, the clustering-based label fitting in unsupervised object detection often generates low-quality pseudo-labels. Multi-agent collaborative dataset, which involves the sharing of complementary observations among agents, holds the potential to break through this bottleneck. In this paper, we introduce a novel unsupervised method that learns to Detect Objects from Multi-Agent LiDAR scans, termed DOtA, without using labels from external. DOtA first uses the internally shared ego-pose and ego-shape of collaborative agents to initialize the detector, leveraging the generalization performance of neural networks to infer preliminary labels. Subsequently, DOtA uses the complementary observations between agents to perform multi-scale encoding on preliminary labels, then decodes high-quality and low-quality labels. These labels are further used as prompts to guide a correct feature learning process, thereby enhancing the performance of the unsupervised object detection task. Extensive experiments on the V2V4Real and OPV2V datasets show that our DOtA outperforms state-of-the-art unsupervised 3D object detection methods. Additionally, we also validate the effectiveness of the DOtA labels under various collaborative perception frameworks. The code is available at https://***/xmuqimingxia/DOtA. Copyright © 2025, The Authors. All rights reserved.

关键词： Object recognition

TraDiffusion: Trajectory-Based Training-Free Image Generation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Mingrui Huang, Oucheng Ji, Jiayi Li, Jiale Cai, Xinyue Kuang, Huafeng Liu, Jianzhuang Sun, Xiaoshuai Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University 361005 China Shenzhen Institute of Advanced Technology University of Chinese Academy of Sciences China

In this work, we propose a training-free, trajectory-based controllable T2I approach, termed TraDiffusion. This novel method allows users to effortlessly guide image generation via mouse trajectories. To achieve precise control, we design a distance awareness energy function to effectively guide latent variables, ensuring that the focus of generation is within the areas defined by the trajectory. The energy function encompasses a control function to draw the generation closer to the specified trajectory and a movement function to diminish activity in areas distant from the trajectory. Through extensive experiments and qualitative assessments on the COCO dataset, the results reveal that TraDiffusion facilitates simpler, more natural image control. Moreover, it showcases the ability to manipulate salient regions, attributes, and relationships within the generated images, alongside visual input based on arbitrary or enhanced trajectories. The code: https://***/och-mac/TraDiffusion. © 2024, CC BY.

关键词： Mammals

Scale Contrastive Learning with Selective Attentions for Blind Image Quality Assessment

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Zihao Li, Xudong Fu, Bohan Chu, Xiaohui Li, Ke Shen, Yunhang Zhang, Yan School of Information and Electronics Beijing Institute of Technology China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Tencent Youtu Lab

Blind image quality assessment (BIQA) serves as a fundamental task in computer vision, yet it often fails to consistently align with human subjective perception. Recent advances show that multi-scale evaluation strategies are promising due to their ability to replicate the hierarchical structure of human vision. However, the effectiveness of these strategies is limited by a lack of understanding of how different image scales influence perceived quality. This paper addresses two primary challenges: the significant redundancy of information across different scales, and the confusion caused by combining features from these scales, which may vary widely in quality. To this end, a new multi-scale BIQA framework is proposed, namely Contrast-Constrained Scale-Focused IQA Framework (CSFIQA). CSFIQA features a selective focus attention mechanism to minimize information redundancy and highlight critical quality-related information. Additionally, CSFIQA includes a scale-level contrastive learning module equipped with a noise sample matching mechanism to identify quality discrepancies across the same image content at different scales. By exploring the intrinsic relationship between image scales and the perceived quality, the proposed CSFIQA achieves leading performance on eight benchmark datasets, e.g., achieving SRCC values of 0.967 (versus 0.947 in CSIQ) and 0.905 (versus 0.876 in LIVEC). Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Deep Code Search with Naming-Agnostic Contrastive Multi-View Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Feng, Jiadong Li, Wei Wei, Zhao Xu, Yong Wang, Juhong Li, Hui Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University Fujian Xiamen China Faculty of Computing Harbin Institute of Technology Heilongjiang Harbin China Tencent Shenzhen China

Software development is a repetitive task, as developers usually reuse or get inspiration from existing implementations. Code search, which refers to the retrieval of relevant code snippets from a codebase according to the developer’s intent that has been expressed as a query, has become increasingly important in the software development process. Due to the success of deep learning in various applications, a great number of deep learning based code search approaches have sprung up and achieved promising results. However, developers may not follow the same naming conventions and the same variable may have different variable names in different implementations, bringing a challenge to deep learning based code search methods that rely on explicit variable correspondences to understand source code. To overcome this challenge, we propose a naming-agnostic code search method (NACS) based on contrastive multi-view code representation learning. NACS strips information bound to variable names from Abstract Syntax Tree (AST), the representation of the abstract syntactic structure of source code, and focuses on capturing intrinsic properties solely from AST structures. We use semantic-level and syntax-level augmentation techniques to prepare realistically rational data and adopt contrastive learning to design a graph-view modeling component in NACS to enhance the understanding of code snippets. We further model ASTs in a path view to strengthen the graph-view modeling component through multi-view learning. Extensive experiments show that NACS provides superior code search performance compared to baselines and NACS can be adapted to help existing code search methods overcome the impact of different naming conventions. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Haowei Tang, Jiji Ji, Jiayi Sun, Xiaoshuai Zhang, Rongsheng Ma, Yiwei Zhao, Minda Li, Lincheng Zhao, Zeng Lv, Tangjie Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Fuxi AI Lab Netease Inc China

In recent years, 3D understanding has turned to 2D vision-language pre-trained models to overcome data scarcity challenges. However, existing methods simply transfer 2D alignment strategies, aligning 3D representations with single-view 2D images and coarse-grained parent category text. These approaches introduce information degradation and insufficient synergy issues, leading to performance loss. Information degradation arises from overlooking the fact that a 3D representation should be equivalent to a series of multi-view images and more fine-grained subcategory text. Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space, rather than independently aligning with each modality. In this paper, we propose Copyright © 2023, The Authors. All rights reserved.

关键词：