检索结果-内蒙古大学图书馆

Hyper-3dG: text-to-3d Gaussian generation via Hypergraph

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年第5期133卷 2886-2909页

作者： di, donglin Yang, Jiahui Luo, Chaofan Xue, Zhou Chen, Wei Yang, Xun Gao, Yue Li Auto Space AI Chengdu 101399 Peoples R China Tsinghua Univ Sch Software Beijing 100084 Peoples R China Univ Sci & Technol China Sch Informat Sci & Technol Hefei 230026 Anhui Peoples R China Harbin Inst Technol Harbin 150001 Heilongjiang Peoples R China

text-to-3d generation represents an exciting field that has seen rapid advancements, facilitating the transformation of textual descriptions into detailed 3d models. However, current progress often neglects the intricate high-order correlation of geometry and texture within 3d objects, leading to challenges such as over-smoothness, over-saturation and the Janus problem. In this work, we propose a method named "3d Gaussian generation via Hypergraph (Hyper-3dG)", designed to capture the sophisticated high-order correlations present within 3d objects. Our framework is anchored by a well-established mainflow and an essential module, named "Geometry and texture Hypergraph Refiner (HGRefiner)". This module not only refines the representation of 3d Gaussians but also accelerates the update process of these 3d Gaussians by conducting the Patch-3dGS Hypergraph Learning on both explicit attributes and latent visual features. Our framework allows for the production of finely generated 3d objects within a cohesive optimization, effectively circumventing degradation. Extensive experimentation has shown that our proposed method significantly enhances the quality of 3d generation while incurring no additional computational overhead for the underlying framework. (Project code: https://***/yjhboy/Hyper3dG).

关键词： text-to-3d generation 3d Gaussian Splatting Hypergraph learning

来源：评论

学校读者我要写书评

暂无评论

Instant3d: Instant text-to-3d generation

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2024年第10期132卷 4456-4472页

作者： Li, Ming Zhou, Pan Liu, Jia-Wei Keppo, Jussi Lin, Min Yan, Shuicheng Xu, Xiangyu Natl Univ Singapore Inst Data Sci Singapore Singapore Singapore Management Univ Sch Comp & Informat Syst Singapore Singapore Natl Univ Singapore Show Lab Singapore Singapore Natl Univ Singapore Business Sch Singapore Singapore Skywork AI Singapore Singapore Xi An Jiao Tong Univ Xian Peoples R China

text-to-3d generation has attracted much attention from the computer vision community. Existing methods mainly optimize a neural field from scratch for each text prompt, relying on heavy and repetitive training cost which impedes their practical deployment. In this paper, we propose a novel framework for fast text-to-3d generation, dubbed Instant3d. Once trained, Instant3d is able to create a 3d object for an unseen text prompt in less than one second with a single run of a feedforward network. We achieve this remarkable speed by devising a new network that directly constructs a 3d triplane from a text prompt. The core innovation of our Instant3d lies in our exploration of strategies to effectively inject text conditions into the network. In particular, we propose to combine three key mechanisms: cross-attention, style injection, and token-to-plane transformation, which collectively ensure precise alignment of the output with the input text. Furthermore, we propose a simple yet effective activation function, the scaled-sigmoid, to replace the original sigmoid function, which speeds up the training convergence by more than ten times. Finally, to address the Janus (multi-head) problem in 3d generation, we propose an adaptive Perp-Neg algorithm that can dynamically adjust its concept negation scales according to the severity of the Janus problem during training, effectively reducing the multi-head effect. Extensive experiments on a wide variety of benchmark datasets demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods both qualitatively and quantitatively, while achieving significantly better efficiency.

关键词： text-to-3d generation Large-scale generative models Neural radiance fields

来源：评论

学校读者我要写书评

暂无评论

VP3d: Unleashing 2d Visual Prompt for text-to-3d generation

VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generation

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Chen, Yang Pan, Yingwei Yang, Haibo Yao, Ting Mei, Tao HiDream Ai Inc Beijing Peoples R China Fudan Univ Sch Comp Sci Shanghai Peoples R China

ISBN: (纸本)9798350353013;9798350353006

Recent innovations on text-to-3d generation have fea-tured Score distillation Sampling (SdS), which enables the zero-shot learning of implicit 3d models (NeRF) by directly distilling prior knowledge from 2d diffusion models. However, current SdS-based models still struggle with intricate text prompts and commonly result in distorted 3d models with unrealistic textures or cross-view inconsistency issues. In this work, we introduce a novel Visual Prompt-guided text-to-3d diffusion model (VP3d) that explicitly unleashes the visual appearance knowledge in 2d visual prompt to boost text-to-3d generation. Instead of solely supervising SdS with text prompt, VP3d first capitalizes on 2d diffusion model to generate a high-quality image from input text, which subsequently acts as visual prompt to strengthen SdS optimization with explicit visual appearance. Meanwhile, we couple the SdS optimization with additional differentiable reward function that encourages rendering images of 3d models to better visually align with 2d visual prompt and semantically match with text prompt. Through extensive experiments, we show that the 2d Visual Prompt in our VP3d significantly eases the learning of visual appearance of 3d models and thus leads to higher visual fidelity with more detailed textures. It is also appealing in view that when replacing the self-generating visual prompt with a given reference image, VP3d is able to trigger a new task of stylized text-to-3d generation. Our project page is available at https://***.

关键词： 3d generation diffusion model image generation text-to-3d generation

来源：评论

学校读者我要写书评

暂无评论

Taming Mode Collapse in Score distillation for text-to-3d generation

Taming Mode Collapse in Score Distillation for Text-to-3D Ge...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Wang, Peihao Xu, dejia Fan, Zhiwen Wang, dilin Mohan, Sreyas Iandola, Forrest Ranjan, Rakesh Li, Yilei Liu, Qiang Wang, Zhangyang Chandra, Vikas Univ Texas Austin Austin TX 78712 USA Meta Real Labs Menlo Pk CA USA

ISBN: (纸本)9798350353006

despite the remarkable performance of score distillation in text-to-3d generation, such techniques notoriously suffer from view inconsistency issues, also known as "Janus" artifact, where the generated objects fake each view with multiple front faces. Although empirically effective methods have approached this problem via score debiasing or prompt engineering, a more rigorous perspective to explain and tackle this problem remains elusive. In this paper, we reveal that the existing score distillation-based text-to-3d generation frameworks degenerate to maximal likelihood seeking on each view independently and thus suffer from the mode collapse problem, manifesting as the Janus artifact in practice. To tame mode collapse, we improve score distillation by re-establishing the entropy term in the corresponding variational objective, which is applied to the distribution of rendered images. Maximizing the entropy encourages diversity among different views in generated 3d assets, thereby mitigating the Janus problem. Based on this new objective, we derive a new update rule for 3d score distillation, dubbed Entropic Score distillation (ESd). We theoretically reveal that ESd can be simplified and implemented by just adopting the classifier-free guidance trick upon variational score distillation. Although embarrassingly straightforward, our extensive experiments demonstrate that ESd can be an effective treatment for Janus artifacts in score distillation.

关键词： Entropy Maximization Model Collapse Score distillation text-to-3d generation

来源：评论

学校读者我要写书评

暂无评论

Control3d: Towards Controllable text-to-3d generation 23

Control3D: Towards Controllable Text-to-3D Generation

引用

31st ACM International Conference on Multimedia (MM)

作者： Chen, Yang Pan, Yingwei Li, Yehao Yao, Ting Mei, Tao Univ Sci & Technol China Hefei Peoples R China Sun Yat sen Univ Guangzhou Peoples R China HiDream ai Inc Beijing Peoples R China

ISBN: (纸本)9798400701085

Recent remarkable advances in large-scale text-to-image diffusion models have inspired a significant breakthrough in text-to-3d generation, pursuing 3d content creation solely from a given text prompt. However, existing text-to-3d techniques lack a crucial ability in the creative process: interactively control and shape the synthetic 3d contents according to users' desired specifications (e.g., sketch). To alleviate this issue, we present the first attempt for text-to-3d generation conditioning on the additional hand-drawn sketch, namely Control3d, which enhances controllability for users. In particular, a 2d conditioned diffusion model (ControlNet) is remoulded to guide the learning of 3d scene parameterized as NeRF, encouraging each view of 3d scene aligned with the given text prompt and hand-drawn sketch. Moreover, we exploit a pre-trained differentiable photo-to-sketch model to directly estimate the sketch of the rendered image over synthetic 3d scene. Such estimated sketch along with each sampled view is further enforced to be geometrically consistent with the given sketch, pursuing better controllable text-to-3d generation. Through extensive experiments, we demonstrate that our proposal can generate accurate and faithful 3d scenes that align closely with the input text prompts and sketches.

关键词： text-to-3d generation diffusion Model Sketch

来源：评论

学校读者我要写书评

暂无评论

text-image conditioned diffusion for consistent text-to-3d generation

引用

COMPUTER AIdEd GEOMETRIC dESIGN 2024年 111卷

作者： He, Yuze Bai, Yushi Lin, Matthieu Sheng, Jenny Hu, Yubin Wang, Qi Wen, Yu-Hui Liu, Yong-Jin Tsinghua Univ Dept Comp Sci & Technol Beijing 100084 Peoples R China Guizhou Univ Guiyang 550025 Guizhou Peoples R China Beijing Jiaotong Univ Sch Comp Sci & Technol Beijing Key Lab Traff Data Anal & Min Beijing 100044 Peoples R China

By lifting the pre -trained 2d diffusion models into Neural Radiance Fields (NeRFs), text -to3d generation methods have made great progress. Many state-of-the-art approaches usually apply score distillation sampling (SdS) to optimize the NeRF representations, which supervises the NeRF optimization with pre -trained text -conditioned 2d diffusion models such as Imagen. However, the supervision signal provided by such pre -trained diffusion models only depends on text prompts and does not constrain the multi -view consistency. To inject cross -view consistency into diffusion priors, some recent works finetune the 2d diffusion model via multi -view data, but still lack fine-grained view coherence. To tackle this challenge, we incorporate multi -view image conditions into the supervision signal of NeRF optimization, which explicitly enforces fine-grained view consistency . With such stronger supervision, our proposed text -to -3d method effectively mitigates the generation of floaters (due to excessive densities) and completely empty spaces (due to insufficient densities). Our quantitative evaluations on the T 3 Bench dataset demonstrate that our method achieves state-of-the-art performance over existing text -to -3d methods. We will make the code publicly available.

关键词： text-to-3d generation AIGC diffusion Neural radiance fields

来源：评论

学校读者我要写书评

暂无评论

Creating High-quality 3d Content by Bridging the Gap Between text-to-2d and text-to-3d generation

引用

ACM Transactions on Multimedia Computing, Communications, and Applications 1000年

作者： Yiwei Ma Yijun Fan Jiayi Ji Haowei Wang Haibing Yin Xiaoshuai Sun Rongrong Ji Xiamen University China Hangzhou Dianzi University China

In recent times, automatic text-to-3d content creation has made significant progress, driven by the development of pretrained 2d diffusion models. Existing text-to-3d methods typically optimize the 3d representation to ensure that the rendered image aligns well with the given text, as evaluated by the pretrained 2d diffusion model. Nevertheless, a substantial domain gap exists between 2d images and 3d assets, primarily attributed to variations in camera-related attributes and the exclusive presence of foreground objects. Consequently, employing 2d diffusion models directly for optimizing 3d representations may lead to suboptimal outcomes. To address this issue, we present X-dreamer, a novel approach for high-quality text-to-3d content creation that effectively bridges the gap between text-to-2d and text-to-3d synthesis. The key components of X-dreamer are two innovative designs: Camera-Guided Low-Rank Adaptation (CG-LoRA) and Attention-Mask Alignment (AMA) Loss. CG-LoRA dynamically incorporates camera information into the pretrained diffusion models by employing camera-dependent generation for trainable parameters. This integration makes the 2d diffusion model camera-sensitive. AMA loss guides the attention map of the pretrained diffusion model using the binary mask of the 3d object, prioritizing the creation of the foreground object. This module ensures that the model focuses on generating accurate and detailed foreground objects. Extensive evaluations demonstrate the effectiveness of our proposed method compared to existing text-to-3d approaches. Our project webpage: https://***/. Our code is available at https://***/xmu-xiaoma666/X-dreamer.

关键词： text-to-3d generation Vision and Language domain Gap

来源：评论

学校读者我要写书评

暂无评论

dreamFont3d: Personalized text-to-3d Artistic Font generation 24

DreamFont3D: Personalized Text-to-3D Artistic Font Generatio...

引用

SIGGRAPH Conference on Emerging Technologies

作者： Li, Xiang Meng, Lei Wu, Lei Li, Manyi Meng, Xiangxu Shandong Univ Jinan Peoples R China Shandong Res Inst Ind Technol Jinan Peoples R China

ISBN: (纸本)9798400705250

text-to-3d artistic font generation aims to assist users for innovative and customized 3d font design by exploring novel concepts and styles. despite of the advances in the text-to-3d tasks for general objects or scenes, the additional challenge of 3d font generation is to preserve the geometric structures of strokes in an appropriate extent, which determines the generation quality in terms of the recognizability and the local effect control of the 3d fonts. This paper presents a novel approach for text-to-3d artistic font generation, named dreamFont3d, which utilizes multi-view font masks and layout conditions to constrain the 3d font structure and local font effects. Specifically, to enhance the recognizability of 3d fonts, we propose the multi-view mask constraint (MC) to optimize the differentiable 3d representation while preserving the font structure. We also present a progressive mask weighting (MW) module to ensure a trade-off between the text-guided stylization of font effects and the mask-guided preservation of font structure. For precise control over local font effects, we design the multi-view attention modulation (AM) that guides the visual concepts to appear in specific regions according to the provided layout conditions. Compared with existing text-to-3d methods, dreamFont3d shows its own superiority in the consistency between font effects and text prompts, the recognizability, and the localization of font effects. Code and data at https://***/dreamFont3d/.

关键词： text-to-3d generation font generation diffusion model neural radiance fields

来源：评论

学校读者我要写书评

暂无评论

EGAS: Enhanced Geometry-aware 3d Asset generation Using Gaussian Splatting

EGAS: Enhanced Geometry-aware 3D Asset Generation Using Gaus...

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Hu, Shengjie Zhang, Xiaogang Chen, Hua Yan, Wenbin College of Electrical and Information Engineering Hunan University Changsha China College of Computer Science and Electronic Engineering Hunan University Changsha China

ISBN: (纸本)9798350368741

Current text-to-3d generation methods that rely solely on 2d diffusion supervision often suffer from incorrect geometry (e.g. geometric collapse) and unrealistic appearance (e.g. Janus issues) due to the inherent ambiguity in 2d lifting methods for scene representation optimization, which also leads to significant time consumption. In this paper, we introduce EGAS, a novel 3dG-based framework with enhanced geometry awareness, comprising geometry generation and texture refinement. Specifically, we extract pseudo-depth constraints and preliminary appearance guidance from coarse 3d priors to modulate the optimization direction and encourage consistent representation. Moreover, the reasonable geomtry formation is further secured through elaborately designed unsupervised constraints leveraging the particle property of 3d Gaussians. Extensive experiments demonstrate that our method achieves an excellent balance between efficiency and effectiveness, resulting in generated 3d assets with reasonable geometry, high fidelity appearance, and intricate details. © 2025 IEEE.

关键词： 3d gaussian splatting text-to-3d generation

来源：评论

学校读者我要写书评

暂无评论

Freestyle 3d-Aware Portrait Synthesis Based on Compositional Generative Priors 27th

Freestyle 3D-Aware Portrait Synthesis Based on Compositiona...

引用

27th International Conference on Pattern Recognition, ICPR 2024

作者： Ma, Tianxiang Zhao, Kang Sun, Jianxin Zhang, Yingya dong, Jing School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China CRIPAC MAISInstitute of Automation Chinese Academy of Sciences Beijing China Alibaba Group Hangzhou China

ISBN: (纸本)9783031781711

Efficiently generating a freestyle 3d portrait with high quality and 3d-consistency is a promising yet challenging task. The portrait styles generated by most existing methods are usually restricted by their 3d generators, which are learned in specific facial datasets, such as FFHQ. To get the diverse 3d portraits, one can build a large-scale multi-style database to retrain a 3d-aware generator, or use a off-the-shelf tool to do the style translation. However, the former is time-consuming due to data collection and training process, the latter may destroy the multi-view consistency. To tackle this problem, we propose a novel text-driven 3d-aware portrait synthesis framework that can generate out-of-distribution portrait styles. Specifically, for a given portrait style prompt, we first composite two generative priors, a 3d-aware GAN generator and a text-guided image editor, to quickly construct a few-shot stylized portrait set. Then we map the special style domain of this set to our proposed 3d latent feature generator and obtain a 3d representation containing the given style information. Finally we use a pre-trained 3d renderer to generate view-consistent stylized portraits from the 3d representation. Extensive experimental results show that our method is capable of synthesizing high-quality 3d portraits with specified styles in a few minutes, outperforming the state-of-the-art. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： 3d Portrait Synthesis text-to-3d generation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：