检索结果-内蒙古大学图书馆

RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Xiao, Junjin Zhang, Qing Nie, Yonewei Zhu, Lei Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China South China University of Technology China China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

This paper presents RoGSplat, a novel approach for synthesizing high-fidelity novel views of unseen human from sparse multi-view images, while requiring no cumbersome per-subject optimization. Unlike previous methods that typically struggle with sparse views with few overlappings and are less effective in reconstructing complex human geometry, the proposed method enables robust reconstruction in such challenging conditions. Our key idea is to lift SMPL vertices to dense and reliable 3D prior points representing accurate human body geometry, and then regress human Gaussian parameters based on the points. To account for possible misalignment between SMPL model and images, we propose to predict image-aligned 3D prior points by leveraging both pixel-level features and voxel-level features, from which we regress the coarse Gaussians. To enhance the ability to capture high-frequency details, we further render depth maps from the coarse 3D Gaussians to help regress fine-grained pixel-wise Gaussians. Experiments on several benchmark datasets demonstrate that our method outperforms state-of-the-art methods in novel view synthesis and cross-dataset generalization. Our code is available at https: //***/iSEE-laboratory/RoGSplat. Copyright © 2025, The Authors. All rights reserved.

关键词： Pixels

ThicknesVAE: Learning a Latera Prior for Clothed Human Body Reconsuction

学校读者我要写书评

暂无评论

ThicknesVAE: Learning a Latera Prior for Clothed Human Body ...

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Wu, Xiaotao Fan, Zhaoxin He, Huiguang Shen, Dinggang School of Biomedical Engineering State Key Laboratory of Advanced Medical Materials and Devices ShanghaiTech University Shanghai201210 China NeuBCI Group State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology Institute of Automation Chinese Academy of Sciences Beijing China Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing Institute of Artificial Intelligence Beihang University Beijing100191 China Beijing Academy of Blockchain and Edge Computing China Shanghai United Imaging Intelligence Co. Ltd. Shanghai200232 China Shanghai Clnical Research and Trial Center Shanghai201210 China

ISBN: (纸本)9798350368741

Sandwich-like structures have shown remarkable efficacy in clothed human reconstruction. However, these approaches often generate unrealistic side geometries due to inadequate handling of lateral regions. This paper addresses this limitation by incorporating the side geometry of clothed humans as a prior. We propose ThicknessVAE, a novel two-stage method that makes two key contributions: (1) We learn a prototype from point clouds for the lateral regions of clothed humans to extract common and detailed geometric features. (2) We utilize this prototype as a prior to transform geometric features into a thickness map associated with clothed human images, enabling refined normal integration for sandwich-like reconstruction methods. By seamlessly integrating our model into the sandwich-like reconstruction pipeline, we achieve highly realistic side views. Both qualitative and quantitative experiments demonstrate that our approach is comparable to state-of-the-art methods in terms of side-view realism. © 2025 IEEE.

关键词： 3D Human Reconsruction Normal Integraton Prior Learning

Augmenting Short Enrollment Speech via Synthesis for Target Speaker Extraction

学校读者我要写书评

暂无评论

Augmenting Short Enrollment Speech via Synthesis for Target ...

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Huang, Zikang Lin, Jingru Ge, Meng Jiang, Yu Wang, Xiaobao Wang, Longbiao Dang, Jianwu Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Shenzhen China Department of Electrical and Computer and Engineering National University of Singapore Singapore Co. Ltd. Tianjin China Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China

ISBN: (纸本)9798350368741

A high-quality enrollment speech is crucial to target speaker extraction (TSE), since it provides essential cues for identifying the target speaker in the mixture. However, real applications usually only permit a short enrollment speech, e.g. a wakeup word for a mobile device, that provides limited cues. To address this issue, we propose an enrollment augmentation strategy that allows us to enrich the limited enrollment speech with massive text data through speech synthesis. By doing so, the extended enrollment speech contains enhanced speaker timbre and phonetic content which leads to better extraction quality. Furthermore, we propose a training data augmentation strategy to improve the model's robustness and generalization in short enrollment speech scenarios. Experiments on Libri2Mix demonstrate that our proposed strategies bring a significant improvement in extreme scenarios where only 0.5s and 1-word enrollment speech is provided. We also release our code at https://***/HuangZikang-TJU/Aug4TSE. © 2025 IEEE.

关键词： continue speaking data augmentation short enrollment speech synthesis Target speaker extraction

LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fu, Shenghao Yang, Qize Mo, Qijie Yan, Junkai Wei, Xihan Meng, Jingke Xie, Xiaohua Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Tongyi Lab Alibaba Group China Peng Cheng Laboratory China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Guangdong Province Key Laboratory of Information Security Technology China China

Recent open-vocabulary detectors achieve promising performance with abundant region-level annotated data. In this work, we show that an open-vocabulary detector co-training with a large language model by generating image-level detailed captions for each image can further improve performance. To achieve the goal, we first collect a dataset, GroundingCap-1M, wherein each image is accompanied by associated grounding labels and an image-level detailed caption. With this dataset, we finetune an open-vocabulary detector with training objectives including a standard grounding loss and a caption generation loss. We take advantage of a large language model to generate both region-level short captions for each region of interest and image-level long captions for the whole image. Under the supervision of the large language model, the resulting detector, LLMDet, outperforms the baseline by a clear margin, enjoying superior open-vocabulary ability. Further, we show that the improved LLMDet can in turn build a stronger large multi-modal model, achieving mutual benefits. The code, model, and dataset is available at https://***/iSEE-laboratory/LLMDet. Copyright © 2025, The Authors. All rights reserved.

关键词： Object detection

ViSpeak: Visual Instruction Feedback in Streaming Videos

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Fu, Shenghao Yang, Qize Li, Yuan-Ming Peng, Yi-Xing Lin, Kun-Yu Wei, Xihan Hu, Jian-Fang Xie, Xiaohua Zheng, Wei-Shi School of Computer Science and Engineering Sun Yat-sen University China Tongyi Lab Alibaba Group China Peng Cheng Laboratory China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China Guangdong Province Key Laboratory of Information Security Technology China China

Recent advances in Large Multi-modal Models (LMMs) are primarily focused on offline video understanding. Instead, streaming video understanding poses great challenges to recent models due to its time-sensitive, omni-modal and interactive characteristics. In this work, we aim to extend the streaming video understanding from a new perspective and propose a novel task named Visual Instruction Feedback in which models should be aware of visual contents and learn to extract instructions from them. For example, when users wave their hands to agents, agents should recognize the gesture and start conversations with welcome information. Thus, following instructions in visual modality greatly enhances user-agent interactions. To facilitate research, we define seven key subtasks highly relevant to visual modality and collect the ViSpeak-Instruct dataset for training and the ViSpeak-Bench for evaluation. Further, we propose the ViSpeak model, which is a SOTA streaming video understanding LMM with GPT-4o-level performance on various streaming video understanding benchmarks. After finetuning on our ViSpeak-Instruct dataset, ViSpeak is equipped with basic visual instruction feedback ability, serving as a solid baseline for future research. Copyright © 2025, The Authors. All rights reserved.

关键词： Video streaming

Discrete Unit-based Low-latency Multi-lingual Speech Synthesis for LIMMITS’25 Challenge

学校读者我要写书评

暂无评论

Discrete Unit-based Low-latency Multi-lingual Speech Synthes...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Yu Jiang Cheng Gong Tianrui Wang Chunyu Qiang Haoyu Wang Qiuyu Liu Yuheng Lu Xiaobao Wang Xiaolei Zhang Longbiao Wang Jianwu Dang Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Institute of Artificial Intelligence (TeleAI) China Telecom Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

In this paper, we present the system developed by our team, CCATTS, for the LIMMITS’25 challenge, focusing on few-shot and zero-shot TTS. We adopt a two-stage TTS strategy. In track 1, we fine-tune the pre-trained ZMM-TTS model and successfully achieve multilingual low-latency TTS. In track 2, we propose a token-based framework by modifying the first-stage model of ZMM-TTS, disentangling speech features into four types of discrete tokens—Content, Acoustic, Emotion, and Speaker—and integrating it with our designed token2wav module. This module consists of a HiFi-GAN-style decoder, an acoustic refiner, and U-Net flow matching, to generate high-quality speech. The official competition results demonstrate that our method achieves strong performance in both tracks.

关键词： Representation learning Focusing Signal processing Acoustics Multilingual Text to speech Decoding Low latency communication

Short-term wind speed prediction method based on prior wind direction knowledge and multi-period decoupling

学校读者我要写书评

暂无评论

Engineering Applications of Artificial intelligence 2025年 153卷

作者： Shang, Zewen Li, Xuewei Liu, Zhiqiang Sun, Yingzhou Yu, Jian Yu, Mei Xu, Tianyi Xiong, Wei College of Intelligence and Computing Tianjin University Tianjin300350 China Tianjin Key Laboratory of Cognitive Computing and Application Tianjin300350 China Tianjin Key Laboratory of Advanced Networking Tianjin300350 China TCU School of Civil Engineering Tianjin Chengjian University Tianjin300384 China

Accurate wind speed prediction improves power system stability and efficiency. Wind direction contains airflow dynamic information, but existing methods fail to fully capture it. The multi-periodicity of meteorological data causes irregular wind speed fluctuations, complicating the capture of correlations and variances across periods. To address these issues, this paper proposes the Multi-Period Wind Direction Graph Network(MWGN). Gaussian Spatial Module(GSM) calculates the relationship between wind turbines using Gaussian distribution along the crosswind direction, thus enhancing airflow motion capture. Multi-Period Time Series Decoupling Module(MPTSDM) selects the primary periods by analyzing the frequency domain and extracts local correlation and periodic correlation, to better model the changing pattern of time series. Compared with the existing models, MWGN achieves the consistent state-of-the-art in three datasets. © 2025 Elsevier Ltd

关键词： Wind forecasting

Progressive Human Motion Generation Based on Text and Few Motion Frames

学校读者我要写书评

暂无评论

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Zeng, Ling-An Wu, Gaojie Wu, Ancong Hu, Jian-Fang Zheng, Wei-Shi Sun Yat-sen University School of Artificial Intelligence Guangdong Zhuhai519082 China Sun Yat-sen University School of Computer Science and Engineering Guangdong Guangzhou510275 China Sun Yat-sen University School of Computer Science and Engineering China Guangdong Key Laboratory of Information Security Technology China Ministry of Education Key Laboratory of Machine Intelligence and Advanced Computing China Peng Cheng Laboratory China

Although existing text-to-motion (T2M) methods can produce realistic human motion from text description, it is still difficult to align the generated motion with the desired postures since using text alone is insufficient for precisely describing diverse postures. To achieve more controllable generation, an intuitive way is to allow the user to input a few motion frames describing precise desired postures. Thus, we explore a new Text-Frame-to-Motion (TF2M) generation task that aims to generate motions from text and very few given frames. Intuitively, the closer a frame is to a given frame, the lower the uncertainty of this frame is when conditioned on this given frame. Hence, we propose a novel Progressive Motion Generation (PMG) method to progressively generate a motion from the frames with low uncertainty to those with high uncertainty in multiple stages. During each stage, new frames are generated by a Text-Frame Guided Generator conditioned on frame-aware semantics of the text, given frames, and frames generated in previous stages. Additionally, to alleviate the train-test gap caused by multi-stage accumulation of incorrectly generated frames during testing, we propose a Pseudo-frame Replacement Strategy for training. Experimental results show that our PMG outperforms existing T2M generation methods by a large margin with even one given frame, validating the effectiveness of our PMG. Code is available here. © 1991-2012 IEEE.

关键词： Controllable Motion Generation Human Motion Text2Motion Generation

Generative Visual Commonsense Answering and Explaining with Generative Scene Graph Constructing

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yuan, Fan Fang, Xiaoyuan Quan, Rong Li, Jing Bi, Wei Xu, Xiaogang Li, Piji College of Artificial Intelligence Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing211106 China College of Artificial Intelligence Nanjing University of Aeronautics and Astronautics Key Laboratory of Brain-Machine Intelligence Technology Ministry of Education Nanjing211106 China Department of Computing The Hong Kong Polytechnic University Hong Kong The Chinese University of Hong Kong Hong Kong

Visual Commonsense Reasoning, which is regarded as one challenging task to pursue advanced visual scene comprehension, has been used to diagnose the reasoning ability of AI systems. However, reliable reasoning requires a good grasp of the scene’s details. Existing work fails to effectively exploit the real-world object relationship information present within the scene, and instead overly relies on knowledge from training memory. Based on these observations, we propose a novel scene-graph-enhanced visual commonsense reasoning generation method named G2, which first utilizes the image patches and LLMs to construct a location-free scene graph, and then answer and explain based on the scene graph’s information. We also propose automatic scene graph filtering and selection strategies to absorb valuable scene graph information during training. Extensive experiments are conducted on the tasks and datasets of scene graph constructing and visual commonsense answering and explaining, respectively. Experimental results and ablation analysis demonstrate the effectiveness of our proposed framework. Copyright © 2025, The Authors. All rights reserved.

关键词： Question answering