检索结果-内蒙古大学图书馆

arXiv 2025年

作者： Li, Haoyuan Zhou, Yanpeng Tang, Tao Song, Jifei Zeng, Yihan Kampffmeyer, Michael Xu, Hang Liang, Xiaodan Shenzhen campus of Sun Yat-sen University China Huawei Noah’s Ark Lab Canada UiT The Arctic University of Norway Norway Peng Cheng Laboratory China Guangdong Key Laboratory of Big Data Analysis and Processing China

Recent advancements in multi-modal 3D pre-training methods have shown promising efficacy in learning joint representations of text, images, and point clouds. However, adopting point clouds as 3D representation fails to fully capture the intricacies of the 3D world and exhibits a noticeable gap between the discrete points and the dense 2D pixels of images. To tackle this issue, we propose UniGS, integrating 3D Gaussian Splatting (3DGS) into multi-modal pre-training to enhance the 3D representation. We first rely on the 3DGS representation to model the 3D world as a collection of 3D Gaussians with color and opacity, incorporating all the information of the 3D scene while establishing a strong connection with 2D images. Then, to achieve Language-Image-3D pertaining, UniGS starts with a pre-trained vision-language model to establish a shared visual and textual space through extensive real-world image-text pairs. Subsequently, UniGS employs a 3D encoder to align the optimized 3DGS with the Language-Image representations to learn unified multi-modal representations. To facilitate the extraction of global explicit 3D features by the 3D encoder and achieve better cross-modal alignment, we additionally introduce a novel Gaussian-Aware Guidance module that guides the learning of fine-grained representations of the 3D domain. Through extensive experiments across the Objaverse, ABO, MVImgNet and SUN RGBD datasets with zero-shot classification, text-driven retrieval and open-world understanding tasks, we demonstrate the effectiveness of UniGS in learning a more general and stronger aligned multi-modal representation. Specifically, UniGS achieves leading results across different 3D tasks with remarkable improvements over previous SOTA, Uni3D, including on zero-shot classification (+9.36%), text-driven retrieval (+4.3%) and open-world understanding (+7.92%). © 2025, CC BY-NC-SA.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

UniAdapter: All-in-One Control for Flexible Video Generation

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Wang, Cong Hu, Panwen Zhao, Haoyu Guo, Yuanfan Gu, Jiaxi Dong, Xiao Han, Jianhua Xu, Hang Liang, Xiaodan Chinese University of Hong Kong Hong Kong School of Science and Engineering Shenzhen China Fudan University School of Computer Science and Technology Shanghai China Huawei Noah'ark Lab Shanghai201206 China Sun Yat-Sen University School of Artificial Intelligence Zhuhai Campus Zhuhai519082 China Shenzhen Campus of Sun Yat-sen University School of Intelligent Systems Engineering Guangdong No. 66 Gongchang Road Guangming District Shenzhen518107 China Peng Cheng Laboratory and Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou510006 China

Condition-based video generation aims to create video content based on given information that describes specific subjects. However, most existing works can only utilize a single condition to guide the denoising process, thereby limiting their applicability to specific scenarios. Although some works attempt to accommodate multiple conditions within one framework, they often require multiple encoders, leading to inefficiencies in integrating multi-condition features. In this work, we present a framework that, with the support of the proposed Unified Adapter (UniAdapter), enables simultaneous multi-condition control of video generation within a single model. To effectively merge these conditions, we propose a novel Probabilistic Multi-condition Concatenator (PMC) module, which employs a unified encoder to accommodate multiple conditions and concatenate condition features at the pixel level to achieve fine-grained control. Following the PMC module, we employ 2D down-sampling blocks to refine features for injection into the Video Diffusion Model (VDM). Moreover, our UniAdapter is designed to be model-agnostic and compatible with any U-Net-based VDM, offering a versatile solution for improving video generation quality. Experimental results on public benchmarks UCF-101 and MSR-VTT show that our method achieves superior results in both quantitative and qualitative evaluations. © 2025 IEEE.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

Semantic Feature Division Multiple Access for Digital Semantic Multiple Access Channels

引用

IEEE Transactions on Cognitive Communications and Networking 2025年

作者： Shen, Bin Ma, Shuai Chen, Rui Wu, Youlong Li, Hang Shi, Guangming Li, Shiyin Al-Dhahir, Naofal China University of Mining and Technology School of Information and Control Engineering Xuzhou221116 China Xidian University State Key Laboratory of Integrated Service Networks School of Telecommunications Engineering Shaanxi Xi'an710071 China ShanghaiTech University School of Information Science and Technology Shanghai201210 China Shenzhen Research Institute of Big Data Guangdong Shenzhen518172 China Xidian University School of Artificial Intelligence Shaanxi Xi'an710071 China Peng cheng Laboratory Shenzhen518066 China The University of Texas at Dallas Department of Electrical and Computer Engineering RichardsonTX75080 United States

To solve the multi-user semantic communication network capacity bottlenecks, this paper proposes a semantic feature domain multiple access (SFDMA) scheme. By extracting and encoding semantic features and jointly processing multi-user data, the scheme can realize the encoding and transmission of multi-user semantic information in the separated feature subspace. Based on this scheme, a semantic digital multiple access network model is proposed to perform multi-user inference tasks, and an efficient semantic encoding and decoding scheme is developed by using variational information bottleneck (VIB) theory to ensure both the orthogonality in the feature domain and the classification accuracy. Furthermore, a SFDMA model for image reconstruction is designed based on the Swin Transformer. The modified model also considers the reconstruction quality and orthogonality of semantic feature domain. In addition, we design an optimization target for multiple access to implement an adaptive power allocation method for multi-user semantic communications. Simulation results show that our proposed SFDMA scheme exhibits semantic feature domain orthogonality in different applications, and the adaptive power allocation method shows excellent performance in the classification task, and effectively guarantee the quality of service (QoS) of the semantic communications. © 2015 IEEE.

关键词： Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

VBSSR: Variable Bitrate Encoded Video Streaming with Super-Resolution on HPC Education Platform 11th

VBSSR: Variable Bitrate Encoded Video Streaming with Super-R...

引用

11th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2020

作者： Wu, Run Zhou, Gangqiang Hu, Miao Wu, Di Department of Computer Science Sun Yat-sen University Guangzhou China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Peng Cheng Laboratory Shenzhen China

ISBN: (纸本)9789811600098

In online HPC education platforms, a large amount of educational resources are in the form of video. It is desirable to provide better QoE (Quality of Experience) for students when they are viewing these educational resources. Compared to traditional constant bitrate (CBR) encoding, variable bitrate (VBR) encoding can achieve better video quality and reduce network bandwidth. However, previous adaptive bitrate (ABR) schemes were commonly designed for CBR encoded videos. Such ABR schemes are not suitable to stream VBR encoded videos whose chunk sizes fluctuate rapidly. In this paper, we propose a novel ABR scheme call VBSSR, which takes the characteristics of VBR encoded video into consideration. The basic idea of VBSSR is to stream video chunks with complex scenes at a low bitrate level to reduce bandwidth consumption, and then boost the video quality by leveraging the technique of super-resolution (SR) at the client-side. VBSSR trains a deep reinforcement learning (DRL) based neural model to jointly make bitrate selections and decide which chunks to be enhanced. We conduct extensive trace-driven evaluations to compare VBSSR with other state-of-the-art methods. The experiment results show that our method significantly outperforms existing approaches with improvements in the average video quality by at least 37.1% while reducing the rebuffering time by 24.3%–75.9%. © 2021, Springer Nature Singapore Pte Ltd.

关键词： Video streaming

来源：评论

学校读者我要写书评

暂无评论

Inferring Prerequisite Relationships Among Learning Resources for HPC Education 11th

Inferring Prerequisite Relationships Among Learning Resource...

引用

11th International Symposium on Parallel Architectures, Algorithms and Programming, PAAP 2020

作者： Wu, Run Luo, Chenyu Hu, Miao Wu, Di School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Peng Cheng Laboratory Shenzhen China

ISBN: (纸本)9789811600098

For online HPC (High-Performance Computing) education, identifying the prerequisite relationships among different HPC learning resources is important for generating learning paths and personalized course recommendations for students. Previously, such prerequisite relationships were often created by domain experts manually. In this paper, we propose a novel framework to infer both concept and learning resource prerequisite relationships and integrate such a framework into an online HPC education platform. We first construct a bi-directional long short-term (BiLSTM) neural network with attention mechanism to automatically mine the latent semantic features from the formal description of concepts. Then, we input two kinds of features into a fully connected neural network for classification and obtain the relationships among concepts. Considering the asymmetry and directivity of the prerequisite relation, we adopt a network embedding model to learn the vector representations of each concept as the prerequisite and subsequent roles. The concept vectors are weighted and summed to generate the feature vectors of learning resources. For a given pair of learning resources, their feature vectors are input into a classifier that outputs the result of relation classification. Finally, we conduct a series of experiments on two large benchmark datasets. The experimental results show that our method outperforms existing methods with significant improvements. © 2021, Springer Nature Singapore Pte Ltd.

关键词： Vectors

来源：评论

学校读者我要写书评

暂无评论

Multimodal Emotional Talking Face Generation based on Action Units

引用

IEEE Transactions on Circuits and Systems for Video Technology 2024年第5期35卷 4026-4038页

作者： Lyu, Jiayi Lan, Xing Hu, Guohong Jiang, Hanyu Gan, Wei Wang, Jinbao Xue, Jian University of Chinese Academy of Sciences School of Engineering Science Beijing100049 China Peng Cheng Laboratory Shenzhen518055 China Shenzhen University National Engineering Laboratory for Big Data System Computing Shenzhen518060 China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen518060 China

Talking face generation focuses on creating natural facial animations that align with the provided text or audio input. Current methods in this field primarily rely on facial landmarks to convey emotional changes. However, spatial key-points are valuable, yet limited in capturing the intricate dynamics and subtle nuances of emotional expressions due to their restricted spatial coverage. Consequently, this reliance on sparse landmarks can result in decreased accuracy and visual quality, especially when representing complex emotional states. To address this issue, we propose a novel method called Emotional Talking with Action Unit (ETAU), which seamlessly integrates facial Action Units (AUs) into the generation process. Unlike previous works that solely rely on facial landmarks, ETAU employs both Action Units and landmarks to comprehensively represent facial expressions through interpretable representations. Our method provides a detailed and dynamic representation of emotions by capturing the complex interactions among facial muscle movements. Moreover, ETAU adopts a multi-modal strategy by seamlessly integrating emotion prompts, driving videos, and target images, and by leveraging various input data effectively, it generates highly realistic and emotional talking-face videos. Through extensive evaluations across multiple datasets, including MEAD, LRW, GRID and HDTF, ETAU outperforms previous methods, showcasing its superior ability to generate high-quality, expressive talking faces with improved visual fidelity and synchronization. Moreover, ETAU exhibits a significant improvement on the emotion accuracy of the generated results, reaching an impressive average accuracy of 84% on the MEAD dataset. © 1991-2012 IEEE.

关键词： Faces Face recognition Head Accuracy Mouth Feature extraction Lips Facial muscles Facial animation Diffusion models

来源：评论

学校读者我要写书评

暂无评论

Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method

arXiv

引用

arXiv 2024年

作者： Song, Xinshuai Chen, Weixing Liu, Yang Chen, Weikai Li, Guanbin Lin, Liang Sun Yat-sen University China Peng Cheng Laboratory China Guangdong Key Laboratory of Big Data Analysis and Processing China

Existing Vision-Language Navigation (VLN) methods primarily focus on single-stage navigation, limiting their effectiveness in multi-stage and long-horizon tasks within complex and dynamic environments. To address these limitations, we propose a novel VLN task, named Long-Horizon Vision-Language Navigation (LH-VLN), which emphasizes long-term planning and decision consistency across consecutive subtasks. Furthermore, to support LH-VLN, we develop an automated data generation platform NavGen, which constructs datasets with complex task structures and improves data utility through a bidirectional, multi-granularity generation approach. To accurately evaluate complex tasks, we construct the Long-Horizon Planning and Reasoning in VLN (LHPR-VLN) benchmark consisting of 3,260 tasks with an average of 150 task steps, serving as the first dataset specifically designed for the long-horizon vision-language navigation task. Furthermore, we propose Independent Success Rate (ISR), Conditional Success Rate (CSR), and CSR weight by Ground Truth (CGT) metrics, to provide fine-grained assessments of task completion. To improve model adaptability in complex tasks, we propose a novel Multi-Granularity Dynamic Memory (MGDM) module that integrates short-term memory blurring with long-term memory retrieval to enable flexible navigation in dynamic environments. Our platform, benchmark and method supply LH-VLN with a robust data generation pipeline, comprehensive model evaluation dataset, reasonable metrics, and a novel VLN model, establishing a foundational framework for advancing LH-VLN. Copyright © 2024, The Authors. All rights reserved.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

WhodunitBench: Evaluating Large Multimodal Agents via Murder Mystery Games 38

WhodunitBench: Evaluating Large Multimodal Agents via Murder...

引用

38th Conference on Neural Information processing Systems, NeurIPS 2024

作者： Xie, Junlin Zhang, Ruifei Chen, Zhihong Wan, Xiang Li, Guanbin The Chinese University of Hong Kong Shenzhen China Shenzhen Research Institute of Big Data China Sun Yat-sen University China Peng Cheng Laboratory China GuangDong Province Key Laboratory of Information Security Technology China

Recently, large language models (LLMs) have achieved superior performance, empowering the development of large multimodal agents (LMAs). An LMA expected to perform practical tasks must possess a range of capabilities, including multimodal perception, interaction, reasoning, and decision-making skills. However, existing benchmarks are limited in assessing compositional skills and actions demanded by practical scenarios, where they primarily focused on single tasks and static scenarios. To bridge this gap, we introduce WhodunitBench, a benchmark rooted from murder mystery games, where players are required to utilize the aforementioned skills to achieve their objective (i.e., identifying the 'murderer' or hiding themselves), providing a simulated dynamic environment for evaluating LMAs. Specifically, WhodunitBench includes two evaluation modes. The first mode, the arena-style evaluation, is constructed from 50 meticulously curated scripts featuring clear reasoning clues and distinct murderers;The second mode, the chain of evaluation, consists of over 3000 curated multiple-choice questions and open-ended questions, aiming to assess every facet of the murder mystery games for LMAs. Experiments show that although current LMAs show promising performance in basic perceptual tasks, they are insufficiently equipped for complex multi-agent collaboration and multi-step reasoning tasks. Furthermore, the full application of the theory of mind to complete games in a manner akin to human behavior remains a significant challenge. We hope this work can illuminate the path forward, providing a solid foundation for the future development of LMAs. Our WhodunitBench is open-source and accessible at: https://***/jun0wanan/WhodunitBench-Murder_Mystery_Games. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation 38

AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmenta...

引用

38th Conference on Neural Information processing Systems, NeurIPS 2024

作者： Han, Boyu Xu, Qianqian Yang, Zhiyong Bao, Shilong Wen, Peisong Jiang, Yangbangyan Huang, Qingming Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China School of Computer Science and Tech. University of Chinese Academy of Sciences China Peng Cheng Laboratory China Key Laboratory of Big Data Mining and Knowledge Management CAS China

The Area Under the ROC Curve (AUC) is a well-known metric for evaluating instance-level long-tail learning problems. In the past two decades, many AUC optimization methods have been proposed to improve model performance under long-tail distributions. In this paper, we explore AUC optimization methods in the context of pixel-level long-tail semantic segmentation, a much more complicated scenario. This task introduces two major challenges for AUC optimization techniques. On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis. On the other hand, we find that mini-batch estimation of AUC loss in this case requires a larger batch size, resulting in an unaffordable space complexity. To address these issues, we develop a pixel-level AUC loss function and conduct a dependency-graph-based theoretical analysis of the algorithm's generalization ability. Additionally, we design a Tail-Classes Memory Bank (T-Memory Bank) to manage the significant memory demand. Finally, comprehensive experiments across various benchmarks confirm the effectiveness of our proposed AUCSeg method. The code is available at https://***/boyuh/AUCSeg. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Gaming at the Edge: A Weighted Congestion Game Approach for Latency-Sensitive Scheduling

Gaming at the Edge: A Weighted Congestion Game Approach for ...

引用

International Conference on Mobile Ad-hoc and Sensor Networks, MSN

作者： Xuezheng Liu Ke Liu Guoqiao Ye Miao Hu Yipeng Zhou Di Wu Sun Yat-sen University Guangzhou China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China Peng Cheng Laboratory Shenzhen China Macquarie University Sydney NSW Australia

ISBN: (纸本)9781665406697

The rapidly evolving technology of edge computing shows great potential in revolutionizing the market of cloud gaming. Edge computing can significantly lower the latency for better gaming experiences by performing computation at the proximity of game players. However, the acceleration of latency-sensitive cloud gaming services at the edge is challenging due to the heterogeneity of edge servers, different requirements among game players, and so on. In this paper, we propose an efficient latency-sensitive scheduling algorithm called EGSA to satisfy latency constraints for cloud gaming services at the edge. We first formulate the problem as a weighted congestion game, which takes a number of key factors (e.g., game genres, user strategies, latency constraint and device heterogeneity) into account. Based on the weighted congestion game model, we further design an efficient latency-sensitive scheduling algorithm, which can approximate the pure Nash equilibrium under Shapley cost-sharing method. We also perform theoretic analysis to prove that our proposed algorithm converges in polynomial steps. Finally, we conduct a set of experiments and the results show that our algorithm outperforms alternative strategies with up to 46% performance improvement.

关键词： Performance evaluation Graphics Scheduling algorithms Cloud gaming Approximation algorithms Nash equilibrium Rendering (computer graphics)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：