检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Ge, Qihang Sun, Wei Zhang, Yu Li, Yunhao Ji, Zhongpeng Sun, Fengyu Jui, Shangling Min, Xiongkuo Zhai, Guangtao The Institute of Image Communication and Information Processing Shanghai Jiao Tong University Shanghai200240 China The MoE Key Laboratory of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai200240 China The Huawei Technologies Shanghai200240 China

The explosive growth of videos on streaming media platforms has underscored the urgent need for effective video quality assessment (VQA) algorithms to monitor and perceptually optimize the quality of streaming videos. However, VQA remains an extremely challenging task due to the diverse video content and the complex spatial and temporal distortions, thus necessitating more advanced methods to address these issues. Nowadays, large multimodal models (LMMs), such as GPT-4V, have exhibited strong capabilities for various visual understanding tasks, motivating us to leverage the powerful multimodal representation ability of LMMs to solve the VQA task. Therefore, we propose the first Large Multi-Modal based Video Quality Assessment (LMM-VQA) model, which introduces a novel spatiotemporal visual modeling strategy for quality-aware feature extraction. Specifically, we first reformulate the quality regression problem into a question and answering (Q&A) task and construct Q&A prompts for VQA instruction tuning. Then, we design a spatiotemporal vision encoder to extract spatial and temporal features to represent the quality characteristics of videos, which are subsequently mapped into the language space by the spatiotemporal projector for modality alignment. Finally, the aligned visual tokens and the quality-inquired text tokens are aggregated as inputs for the large language model (LLM) to generate the quality score as well as the quality level. Extensive experiments demonstrate that LMM-VQA achieves state-of-the-art performance across five VQA benchmarks, exhibiting an average improvement of 5% in generalization ability over existing methods. Furthermore, due to the advanced design of the spatiotemporal encoder and projector, LMM-VQA also performs exceptionally well on general video understanding tasks, further validating its effectiveness. Our code will be released at https://***/Sueqk/LMM-VQA. © 2024, CC BY.

关键词： Video streaming

来源：评论

学校读者我要写书评

暂无评论

UniAnimate: Taming Unified Video Diffusion Models for Consistent Human image Animation

arXiv

引用

arXiv 2024年

作者： Wang, Xiang Zhang, Shiwei Gao, Changxin Wang, Jiayu Zhou, Xiaoqiang Zhang, Yingya Yan, Luxin Sang, Nong Key Laboratory of Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Alibaba Group Hangzhou310052 China University of Science and Technology of China Hefei230026 China

Recent diffusion-based human image animation techniques have demonstrated impressive success in synthesizing videos that faithfully follow a given reference identity and a sequence of desired movement poses. Despite this, there are still two limitations: i) an extra reference model is required to align the identity image with the main video branch, which significantly increases the optimization burden and model parameters;ii) the generated video is usually short in time (e.g., 24 frames), hampering practical applications. To address these shortcomings, we present a UniAnimate framework to enable efficient and long-term human video generation. First, to reduce the optimization difficulty and ensure temporal coherence, we map the reference image along with the posture guidance and noise video into a common feature space by incorporating a unified video diffusion model. Second, we propose a unified noise input that supports random noised input as well as first frame conditioned input, which enhances the ability to generate long-term video. Finally, to further efficiently handle long sequences, we explore an alternative temporal modeling architecture based on state space model to replace the original computation-consuming temporal Transformer. Extensive experimental results indicate that UniAnimate achieves superior synthesis results over existing state-of-the-art counterparts in both quantitative and qualitative evaluations. Notably, UniAnimate can even generate highly consistent one-minute videos by iteratively employing the first frame conditioning strategy. Code and models will be publicly available. Project page: https://***/. Copyright © 2024, The Authors. All rights reserved.

关键词： Animation

来源：评论

学校读者我要写书评

暂无评论

Joint Masked Reconstruction and Contrastive Learning for Mining Interactions Between Proteins

arXiv

引用

arXiv 2025年

作者： Li, Jiang Wang, Xiaoping School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Institute of Artificial Intelligence Huazhong University of Science and Technology Wuhan430074 China Hubei Key Laboratory of Brain-Inspired Intelligent Systems Huazhong University of Science and Technology Wuhan430074 China Key Laboratory of Image Processing and Intelligent Control Huazhong University of Science and Technology Ministry of Education Wuhan430074 China

Protein-protein interaction (PPI) prediction is an instrumental means in elucidating the mechanisms underlying cellular operations, holding significant practical implications for the realms of pharmaceutical development and clinical treatment. Presently, the majority of research methods primarily concentrate on the analysis of amino acid sequences, while investigations predicated on protein structures remain in the nascent stages of exploration. Despite the emergence of several structure-based algorithms in recent years, these are still confronted with inherent challenges: (1) the extraction of intrinsic structural information of proteins typically necessitates the expenditure of substantial computational resources;(2) these models are overly reliant on seen protein data, struggling to effectively unearth interaction cues between unknown proteins. To further propel advancements in this domain, this paper introduces a novel PPI prediction method jointing masked reconstruction and contrastive learning, termed JmcPPI. This methodology dissects the PPI prediction task into two distinct phases: during the residue structure encoding phase, JmcPPI devises two feature reconstruction tasks and employs graph attention mechanism to capture structural information between residues;during the protein interaction inference phase, JmcPPI perturbs the original PPI graph and employs a multi-graph contrastive learning strategy to thoroughly mine extrinsic interaction information of novel proteins. Extensive experiments conducted on three widely utilized PPI datasets demonstrate that JmcPPI surpasses existing optimal baseline models across various data partition schemes. The associated code can be accessed via https://***/lijfrankopen/JmcPPI. Copyright © 2025, The Authors. All rights reserved.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

DAFormer: Depth-aware 3D Object Detection Guided by Camera Model via Transformers

DAFormer: Depth-aware 3D Object Detection Guided by Camera M...

引用

IEEE International Conference on Cyborg and Bionic Systems (CBS)

作者： Junbin Gao Hao Ruan Bingrong Xu Zhigang Zeng School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan China Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China School of Automation Wuhan University of Technology Wuhan China

3D object detection plays a critical role in autonomous driving perception. Although multi-view-based perception solutions have made significant progress, their performance is still far from being ready for practical use. The estimation of pixel depth is dependent on camera intrinsic properties, which led us to explore a depth-aware model guided by camera parameters. Our contribution in this paper is the DAFormer, which incorporates camera parameters and position-aware image features to detect 3D objects. The depthaware module uses camera parameters to reweight image features and estimate depth, enhancing and offsetting the 3D position embedding. The object query uses depth information enhanced features for end-to-end 3D detection by Attention layers. DAFormer achieves impressive results on the standard nuScenes dataset without any additional embellishments.

关键词： Solid modeling Three-dimensional displays Estimation Object detection Detectors Cameras Feature extraction

来源：评论

学校读者我要写书评

暂无评论

analysis of Government-Taxation Methods and Maritime Companies’ Strategies Under Emission-Reduction Incentives

SSRN

引用

SSRN 2023年

作者： Wang, Jian Wenbo, Zhu School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Key Laboratory of Image Processing and Intelligent Control Huazhong University of Science and Technology Ministry of Education Wuhan430074 China

The current emission-reduction trend in the shipping industry is inevitable. This study uses a multi-stage Stackelberg game to establish a low-carbon maritime supply chain model that includes a government, a carrier, and a port. The government can choose to impose carbon taxes only on the port or on both the port and carrier, while there are different power structures: port-led or carrier-led. Equilibrium solutions in different scenarios indicate that the leading port’s profit is not affected by the implementation of a carbon tax, while the carrier, surprisingly, hopes to join the carbon-tax policy. Both the government and the carrier prefer a bilateral carbon-tax scenario led by the carrier, while the port has the opposite preference. In the process of seeking coordination opportunities, we find that, driven by shippers’ low-carbon preferences, the maritime supply chain exhibits a "triple increasing" phenomenon, which means higher cargo volume, more emission reduction, and higher freight prices. The government and carrier can control the occurrence of this phenomenon and promote an all-win scenario through the use of carbon taxes and subsidies. We provide coordination strategies for unilateral carbon taxes in the maritime supply chain under different power structures and demonstrate the important role of the government in coordination. This provides a new direction for the implementation of carbon taxes in the maritime supply chain. © 2023, The Authors. All rights reserved.

关键词： Taxation

来源：评论

学校读者我要写书评

暂无评论

Enhancing Event Tagger with Automatic Speech Recognizer for Audio Multi-task Scenarios by Distillation with Pre-Trained Large Models

Enhancing Event Tagger with Automatic Speech Recognizer for ...

引用

IEEE International Conference on Systems, Man and Cybernetics

作者： Jianfeng Cheng Ye Liu Jian Yin Liangdao Wang Yan Pan Artificial Intelligence Department Lizhi Inc. Beijing & Guangzhou China School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China School of Artificial Intelligence Sun Yat-sen University Zhuhai China

ISBN: (数字)9781665410205

ISBN: (纸本)9781665410212

With the continuous expansion of robotics and digital humans in practical applications, the demand for the auditory system is becoming deeper, usually requiring more efficient speech recognition framework capabilities to handle multiple tasks and use fewer resources. In previous audio processing frameworks, each type of audio processing task typically requires constructing a standalone deep network model for training, which results in more training data and higher training time when constructing models for multi-task audio scenarios simultaneously. The recent improvement of audio models based on transformers have brought about methods that can handle multiple audio tasks concurrently. However, recent related methods still require retraining multi-task targets with an amount of data, and achieve the general effect after training for multi-task scenarios than the simple combination of standalone methods processed separately. In order to better build a model that can handle multiple audio tasks, we propose a novel framework of distillation through pre-trained large models for enhancing event tagger with automatic speech recognizer. Through multiple rounds of experiments on several audio datasets, it has been verified that the proposed framework can achieve better results than the baseline for multitasking, and comparative results with less parameters compared to the baselines for single-task scenarios.

关键词： Training Adaptation models Analytical models Training data Speech recognition Speech enhancement Multitasking Transformers Data models Robots

来源：评论

学校读者我要写书评

暂无评论

Computationally Expensive High-Dimensional Multiobjective Optimization via Surrogate-Assisted Reformulation and Decomposition

引用

IEEE Transactions on Evolutionary Computation 2024年 1-1页

作者： Pan, Linqiang Lin, Jianqing Wang, Handing He, Cheng Tan, Kay Chen Jin, Yaochu School of Artificial Intelligence and Automation Key Laboratory of Image Information Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan China School of Artificial Intelligence Xidian University Xi’an China School of Electrical and Electronic Engineering Huazhong University of Science and Technology Wuhan China Department of Computing The Hong Kong Polytechnic University Hong Kong Hong Kong School of Engineering Trustworth and General Artificial Intelligence Laboratory Westlake University Hangzhou China

In recent decades, various surrogate-assisted evolutionary algorithms (SAEAs) have been proposed to solve computationally expensive multiobjective optimization problems (EMOPs). Nevertheless, designing an SAEA to handle high-dimensional EMOPs and balance convergence, diversity, and computational complexity remains challenging. Here, we propose a two-phase SAEA (TP-SAEA), which follows the idea of convergence first and diversity second, for solving high-dimensional EMOPs. In Phase I, a surrogate-assisted problem reformulation method is proposed to fast-track the Pareto optimal set in association with some reference solutions. Specifically, the high-dimensional EMOP is reformulated into an expensive single-objective one with low-dimensional decision space. Then, the surrogate-assisted optimization is utilized to obtain well-converged solutions. In Phase II, the high-dimensional EMOP is decomposed into two subproblems to explore subregions of the decision space that can effectively promote the diversity of the solutions. The two subproblems are optimized independently via surrogate-assisted optimization, aiming to push the population towards different regions of the Pareto optimal front. Experiments are conducted on EMOPs with 100 to 500 decision variables compared with four state-of-the-art SAEAs. The proposed TP-SAEA obtains well-converged and diverse solutions with only 509 real function evaluations. Moreover, its superiority is examined in six real-world instances with up to 12,000 decision variables. IEEE

关键词： Iron

来源：评论

学校读者我要写书评

暂无评论

Progressive Semi-Decoupled Detector for Accurate Object Detection

引用

IEEE Transactions on Multimedia 2025年

作者： Han, Bo He, Lihuo Ke, Junjie Wu, Jinjian Gao, Xinbo Xidian University Visual Information Processing Laboratory School of Electronic Engineering Xi'an710071 China Xidian University School of Artificial Intelligence Xi'an710071 China Chongqing University of Posts and Telecommunications Chongqing Key Laboratory of Image Cognition Chongqing400065 China

Inconsistent accuracy between classification and localization tasks is a common challenge in modern object detection. Task decoupling, which employs distinct features or labeling strategies for each task, is a widely used approach to address this issue. Although it has led to noteworthy advancements, this approach is insufficient as it neglects task interdependence and lacks an explicit consistency constraint. To bridge this gap, this paper proposes the Progressive Semi-Decoupled Detector (ProSDD) to enhance both classification and localization accuracy. Specifically, a new detection head is designed that incorporates feature suppression and enhancement mechanism (FSEM) and bidirectional interaction module (BIM). Compared with the decoupled head, it not only filters out task-irrelevant information and enhances task-related information, but also avoids excessive decoupling at the feature level. Moreover, both FSEM and BIM are used multiple times, thus forming a progressive semi-decoupled head. Then, a novel consistency loss is proposed and integrated into the loss function of object detection, ensuring harmonic performance in classification and localization. Experimental results demonstrate that the proposed ProSDD effectively alleviates inconsistent accuracy and achieves high-quality object detection. Taking the pretrained ResNet-50 as the backbone, ProSDD achieves a remarkable 43.3 AP on the MS COCO dataset, surpassing contemporary state-of-the-art detectors by a substantial margin under the equivalent configurations. © 1999-2012 IEEE.

关键词： Object detection

来源：评论

学校读者我要写书评

暂无评论

CMCRD: Cross-Modal Contrastive Representation Distillation for Emotion Recognition

arXiv

引用

arXiv 2025年

作者： Kan, Siyuan Wu, Huanyu Cui, Zhenyao Huang, Fan Xu, Xiaolong Wu, Dongrui Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Wuhan Institute of Digital Engineering Wuhan430074 China Shanghai Jiao Tong University Shanghai200240 China

Emotion recognition is an important component of affective computing, and also human-machine interaction. Unimodal emotion recognition is convenient, but the accuracy may not be high enough;on the contrary, multi-modal emotion recognition may be more accurate, but it also increases the complexity and cost of the data collection system. This paper considers cross-modal emotion recognition, i.e., using both electroencephalography (EEG) and eye movement in training, but only EEG or eye movement in test. We propose cross-modal contrastive representation distillation (CMCRD), which uses a pre-trained eye movement classification model to assist the training of an EEG classification model, improving feature extraction from EEG signals, or vice versa. During test, only EEG signals (or eye movement signals) are acquired, eliminating the need for multi-modal data. CMCRD not only improves the emotion recognition accuracy, but also makes the system more simplified and practical. Experiments using three different neural network architectures on three multi-modal emotion recognition datasets demonstrated the effectiveness of CMCRD. Compared with the EEG-only model, it improved the average classification accuracy by about 6.2%. © 2025, CC BY.

关键词： Modal analysis

来源：评论

学校读者我要写书评

暂无评论

Short-Term Residential Load Forecasting Via Pooling-Ensemble Model with Smoothing Clustering

SSRN

引用

SSRN 2023年

作者： Fang, Hongliang Xiao, Jiang-Wen Wang, Yan-Wu School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Key Laboratory of Image Processing and Intelligent Control Huazhong University of Science and Technology Ministry of Education Wuhan430074 China

Short-term residential load forecasting is essential to demand side response. However, the frequent spikes in the load and the volatile daily load patterns make it difficult to accurately forecast the load. To deal with these problems, this paper proposes a smoothing clustering method for daily load clustering and a pooling-ensemble model for one day ahead load forecasting. The whole shortterm load forecasting framework in this paper contains three steps. Specifically and firstly, the states of the residents are obtained by clustering the daily load curves with the proposed smoothing clustering method. Secondly, a weighted mixed Markov model is built to predict the probability distribution of the load state in the next day. Thirdly, multiple predictors in the pooling-ensemble model are selected for different states and the load is forecasted by weighing the results of the multiple predictors based on the predicted states. Results of the case studies and comparison studies on two public datasets verify the advantages of the smoothing clustering method and the pooling-ensemble model. © 2023, The Authors. All rights reserved.

关键词： Probability distributions

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：