检索结果-内蒙古大学图书馆

Exploring Rich Subjective Quality Information for image Quality Assessment in the Wild

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Min, Xiongkuo Gao, Yixuan Cao, Yuqin Zhai, Guangtao Zhang, Wenjun Sun, Huifang Chen, Chang Wen Shanghai Jiao Tong University Institute of Image Communication and Network Engineering Shanghai200240 China Shenzhen518055 China The Hong Kong Polytechnic University Hong Kong Hong Kong

Traditional in the wild image quality assessment (IQA) models are generally trained with the quality labels of mean opinion score (MOS), while missing the rich subjective quality information contained in the quality ratings, for example, the standard deviation of opinion scores (SOS) or even distribution of opinion scores (DOS). In this paper, we propose a novel IQA method named RichIQA to explore the rich subjective rating information beyond MOS to predict image quality in the wild. RichIQA is characterized by two key novel designs: 1 a three-stage image quality prediction network which exploits the powerful feature representation capability of the Convolutional vision Transformer (CvT) and mimics the short-term and long-term memory mechanisms of human brain;2 a multi-label training strategy in which rich subjective quality information like MOS, SOS and DOS are concurrently used to train the quality prediction network. Powered by these two novel designs, RichIQA is able to predict the image quality in terms of a distribution, from which the mean image quality can be subsequently obtained. Extensive experimental results verify that the three-stage network is tailored to predict rich quality information, while the multi-label training strategy can fully exploit the potentials within subjective quality rating and enhance the prediction performance and generalizability of the network. RichIQA outperforms state-of-the-art competitors on multiple large-scale in the wild IQA databases with rich subjective rating labels. The code of RichIQA will be made publicly available on GitHub. © 2025 IEEE.

关键词： Distribution transformers

来源：评论

学校读者我要写书评

暂无评论

Exploring Bidirectional Bounds for Minimax-Training of Energy-Based Models

引用

International Journal of Computer Vision 2025年 1-22页

作者： Geng, Cong Wang, Jia Chen, Li Gao, Zhiyong Frellsen, Jes Hauberg, Søren China Mobile Research Institute Beijing China Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Shanghai China Department of Applied Mathematics and Computer Science Technical University of Denmark Kgs.@ Lyngby Denmark

Energy-based models (EBMs) estimate unnormalized densities in an elegant framework, but they are generally difficult to train. Recent work has linked EBMs to generative adversarial networks, by noting that they can be trained through a minimax game using a variational lower bound. To avoid the instabilities caused by minimizing a lower bound, we propose to instead work with bidirectional bounds, meaning that we maximize a lower bound and minimize an upper bound when training the EBM. We investigate four different bounds on the log-likelihood derived from different perspectives. We derive lower bounds based on the singular values of the generator Jacobian and on mutual information. To upper bound the negative log-likelihood, we consider a gradient penalty-like bound, as well as one based on diffusion processes. In all cases, we provide algorithms for evaluating the bounds. We compare the different bounds to investigate, the pros and cons of the different approaches. Finally, we demonstrate that the use of bidirectional bounds stabilizes EBM training and yields high-quality density estimation and sample generation.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Aggregate and Discriminate: Pseudo Clips-Guided Boundary Perception for Video Moment Retrieval

引用

IEEE Transactions on Multimedia 2025年

作者： Liu, Jin Zhang, Zongbing Su, Yuting Yang, Bing Min, Xiongkuo Zhai, Guangtao Tianjin University School of Electrical and Information Engineering Tianjin China China Academy of Electronics and Information Technology Beijing100041 China Shanghai Jiao Tong University Institute of Image Communication and Network Engineering Shanghai China

Video moment retrieval (VMR) aims to localize a video segment in an untrimmed video that is semantically relevant to a language query. The challenge of this task lies in effectively aligning the intricate and information-dense video modality with the succinctly summarized textual modality, and further localizing the starting and ending timestamps of the target moments. Previous works have attempted to achieve multi-granularity alignment of video and query in a coarse-to-fine manner, yet these efforts still fall short in addressing the inherent disparities in representation and information density between videos and queries, leading to modal misalignments. In this paper, we propose a progressive video moment retrieval framework, initially retrieving the most relevant and irrelevant video clips to the query as semantic guidance, thereby bridging the semantic gap between video modality and language modality. Futhermore, we introduce a pseudo clips guided aggregation module to aggregate densely relevant moment clips closer together and propose a discriminative boundary-enhanced decoder with the guidance of pseudo clips to push the semantically confusing proposals away. Extensive experiments on the Charades-STA, ActivityNet Captions and TACoS datasets demonstrate that our method outperforms existing methods. © 2025 IEEE.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Teaching LMMs for image Quality Scoring and Interpreting

arXiv

引用

arXiv 2025年

作者： Zhang, Zicheng Wu, Haoning Jia, Ziheng Lin, Weisi Zhai, Guangtao Institute of Image Communication and Network Engineering Shanghai Jiao Tong University China S-Lab Nanyang Technological University Singapore School of Computer Science and Engineering Nanyang Technological University Singapore

image quality scoring and interpreting are two fundamental components of image Quality Assessment (IQA). The former quantifies image quality, while the latter enables descriptive question answering about image quality. Traditionally, these two tasks have been addressed independently. However, from the perspective of the Human Visual System (HVS) and the Perception-Decision Integration Model, they are inherently interconnected: interpreting serves as the foundation for scoring, while scoring provides an abstract summary of interpreting. Thus, unifying these capabilities within a single model is both intuitive and logically coherent. In this paper, we propose Q-SIT (Quality Scoring and Interpreting joint Teaching), a unified framework that enables large multimodal models (LMMs) to learn both image quality scoring and interpreting simultaneously. We achieve this by transforming conventional IQA datasets into learnable question-answering datasets and incorporating human-annotated quality interpreting data for training. Furthermore, we introduce an efficient scoring & interpreting balance strategy, which first determines the optimal data mix ratio on lightweight LMMs and then maps this ratio to primary LMMs for fine-tuning adjustment. This strategy not only mitigates task interference and enhances cross-task knowledge transfer but also significantly reduces computational costs compared to direct optimization on full-scale LMMs. With this joint learning framework and corresponding training strategy, we develop Q-SIT, the first model capable of simultaneously performing image quality scoring and interpreting tasks, along with its lightweight variant, Q-SIT-MINI. Experimental results demonstrate that Q-SIT achieves strong performance in both tasks with superior generalization IQA abilities, while Q-SIT-MINI significantly reduces computational overhead while maintaining competitive performance. Project page at https://***/Q-Future/Q-SiT. © 2025, CC BY.

关键词： Personnel training

来源：评论

学校读者我要写书评

暂无评论

ScanDTM: A Novel Dual-Temporal Modulation Scanpath Prediction Model for Omnidirectional images

引用

IEEE Transactions on Circuits and Systems for Video Technology 2025年

作者： Zhu, Dandan Zhang, Kaiwei Min, Xiongkuo Zhai, Guangtao Yang, Xiaokang East China Normal University School of Computer Science and Technology Shanghai200333 China Shanghai Jiao Tong University Institute of Image Communication and Network Engineering Shanghai200240 China Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence AI Institute Shanghai200240 China

Scanpath prediction for omnidirectional images aims to effectively simulate the human visual perception mechanism to generate dynamic realistic fixation trajectories. However, the majority of scanpath prediction methods for omnidirectional images are still in their infancy as they fail to accurately capture the time-dependency of viewing behavior and suffer from sub-optimal performance along with limited generalization capability. A desirable solution should achieve a better trade-off between prediction performance and generalization ability. To this end, we propose a novel dual-temporal modulation scanpath prediction (ScanDTM) model for omnidirectional images. Such a model is designed to effectively capture long-range time-dependencies between various fixation regions across both internal and external time dimensions, thereby generating more realistic scanpaths. In particular, we design a Dual Graph Convolutional network (Dual-GCN) module comprising a semantic-level GCN and an image-level GCN. This module servers as a robust visual encoder that captures spatial relationships among various object regions within an image and fully utilizes similar images as complementary information to capture similarity relations across relevant images. Notably, the proposed Dual-GCN focuses on modeling temporal correlations from both local and global perspectives within the internal time dimension. Furthermore, drawing inspiration from the promising generalization capabilities of diffusion models across various generative tasks, we introduce a novel diffusion-guided saliency module. This module formulates the prediction issue as a conditional generative process for the saliency map, utilizing extracted semantic-level and image-level visual features as conditions. With the well-designed diffusion-guided saliency module, our proposed ScanDTM model acting as an external temporal modulator, we can progressively refine the generated scanpath from the noisy map. We conduct extensive expe

关键词： Prediction models

来源：评论

学校读者我要写书评

暂无评论

E2DAS: An Efficient Equivariant Dynamic Aggregation Saliency Model for Omnidirectional images 27th

E2DAS: An Efficient Equivariant Dynamic Aggregation Saliency...

引用

27th International Conference on Pattern Recognition, ICPR 2024

作者： Zhang, Nana Liu, Qian Zhu, Dandan Zhu, Kun Zhai, Guangtao Yang, Xiaokang School of Computer Science and Technology Donghua University No.2999 Renmin North Road Songjiang District Shanghai201620 China Institute of AI Education Shanghai East China Normal University No.3663 Zhongshan North Road Putuo District Shanghai200333 China Key Laboratory of Embedded System and Service Computing Ministry of Education Tongji University No.4800 Cao’an Highway Jiading District Shanghai201804 China Institute of Image Communication and Network Engineering Shanghai Jiao Tong University No. 800 Dongchuan Road Minhang District Shanghai200240 China

ISBN: (纸本)9783031781216

Recent years have witnessed rapid progress of convolutional neural networks (CNNs) and their successful application in the task of saliency prediction for omnidirectional images (ODIs). Albeit achieving tremendous performance improvements, these CNNs-based saliency models are plagued by two major shortcomings: spatial content-agnostic and computationally intensive. Inspired by the effectiveness of equivariant network in the majority of computer vision tasks, we propose a novel efficient equivariant dynamic aggregation saliency (E2DAS) model to efficiently tackle the issue of human fixation prediction in ODIs. To be specific, our proposed model consists of an efficient equivariant module, a dynamic convolutional aggregation module, and an optimization computation module. Different from existing saliency models for ODIs, we are the first attempt to introduce an efficient equivariant dynamic convolutional aggregation operation into the saliency prediction task, which can fundamentally alleviate the projection distortion problem and can effectively learn spatial content-adaptive features. Moreover, we clearly observe a considerable decrease in the number of parameters resulting from the replacement of standard convolution with dynamic convolution aggregation. Extensive experiments on several benchmark datasets show the proposed model’s superiority over other state-of-the-art methods in terms of performance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Equivariant dynamic aggregation light-weight model omnidirectional images saliency prediction spatial content-adaptive

来源：评论

学校读者我要写书评

暂无评论

Human-Centered Financial Signal Analysis Based on Visual Patterns in Stock Charts

引用

IEEE Transactions on Multimedia 2025年

作者： Luo, Ji-Feng Chen, Yuzhen Zhang, Kaixun An, Xudong Hu, Menghan Zhai, Guangtao Zhang, Xiao-Ping East China Normal University Shanghai Key Laboratory of Multidimensional Information Processing School of Communication and Electronic Engineering Shanghai200241 China University of Chicago Committee on Computational and Applied Mathematics ChicagoIL60637 United States Beijing University of Posts and Telecommunications Beijing100876 China Shanghai Jiao Tong University Institute of Image Communication and Network Engineering Shanghai200240 China Tsinghua Shenzhen International Graduate School Guangdong Shenzhen518055 China

The study utilized a human-centered perspective to research the financial markets, focusing on identifying variations in eye movement patterns between professional and non-professional traders as they analyze a series of stock charts. We use eye movement data as the analysis target because we hypothesize that it represents a behavioral phenotype indicative of stock analysts' cognitive processes during market analysis. Disparities were identified by conducting variance analysis and Wilcoxon signed-rank test on statistical metrics derived from eye fixations and saccades. We provided psychological and economic interpretations to understand the underlying reasons for these observed patterns. To showcase the practical application potential of the human-centered perspective, eye movement data and human visual characteristics were used to construct visual saliency prediction models of professional stock analysts. Leveraging this human-centered model, we developed two practical application demonstrations specifically designed to support and instruct novice traders. Based on the above demonstrations, we designed a training program that demonstrates how, with ongoing training, the non-professional traders' ability to observe stock charts improves progressively. The experimental data and the analyzing code are publicly available at doi: 10.5281/zenodo.10362154. © 1999-2012 IEEE.

关键词： Financial markets

来源：评论

学校读者我要写书评

暂无评论

TDiffSal: Text-Guided Diffusion Saliency Prediction Model for images 27th

TDiffSal: Text-Guided Diffusion Saliency Prediction Model fo...

引用

27th International Conference on Pattern Recognition, ICPR 2024

作者： Zhang, Nana Xiong, Min Zhu, Dandan Zhu, Kun Zhai, Guangtao School of Computer Science and Technology Donghua University No. 2999 Renmin North Road Songjiang District Shanghai201620 China State Key Laboratory for Novel Software Technology Nanjing University Nanjing210093 China Institute of AI Education Shanghai East China Normal University No. 3663 Zhongshan North Road Putuo District Shanghai200333 China Key Laboratory of Embedded System and Service Computing Ministry of Education Tongji University No. 4800 Cao’an Highway Jiading District Shanghai201804 China Institute of Image Communication and Network Engineering Shanghai Jiao Tong University No. 800 Dongchuan Road Minhang District Shanghai200240 China

ISBN: (纸本)9783031781858

Existing visual saliency prediction methods mainly focus on single-modal visual saliency prediction, while ignoring the significant impact of text on visual saliency. To more comprehensively explore the influence of text on human attention in images, we propose a text-guided diffusion saliency prediction model, named TDiffSal. In specific, recent studies on stable diffusion models have shown promising performance in unifying tasks due to their inherent generalization ability. Inspired by this, a novel diffusion model for generalized visual-text saliency prediction is proposed, which formulates the prediction issue as a conditional generative task of the saliency map by employing input visual and text as the conditions. Meanwhile, we introduce a multi-head fusion module to effectively integrate text features and image features, which can efficiently guide the image denoising process and progressively refine the generated saliency map to make it semantically relevant to the text. Additionally, we employ an efficient pre-training strategy to enhance the robustness and generalization of the proposed model. We conduct extensive experiments on benchmark datasets to demonstrate its superior performance compared to other state-of-the-art methods. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

关键词： Prediction models

来源：评论

学校读者我要写书评

暂无评论

For techies of all ages

引用

IEEE Spectrum 2018年第12期55卷 17-20页

作者： Cass, Stephen Institute of Image Communication and Network Engineering Shanghai Jiao Tong University

Pong Coffee Table Retro tech is all the rage - witness the revived popularity of vinyl records and typewriters. So why not win the game with something that's even more retro than the original? Enter this version of Atari's 1972 Pong, the world's first commercially successful video arcade game. At first glance, it looks just like regular Pong in a coffee-table cabinet - but look closer: The ball and the paddle are mechanical, driven by magnets beneath the surface. Made by Calinfer under license from Atari, it will set you back US 3,000. Avoid missing ball for high score. © 1964-2012 IEEE.

关键词： Typewriters

来源：评论

学校读者我要写书评

暂无评论

Efficient Scheduling with Random network Coding in Multi-Relay Wireless network

引用

Journal of Shanghai Jiaotong university(Science) 2014年第1期19卷 59-64页

作者：丁良辉毕琰孙大佳杨峰钱良 Institute of Image Communication and Network Engineering Shanghai Jiaotong University Institute of Wireless Communication Technology Shanghai Jiaotong University

In this paper, we propose two novel efficient scheduling schemes with network coding in multi-relay wireless network to maximize the transmission efficiency. The first one uses adaptive forwarding with network coding(AF-NC), in which each relay adaptively calculates the number of packets having innovative information according to the feedback from the sink. With AF-NC, duplicate packets are not sent, and the total number of time slots needed to complete transmission can be significantly reduced. The second scheme, named adaptive forwarding with network coding and retransmission(AFR-NC), combines AF-NC with automatic repeat request(ARQ) to guarantee reliable end-to-end communication with limited resource occupation. Numerical results show that compared with simple forwarding with network coding(F-NC), AF-NC has close successful delivery rate with dramatically less time slots, while AFR-NC achieves strict reliability with limited resource cost.

关键词： random network coding scheduling automatic repeat request (ARQ) multi-relay wireless networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：