检索结果-内蒙古大学图书馆

2D+3D facial expression recognition via embedded tensor manifold regularization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Yunfang Ruan, Qiuqi Luo, Ziyan An, Gaoyun Jin, Yi Wan, Jun School of Computer Science and Engineering Shijiazhuang University Shijiazhuang050035 China Institute of Information Science Beijing Jiaotong University Beijing100044 China Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing100044 China Department of Mathematics Beijing Jiaotong University Beijing100044 China National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing100190 China

In this paper, a novel approach via embedded tensor manifold regularization for 2D+3D facial expression recognition (FERETMR) is proposed. Firstly, 3D tensors are constructed from 2D face images and 3D face shape models to keep the structural information and correlations. To maintain the local structure (geometric information) of 3D tensor samples in the low-dimensional tensors space during the dimensionality reduction, the l0-norm of the core tensors and a tensor manifold regularization scheme embedded on core tensors are adopted via a low-rank truncated Tucker decomposition on the generated tensors. As a result, the obtained factor matrices will be used for facial expression classification prediction. To make the resulting tensor optimization more tractable, l1-norm surrogate is employed to relax l0-norm and hence the resulting tensor optimization problem has a nonsmooth objective function due to the l1-norm and orthogonal constraints from the orthogonal Tucker decomposition. To efficiently tackle this tensor optimization problem, we establish the first-order optimality condition in terms of stationary points, and then design a block coordinate descent (BCD) algorithm with convergence analysis and the computational complexity. Numerical results on BU-3DFE database and Bosphorus databases demonstrate the effectiveness of our proposed approach. Copyright © 2022, The Authors. All rights reserved.

关键词： Tensors

Zero-Shot Audio Captioning Using Soft and Hard Prompts

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yiming Xu, Xuenan Du, Ruoyi Liu, Haohe Dong, Yuan Tan, Zheng-Hua Wang, Wenwu Ma, Zhanyu The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China The Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai200240 China The Department of Electronic Systems Aalborg University Aalborg9220 Denmark The Centre for Vision Speech and Signal Processing University of Surrey GuildfordGU2 7XH United Kingdom

In traditional audio captioning methods, a model is usually trained in a fully supervised manner using a human-annotated dataset containing audio-text pairs and then evaluated on the test sets from the same dataset. Such methods have two limitations. First, these methods are often data-hungry and require time-consuming and expensive human annotations to obtain audio-text pairs. Second, these models often suffer from performance degradation in cross-domain scenarios, i.e., when the input audio comes from a different domain than the training set, which, however, has received little attention. We propose an effective audio captioning method based on the contrastive language-audio pre-training (CLAP) model to address these issues. Our proposed method requires only textual data for training, enabling the model to generate text from the textual feature in the cross-modal semantic space. In the inference stage, the model generates the descriptive text for the given audio from the audio feature by leveraging the audio-text alignment from CLAP. We devise two strategies to mitigate the discrepancy between text and audio embeddings: a mixed-augmentation-based soft prompt and a retrieval-based acoustic-aware hard prompt. These approaches are designed to enhance the generalization performance of our proposed model, facilitating the model to generate captions more robustly and accurately. Extensive experiments on AudioCaps and Clotho benchmarks show the effectiveness of our proposed method, which outperforms other zero-shot audio captioning approaches for in-domain scenarios and outperforms the compared methods for cross-domain scenarios, underscoring the generalization ability of our method. © 2024, CC BY.

关键词： Semantics

Study of software fault tolerance system evaluation strategy

学校读者我要写书评

暂无评论

Wuhan Daxue Xuebao (Xinxi Kexue Ban)/ Geomatics and Information Science of Wuhan University 2008年第10期33卷 1018-1021页

作者： Liu, Xin Yin, Qian Guo, Ping Lab. of Pattern Recognition and Intelligent System Beijing Normal University 19 Xinjiekou Beijing 100875 China State Key Laboratory of Computer Science Institute of Software CAS 4 Zhongguancun Beijing 100080 China

Software fault tolerance technique is used to ensure high reliability. In order to evaluate the quality of fault-tolerant systems, expected execution time and cost must be considered. A new evaluation strategy is proposed to evaluate, evaluating the system with three groups of quality factors, which is the capability, the time and the cost. By using them, the system's integrative quality factor can be calculated. When applying this strategy, we compare the quality of software systems which is consist of different fault-tolerant components. The results can be used to improve fault-tolerant system. To achieve the required optimal result, several optimization schemes are given to meet different project demands.

关键词： Quality control

Entertainment for Education. Digital Techniques and Systems 1

学校读者我要写书评

暂无评论

丛书名： Lecture Notes in computer Science

1000年

作者： Xiaopeng Zhang Shaochun Zhong Zhigeng Pan Kevin Wong Ruwei Yun

ISBN: (数字)9783642145339

ISBN: (纸本)9783642145322

With the technical advancement of digital media and the medium of communication in recent years, there is a widespread interest in digital entertainment. An emerging te- nical research area edutainment, or educational entertainment, has been accepted as education using digital entertainment. Edutainment has been recognized as an eff- tive way of learning using modern digital media tools, like computers, games, mobile phones, televisions, or other virtual reality applications, which emphasizes the use of entertainment with application to the education domain. The Edutainment conference series was established in 2006 and subsequently - ganized as a special event for researchers working in this new interest area of e-learning and digital entertainment. The main purpose of Edutainment conferences is to facilitate the discussion, presentation, and information exchange of the scientific and technological development in the new community. The Edutainment conference series becomes a valuable opportunity for researchers, engineers, and graduate s- dents to communicate at these international annual events. The conference series - cludes plenary invited talks, workshops, tutorials, paper presentation tracks, and panel discussions. The Edutainment conference series was initiated in Hangzhou, China in 2006. Following the success of the first event, the second (Edutainment 2007 in Hong Kong, China), third (Edutainment 2008 in Nanjing, China), and fourth editions (Edutainment 2009 in Banff, Canada) were organized. Edutainment 2010 was held during August 16–18, 2010 in Changchun, China. Two workshops were jointly org- ized together with Edutainment 2010.

关键词： computers and Education

A win-win deal: towards sparse and robust pre-trained language models 22

学校读者我要写书评

暂无评论

A win-win deal: towards sparse and robust pre-trained langua...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： Yuanxin Liu Fandong Meng Zheng Lin Jiangnan Li Peng Fu Yanan Cao Weiping Wang Jie Zhou Institute of Information Engineering Chinese Academy of Sciences and MOE Key Laboratory of Computational Linguistics Peking University and School of Computer Science Peking University Pattern Recognition Center WeChat AI Tencent Inc China Institute of Information Engineering Chinese Academy of Sciences and School of Cyber Security University of Chinese Academy of Sciences Institute of Information Engineering Chinese Academy of Sciences

ISBN: (纸本)9781713871088

Despite the remarkable success of pre-trained language models (PLMs), they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and computation. Second, on the downstream tasks, PLMs tend to rely on the dataset bias and struggle to generalize to out-of-distribution (OOD) data. In response to the efficiency problem, recent studies show that dense PLMs can be replaced with sparse subnetworks without hurting the performance. Such subnetworks can be found in three scenarios: 1) the fine-tuned PLMs, 2) the raw PLMs and then fine-tuned in isolation, and even inside 3) PLMs without any parameter fine-tuning. However, these results are only obtained in the indistribution (ID) setting. In this paper, we extend the study on PLMs subnetworks to the OOD setting, investigating whether sparsity and robustness to dataset bias can be achieved simultaneously. To this end, we conduct extensive experiments with the pre-trained BERT model on three natural language understanding (NLU) tasks. Our results demonstrate that sparse and robust subnetworks (SRNets) can consistently be found in BERT, across the aforementioned three scenarios, using different training and compression methods. Furthermore, we explore the upper bound of SRNets using the OOD information and show that there exist sparse and almost unbiased BERT subnetworks. Finally, we present 1) an analytical study that provides insights on how to promote the efficiency of SRNets searching process and 2) a solution to improve subnetworks' performance at high sparsity.

关键词：

A comprehensive survey of oracle character recognition: challenges, benchmarks, and beyond

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Jing Chi, Xueke Wang, Qiufeng Huang, Kaizhu Wang, Da-Han Liu, Yongge Liu, Cheng-Lin College of Science & Technology Ningbo University Ningbo China School of Advanced Technology Xi’an Jiaotong-Liverpool University Suzhou China Duke Kunshan University Suzhou China Fujian Key Laboratory of Pattern Recognition and Image Understanding School of Computer and Information Engineering Xiamen University of Technology Xiamen China Anyang Normal University Anyang China State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences Beijing China

Oracle character recognition—an analysis of ancient Chinese inscriptions found on oracle bones—has become a pivotal field intersecting archaeology, paleography, and historical cultural studies. Traditional methods of oracle character recognition have relied heavily on manual interpretation by experts, which is not only labor-intensive but also limits broader accessibility to the general public. With recent breakthroughs in pattern recognition and deep learning, there is a growing movement towards the automation of oracle character recognition (OrCR), showing considerable promise in tackling the challenges inherent to these ancient scripts. However, a comprehensive understanding of OrCR still remains elusive. Therefore, this paper presents a systematic and structured survey of the current landscape of OrCR research. We commence by identifying and analyzing the key challenges of OrCR. Then, we provide an overview of the primary benchmark datasets and digital resources available for OrCR. A review of contemporary research methodologies follows, in which their respective efficacies, limitations, and applicability to the complex nature of oracle characters are critically highlighted and examined. Additionally, our review extends to ancillary tasks associated with OrCR across diverse disciplines, providing a broad-spectrum analysis of its applications. We conclude with a forward-looking perspective, proposing potential avenues for future investigations that could yield significant advancements in the field. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep learning

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Wenjie Wang, Mei Zhang, Kai Li, Juncheng Li, Xiaoming Zhang, Yuhang Gao, Guangwei Deng, Weihong Lin, Chia-Wen The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing China The Computer Vision Lab ETH Zürich Zürich Switzerland The School of Communication and Information Engineering Shanghai University Shanghai China The Nanyang Technological University Singapore The Intelligent Visual Information Perception Laboratory Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Department of Electrical Engineering National Tsing Hua University Hsinchu Taiwan

Face restoration (FR) is a specialized field within image restoration that aims to recover low-quality (LQ) face images into high-quality (HQ) face images. Recent advances in deep learning technology have led to significant progress in FR methods. In this paper, we begin by examining the prevalent factors responsible for real-world LQ images and introduce degradation techniques used to synthesize LQ images. We also discuss notable benchmarks commonly utilized in the field. Next, we categorize FR methods based on different tasks and explain their evolution over time. Furthermore, we explore the various facial priors commonly utilized in the restoration process and discuss strategies to enhance their effectiveness. In the experimental section, we thoroughly evaluate the performance of state-of-the-art FR methods across various tasks using a unified benchmark. We analyze their performance from different perspectives. Finally, we discuss the challenges faced in the field of FR and propose potential directions for future advancements. The open-source repository corresponding to this work can be found at https://***/24wenjie-li/Awesome-Face-Restoration. Copyright © 2023, The Authors. All rights reserved.

关键词： Restoration

Image quality assessment for perceptual image restoration: A new dataset, benchmark and metric

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Gu, Jinjin Cai, Haoming Chen, Haoyu Ye, Xiaoxing Ren, Jimmy S. Dong, Chao School of Electrical and Information Engineering University of Sydney Australia Chinese University of Hong Kong Shenzhen Hong Kong ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SenseTime Research Qing Yuan Research Institute Shanghai Jiao Tong University Shanghai China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Image quality assessment (IQA) is the key factor for the fast development of image restoration (IR) algorithms. The most recent perceptual IR algorithms based on generative adversarial networks (GANs) have brought in significant improvement on visual performance, but also pose great challenges for quantitative evaluation. Notably, we observe an increasing inconsistency between perceptual quality and the evaluation results. We present two questions: (1) Can existing IQA methods objectively evaluate recent IR algorithms? (2) With the focus on beating current benchmarks, are we getting better IR algorithms? To answer the questions and promote the development of IQA methods, we contribute a large-scale IQA dataset, called Perceptual Image Processing ALgorithms (PIPAL) dataset. Especially, this dataset includes the results of GAN-based IR algorithms, which are missing in previous datasets. We collect more than 1.13 million human judgements to assign subjective scores for PIPAL images using the more reliable "Elo system". Based on PIPAL, we present new benchmarks for both IQA and super-resolution methods. Our results indicate that existing IQA methods cannot fairly evaluate GAN-based IR algorithms. While using appropriate evaluation methods is important, IQA methods should also be updated along with the development of IR algorithms. At last, we shed light on how to improve the IQA performance on GAN-based distortion. Inspired by the find that the existing IQA methods have an unsatisfactory performance on the GAN-based distortion partially because of their low tolerance to spatial misalignment, we propose to improve the performance of an IQA network on GAN-based distortion by explicitly considering this misalignment. We propose the Space Warping Difference Network, which includes the novel l2 pooling layers and Space Warping Difference layers. Experiments demonstrate the effectiveness of the proposed method. © 2020, CC BY.

关键词： Image reconstruction

Learning data-adaptive non-parametric kernels

学校读者我要写书评

暂无评论

The Journal of Machine Learning Research

The Journal of Machine Learning Research 2020年第1期21卷 8590-8628页

作者： Fanghui Liu Xiaolin Huang Chen Gong Jie Yang Li Li Department of Electrical Engineering ESAT-STADIUS KU Leuven Belgium Institute of Image Processing and Pattern Recognition Institute of Medical Robotics Shanghai Jiao Tong University Shanghai China PCA Lab Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education School of Computer Science and Engineering Nanjing University of Science and Technology China and Department of Computing Hong Kong Polytechnic University Hong Kong SAR China Department of Automation BNRist Tsinghua University China

In this paper, we propose a data-adaptive non-parametric kernel learning framework in margin based kernel methods. In model formulation, given an initial kernel matrix, a data-adaptive matrix with two constraints is imposed in an entry-wise scheme. Learning this data-adaptive matrix in a formulation-free strategy enlarges the margin between classes and thus improves the model flexibility. The introduced two constraints are imposed either exactly (on small data sets) or approximately (on large data sets) in our model, which provides a controllable trade-off between model flexibility and complexity with theoretical demonstration. In algorithm optimization, the objective function of our learning framework is proven to be gradient-Lipschitz continuous. Thereby, kernel and classifier/regressor learning can be efficiently optimized in a unified framework via Nesterov's acceleration. For the scalability issue, we study a decomposition-based approach to our model in the large sample case. The effectiveness of this approximation is illustrated by both empirical studies and theoretical guarantees. Experimental results on various classification and regression benchmark data sets demonstrate that our non-parametric kernel learning framework achieves good performance when compared with other representative kernel learning based algorithms.

关键词： support vector machines non-parametric kernel learning gradient-Lipschitz continuous