检索结果-内蒙古大学图书馆

Deep Audio-visual Learning:A Survey

学校读者我要写书评

暂无评论

International Journal of Automation and computing 2021年第3期18卷 351-376页

作者： Hao Zhu Man-Di Luo Rui Wang Ai-Hua Zheng Ran He Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Computer Science and TechnologyAnhui UniversityHefei 230601China Center for Research on Intelligent Perception and Computing(CRIPAC)and National Laboratory of Pattern Recognition(NLPR) Institute of AutomationChinese Academy of SciencesBeijing 100190China School of Artificial Intelligence University of the Chinese Academy of SciencesBeijing 100049China Center for Excellence in Brain Science and Intelligence Technology Chinese Academy of SciencesShanghai 200031China

Audio-visual learning,aimed at exploiting the relationship between audio and visual modalities,has drawn considerable attention since deep learning started to be used *** tend to leverage these two modalities to improve the performance of previously considered single-modality tasks or address new challenging *** this paper,we provide a comprehensive survey of recent audio-visual learning *** divide the current audio-visual learning tasks into four different subfields:audiovisual separation and localization,audio-visual correspondence learning,audio-visual generation,and audio-visual representation ***-of-the-art methods,as well as the remaining challenges of each subfield,are further ***,we summarize the commonly used datasets and challenges.

关键词： Deep audio-visual learning audio-visual separation and localization correspondence learning generative models representation learning

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Xin Yuan, Kaishen Niu, Xuesong Shi, Jingang Yu, Zitong Yue, Huanjing Yang, Jingyu The School of Electrical and Information Engineering Tianjin University Tianjin300072 China Computer Vision and Pattern Recognition Laboratory School of Engineering Science Lappeenranta-Lahti University of Technology LUT Lappeenranta53850 Finland Beijing Institute for General Artificial Intelligence Beijing100080 China School of Software Engineering Xi’an Jiaotong University Xi’an710049 China Great Bay University Dongguan523000 China

Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics as it helps to identify emotions expressed through facial expressions. Anatomically, there are innumerable correlations between AUs, which contain rich information and are vital for AU detection. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks, but it is challenging to comprehensively reflect complex correlations between AUs via hand-crafted settings. There are alternative methods that employ a fully connected graph to learn these dependencies exhaustively. However, these approaches can result in a computational explosion and high dependency with a large dataset. To address these challenges, this paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less computation for AU detection. This method adaptively learns and updates AU correlation graphs by efficiently leveraging the characteristics of different levels of AU motion and emotion representation information extracted in different stages of the network. Moreover, this paper explores the role of multi-scale learning in correlation information extraction, and design a simple yet effective multi-scale feature learning (MSFL) method to promote better performance in AU detection. By integrating AU correlation information with multi-scale features, the proposed method obtains a more robust feature representation for the final AU detection. Extensive experiments show that the proposed method outperforms the state-of-the-art methods on widely used AU detection benchmark datasets, with only 28.7% and 12.0% of the parameters and FLOPs of the best method, respectively. The code for this method is available at https://***/linuxsino/Self-adjusting-AU. Copyright © 2023, The Authors. All rights reserved.

关键词： Large dataset

Multi-Unit Floor Plan recognition and Reconstruction Using Improved Semantic Segmentation of Raster-Wise Floor Plans

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kratochvila, Lukas de Jong, Gijs Arkesteijn, Monique Bilík, Šimon Zemčík, Tomáš Horak, Karel Rellermeyer, Jan S. Department of Control and Instrumentation Faculty of Electrical Engineering and Communication Brno University of Technology Brno Czech Republic Department of Software Technology Faculty of Electrical Engineering Mathematics and Computer Science TU Delft Delft Netherlands Department of Management in the Built Environment Faculty of Architecture and the Built Environment TU Delft Delft Netherlands Computer Vision and Pattern Recognition Laboratory Department of Computational Engineering Lappeenranta-Lahti University of Technology LUT Lappeenranta Finland Dependable and Scalable Software Systems Institute of Systems Engineering Faculty of Electrical Engineering and Computer Science Leibniz University Hannover Hannover Germany

Digital twins have a major potential to form a significant part of urban management in emergency planning, as they allow more efficient designing of the escape routes, better orientation in exceptional situations, and faster rescue intervention. Nevertheless, creating the twins still remains a largely manual effort, due to a lack of 3D-representations, which are available only in limited amounts for some new buildings. Thus, in this paper we aim to synthesize 3D information from commonly available 2D architectural floor plans. We propose two novel pixel-wise segmentation methods based on the MDA-Unet and MACU-Net architectures with improved skip connections, an attention mechanism, and a training objective together with a reconstruction part of the pipeline, which vectorizes the segmented plans to create a 3D model. The proposed methods are compared with two other state-of-the-art techniques and several benchmark datasets. On the commonly used CubiCasa benchmark dataset, our methods have achieved the mean F1 score of 0.86 over five examined classes, outperforming the other pixel-wise approaches tested. We have also made our code publicly available to support research in the field. © 2024, CC BY.

关键词： Semantic Segmentation

Multi-granulation variable precision rough set based on limited tolerance relation

学校读者我要写书评

暂无评论

UPB Scientific Bulletin, Series A: Applied Mathematics and Physics 2021年第3期83卷 63-74页

作者： Wan, Renxia Yao, Yonghong Kumar, Hussain School of Mathematics and Information Science North Minzu University Ningxia750021 China HongYang Institute for Big Data in Health Fuzhou Fujian350028 China School of Mathematical Sciences Tiangong University Tianjin300387 China The Key Laboratory of Intelligent Information and Data Processing of NingXia Province North Minzu University and Health Big Data Research Institute of North Minzu University Yinchuan750021 China Machine vision and pattern recognition lab University of Regina ReginaSKS4S0A2 Canada

In this paper, the combination of the variable precision rough set and the limited tolerance relation under multi-granularity is explored. As an extension of rough set model, Multi-granularity variable precision limited tolerance rough set model is con-structed. Properties of upper and lower approximation operator and detailed structure of object class through threshold are discussed. Theoretical proofs are given to show the rationality and superiority of the proposed model. © 2021, Politechnica University of Bucharest. All rights reserved.

关键词： Granulation

Retinal blood flow speed quantification at the capillary level using temporal autocorrelation fitting OCTA

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hwang, Yunchan Won, Jungeun Yaghy, Antonio Takahashi, Hiroyuki Girgis, Jessica M. Lam, Kenneth Chen, Siyu Moult, Eric M. Ploner, Stefan B. Maier, Andreas Waheed, Nadia K. Fujimoto, James G. Department of Electrical Engineering and Computer Science Research Laboratory of Electronics Massachusetts Institute of Technology CambridgeMA02139 United States New England Eye Center Tufts Medical Center BostonMA02116 United States Pattern Recognition Lab Friedrich-Alexander-Universität Erlangen-Nürnberg Erlangen Germany

Optical coherence tomography angiography (OCTA) can visualize vasculature structures, but provides limited information about the blood flow speeds. Here, we present a second generation variable interscan time analysis (VISTA) OCTA, which evaluates a quantitative surrogate marker for blood flow speed in vasculature. At the capillary level, spatially compiled OCTA and a simple temporal autocorrelation model, ρ(τ) = exp(-ατ), were used to evaluate a temporal autocorrelation decay constant, α, as the blood flow speed marker. A 600 kHz A-scan rate swept-source provides short interscan time OCTA and fine A-scan spacing acquisition, while maintaining multi mm2field of views for human retinal imaging. We demonstrate the cardiac pulsatility and repeatability of α measured with VISTA. We show different α for different retinal capillary plexuses in healthy eyes and present representative VISTA OCTA of eyes with diabetic retinopathy. Copyright © 2023, The Authors. All rights reserved.

关键词： Optical tomography

Learning dynamical human-joint affinity for 3D pose estimation in videos

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Junhao Wang, Yali Zhou, Zhipeng Luan, Tianyu Wang, Zhe Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences University of California Irvine United States Shanghai AI Laboratory Shanghai China

Graph Convolution Network (GCN) has been successfully used for 3D human pose estimation in videos. However, it is often built on the fixed human-joint affinity, according to human skeleton. This may reduce adaptation capacity of GCN to tackle complex spatio-temporal pose variations in videos. To alleviate this problem, we propose a novel Dynamical Graph Network (DG-Net), which can dynamically identify human-joint affinity, and estimate 3D pose by adaptively learning spatial/temporal joint relations from videos. Different from traditional graph convolution, we introduce Dynamical Spatial/Temporal Graph convolution (DSG/DTG) to discover spatial/temporal human-joint affinity for each video exemplar, depending on spatial distance/temporal movement similarity between human joints in this video. Hence, they can effectively understand which joints are spatially closer and/or have consistent motion, for reducing depth ambiguity and/or motion uncertainty when lifting 2D pose to 3D pose. We conduct extensive experiments on three popular benchmarks, e.g., Human3.6M, HumanEva-I, and MPI-INF-3DHP, where DG-Net outperforms a number of recent SOTA approaches with fewer input frames and model size. Copyright © 2021, The Authors. All rights reserved.

关键词： Convolution

Attention-Driven Dynamic Graph Convolutional Network for Multi-label Image recognition 16th

学校读者我要写书评

暂无评论

Attention-Driven Dynamic Graph Convolutional Network for Mul...

16th European Conference on computer vision, ECCV 2020

作者： Ye, Jin He, Junjun Peng, Xiaojiang Wu, Wenhao Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen China School of Biomedical Engineering the Institute of Medical Robotics Shanghai Jiao Tong University Shanghai China

ISBN: (纸本)9783030585884

Recent studies often exploit Graph Convolutional Network (GCN) to model label dependencies to improve recognition accuracy for multi-label image recognition. However, constructing a graph by counting the label co-occurrence possibilities of the training data may degrade model generalizability, especially when there exist occasional co-occurrence objects in test images. Our goal is to eliminate such bias and enhance the robustness of the learnt features. To this end, we propose an Attention-Driven Dynamic Graph Convolutional Network (ADD-GCN) to dynamically generate a specific graph for each image. ADD-GCN adopts a Dynamic Graph Convolutional Network (D-GCN) to model the relation of content-aware category representations that are generated by a Semantic Attention Module (SAM). Extensive experiments on public multi-label benchmarks demonstrate the effectiveness of our method, which achieves mAPs of 85.2%, 96.0%, and 95.5% on MS-COCO, VOC2007, and VOC2012, respectively, and outperforms current state-of-the-art methods with a clear margin. © 2020, Springer Nature Switzerland AG.

关键词： Semantics

PProCRC: Probabilistic Collaboration of Image Patches for Fine-grained Classification

学校读者我要写书评

暂无评论

PProCRC: Probabilistic Collaboration of Image Patches for Fi...

International Conference on Image and vision Computing New Zealand, IVCNZ

作者： Tapabrata Chakraborti Brendan McCane Steven Mills Umapada Pal University of Otago tapabrata mccane Computer Vision and Pattern recognition Unit Indian Statistical Institute

ISBN: (数字)9781728185798

ISBN: (纸本)9781728185804

We present a conditional probabilistic framework for collaborative representation of image patches. It incorporates background compensation and outlier patch suppression into the main formulation itself, thus doing away with the need for pre-processing steps to handle the same. A closed form non-iterative solution of the cost function is derived. The proposed method (PProCRC) outperforms earlier CRC formulations: patch based (PCRC, GP-CRC) as well as the state-of-the-art probabilistic (ProCRC and EProCRC) on three fine-grained species recognition datasets (Oxford Flowers, Oxford-IIIT Pets and CUB Birds) using two CNN backbones (Vgg-19 and ResNet-50).

关键词： Visualization Image recognition Collaboration Probabilistic logic Cost function Birds Task analysis

Digging into Uncertainty in Self-supervised Multi-view Stereo

学校读者我要写书评

暂无评论

Digging into Uncertainty in Self-supervised Multi-view Stere...

International Conference on computer vision (ICCV)

作者： Hongbin Xu Zhipeng Zhou Yali Wang Wenxiong Kang Baigui Sun Hao Li Yu Qiao ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences South China University of Technology Alibaba Group Pazhou Laboratory Shanghai AI Laboratory

ISBN: (纸本)9781665428132

Self-supervised Multi-view stereo (MVS) with a pretext task of image reconstruction has achieved significant progress recently. However, previous methods are built upon intuitions, lacking comprehensive explanations about the effectiveness of the pretext task in self-supervised MVS. To this end, we propose to estimate epistemic uncertainty in self-supervised MVS, accounting for what the model ignores. Specially, the limitations can be categorized into two types: ambiguious supervision in foreground and invalid supervision in background. To address these issues, we propose a novel Uncertainty reduction Multi-view Stereo (U-MVS) framework for self-supervised learning. To alleviate ambiguous supervision in foreground, we involve extra correspondence prior with a flow-depth consistency loss. The dense 2D correspondence of optical flows is used to regularize the 3D stereo correspondence in MVS. To handle the invalid supervision in background, we use Monte-Carlo Dropout to acquire the uncertainty map and further filter the unreliable supervision signals on invalid regions. Extensive experiments on DTU and Tank&Temples benchmark show that our U-MVS framework 1 achieves the best performance among unsupervised MVS methods, with competitive performance with its supervised opponents.

关键词： Optical losses Optical filters computer vision Uncertainty Three-dimensional displays Monte Carlo methods Benchmark testing