检索结果-内蒙古大学图书馆

Slnspeech: Solving Extended Speech Separation Problem with the Help of Sign Language

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wu, Jiasong Li, Xuan Li, Taotao Meng, Fanman Kong, Youyong Yang, Guanyu Senhadji, Lotfi Shu, Huazhong Laboratory of Image Science and Technology Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education Nanjing210096 China Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing Southeast University Nanjing210096 China Univ Rennes INSERM LTSI - UMR 1099 RennesF-35000 France Univ Rennes INSERM RennesF-35000 France

Previous audio-visual speech separation methods use the synchronization of the speaker's facial movement and speech in the video to supervise the speech separation in a self-supervised way. In this paper, we propose a model to solve the speech separation problem assisted by both face and sign language, which we call the extended speech separation problem. We design a general deep learning network for learning the combination of three modalities, audio, face, and sign language information, for better solving the speech separation problem. To train the model, we introduce a large-scale dataset named the Sign Language News Speech (SLNSpeech) dataset in which three modalities of audio, face, and sign language coexist. Experiment results show that the proposed model has better performance and robustness than the usual audio-visual system. Besides, sign language modality can also be used alone to supervise speech separation task, and the introduction of sign language is helpful for hearing-impaired people to learn and communicate. Last, our model is a general speech separation framework and can achieve very competitive separation performance on two open-source audio-visual datasets. The code is available at https://***/iveveive/SLNSpeech. © 2023, The Authors. All rights reserved.

关键词： Audition

Matching Biomedical Ontologies with Long Short-Term Memory Networks

学校读者我要写书评

暂无评论

Matching Biomedical Ontologies with Long Short-Term Memory N...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Chao Jiang Xingsi Xue Intelligent Information Processing Research Center Fujian University of Technology Fuzhou Fujian China School of Computer Science and Mathematics Fujian University of Technology Fuzhou Fujian China Fujian Provincial Key Laboratory of Big Data Mining and Applications Fujian University of Technology Fuzhou Fujian China Guangxi Key Laboratory of Automatic Detecting Technology and Instruments Guilin University of Electronic Technology Guilin Guangxi China Fujian Key Lab for Automotive Electronics and Electric Drive Fujian University of Technology Fuzhou Fujian China

ISBN: (数字)9781728162157

ISBN: (纸本)9781728162164

Semantic Web (SW) has attracted the increasing attention of researchers, which facilitates people to link and handle various data. Ontology is the kernel technique of SW, and biomedical ontology is a state-of-art biomedical knowledge modeling technique, which formally defines the biomedical concepts and their relationships. However, the same biomedical concepts in different biomedical ontologies could be defined in various contexts or with different terms, which yields the biomedical ontology heterogeneity problem. It is crucial to find mapping among heterogeneity concepts of different biomedical ontologies for bridging the semantic gaps, which is the so-called biomedical ontology matching. Biomedical ontology matching problem is an open challenge due to the rich semantic meaning and the flexible representation on a biomedical concept. To address this challenging problem, in this work, it is regarded as a binary classification problem, and a Long Short-Term Memory Networks (LSTM)-based ontology matching technique is proposed to solve it. Our proposal improves the quality of the alignment by introducing the char-embedding technique, which takes into account the semantic and context information of concepts. The comparing results with OAEI's participants show the effectiveness of our proposal.

关键词： Ontologies Semantics Logic gates Training Human anatomy Proposals Biomedical measurement

DRL-M4MR: An Intelligent Multicast Routing Approach Based on DQN Deep Reinforcement Learning in SDN

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Chenwei Ye, Miao Xue, Xingsi Lv, Jianhui Jiang, Qiuxiang Wang, Yong School of Information and Communication Guilin University of Electronic Technology Guilin541004 China Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing Guilin University of Electronic Technology Guilin541004 China Fujian Provincial Key Laboratory of Big Data Mining and Applications Fujian University of Technology Fujian Fuzhou350118 China Peng Cheng Lab. Guangdong Shenzhen518038 China School of Computer Science and Information Security Guilin University of Electronic Technology Guilin541004 China

Traditional multicast routing methods have some problems in constructing a multicast tree, such as limited access to network state information, poor adaptability to dynamic and complex changes in the network, and inflexible data forwarding. To address these defects, the optimal multicast routing problem in software-defined networking (SDN) is tailored as a multi-objective optimization problem, and an intelligent multicast routing algorithm DRL-M4MR based on the deep Q network (DQN) deep reinforcement learning (DRL) method is designed to construct a multicast tree in SDN. First, the multicast tree state matrix, link bandwidth matrix, link delay matrix, and link packet loss rate matrix are designed as the state space of the DRL agent by combining the global view and control of the SDN. Second, the action space of the agent is all the links in the network, and the action selection strategy is designed to add the links to the current multicast tree under four cases. Third, single-step and final reward function forms are designed to guide the intelligence to make decisions to construct the optimal multicast tree. The experimental results show that, compared with existing algorithms, the multicast tree construct by DRL-M4MR can obtain better bandwidth, delay, and packet loss rate performance after training, and it can make more intelligent multicast routing decisions in a dynamic network environment. Copyright © 2022, The Authors. All rights reserved.

关键词： Reinforcement learning

Multi-level graph convolutional network with automatic graph learning for hyperspectral image classification

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wan, Sheng Gong, Chen Pan, Shirui Yang, Jie Yang, Jian PCA Lab Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education Jiangsu Key Laboratory of Image Video Understanding for Social Security School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing210094 China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai200240 China Faculty of Information Technology Monash University ClaytonVIC3800 Australia

Nowadays, deep learning methods, especially the Graph Convolutional Network (GCN), have shown impressive performance in hyperspectral image (HSI) classification. However, the current GCN-based methods treat graph construction and image classification as two separate tasks, which often results in suboptimal performance. Another defect of these methods is that they mainly focus on modeling the local pairwise importance between graph nodes while lack the capability to capture the global contextual information of HSI. In this paper, we propose a Multi-level GCN with Automatic Graph Learning method (MGCN-AGL) for HSI classification, which can automatically learn the graph information at both local and global levels. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions, which helps encode the spatial context to form the graph information at local level. Moreover, we utilize multiple pathways for local-level graph convolution, in order to leverage the merits from the diverse spatial context of HSI and to enhance the expressive power of the generated representations. To reconstruct the global contextual relations, our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level. Then inference can be performed along the reconstructed graph edges connecting faraway regions. Finally, the multi-level information is adaptively fused to generate the network output. In this means, the graph learning and image classification can be integrated into a unified framework and benefit each other. Extensive experiments have been conducted on three real-world hyperspectral datasets, which are shown to outperform the state-of-the-art methods. Copyright © 2020, The Authors. All rights reserved.

关键词： Image classification

Hand Hygiene Assessment via Joint Step Segmentation and key Action Scorer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Chenglong Zhu, Qiwen Liu, Tubiao Tang, Jin Su, Yu Information Materials and Intelligent Sensing Laboratory of Anhui Province Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Artificial Intelligence Anhui University Hefei230601 China Information Materials and Intelligent Sensing Laboratory of Anhui Province Key Lab of Intelligent Computing and Signal Processing of Ministry of Education Anhui Provincial Key Laboratory of Multimodal Cognitive Computation School of Computer Science and Technology Anhui University Hefei230601 China Hefei Normal University Institute of Artificial Intelligence Hefei Comprehensive National Science Center Hefei230061 China

Hand hygiene is a standard six-step hand-washing action proposed by the World Health Organization (WHO). However, there is no good way to supervise medical staff to do hand hygiene, which brings the potential risk of disease spread. Existing action assessment works usually make an overall quality prediction on an entire video. However, the internal structures of hand hygiene action are important in hand hygiene assessment. Therefore, we propose a novel fine-grained learning framework to perform step segmentation and key action scorer in a joint manner for accurate hand hygiene assessment. Existing temporal segmentation methods usually employ multi-stage convolutional network to improve the segmentation robustness, but easily lead to over-segmentation due to the lack of the long-range dependence. To address this issue, we design a multi-stage convolution-transformer network for step segmentation. Based on the observation that each hand-washing step involves several key actions which determine the hand-washing quality, we design a set of key action scorers to evaluate the quality of key actions in each step. In addition, there lacks a unified dataset in hand hygiene assessment. Therefore, under the supervision of medical staff, we contribute a video dataset that contains 300 video sequences with fine-grained annotations. Extensive experiments on the dataset suggest that our method well assesses hand hygiene videos and achieves outstanding performance. Copyright © 2022, The Authors. All rights reserved.

关键词： Convolution

Converged reconfigurable intelligent surface and mobile edge computing for space information networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Cao, Xuelin Yang, Bo Huang, Chongwen Yuen, Chau Zhang, Yan Niyato, Dusit Han, Zhu Engineering Product Development Pillar Singapore University of Technology and Design Singapore Singapore College of Information Science and Electronic Engineering Zhejiang Provincial Key Lab of information processing communication and networking International Joint Innovation Center Zhejiang University China Department of Informatics University of Oslo Norway School of Computer Science and Engineering Nanyang Technological University Singapore Singapore Department of Electrical and Computer Engineering University of Houston United States

Space information networks (SIN) are facing an ever-increasing thirst for high-speed and high-capacity seamless data transmission due to the integration of ground, air, and space communications. However, this imposes a new paradigm on the architecture design of the integrated SIN. Recently, reconfigurable intelligent surfaces (RISs) and mobile edge computing (MEC) are the most promising techniques, conceived to improve communication and computation capability by reconfiguring the wireless propagation environment and offloading. Hence, converging RISs and MEC in SIN is becoming an effort to reap the double benefits of computation and communication. In this article, we propose an RIS-assisted collaborative MEC architecture for SIN and discuss its implementation. Then we present its potential benefits, major challenges, and feasible applications. Subsequently, we study different cases to evaluate the system data rate and latency. Finally, we conclude with a list of open issues in this research area. Copyright © 2021, The Authors. All rights reserved.

关键词： Mobile edge computing

CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Gao, Guangwei Xu, Zixiang Li, Juncheng Yang, Jian Zeng, Tieyong Qi, Guo-Jun The Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou China The School of Communication & Information Engineering Shanghai University Shanghai China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing China The School of Computer Science and Technology Nanjing University of Science and Technology Nanjing China The Center for Mathematical Artificial Intelligence Department of Mathematics The Chinese University of Hong Kong Hong Kong Research Center for Industries of the Future School of Engineering Westlake University OPPO Research Seattle United States

Recently, deep convolution neural networks (CNNs) steered face super-resolution methods have achieved great progress in restoring degraded facial details by joint training with facial priors. However, these methods have some obvious limitations. On the one hand, multi-task joint learning requires additional marking on the dataset, and the introduced prior network will significantly increase the computational cost of the model. On the other hand, the limited receptive field of CNN will reduce the fidelity and naturalness of the reconstructed facial images, resulting in suboptimal reconstructed images. In this work, we propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks, which uses the multi-scale connected encoder-decoder architecture as the backbone. Specifically, we first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block, to promote the consistency of local facial detail and global facial structure restoration simultaneously. Then, we design an efficient Feature Refinement Module (FRM) to enhance the encoded features. Finally, to further improve the restoration of fine facial details, we present a Multi-scale Feature Fusion Unit (MFFU) to adaptively fuse the features from different stages in the encoder procedure. Extensive evaluations on various datasets have assessed that the proposed CTCNet can outperform other state-of-the-art methods significantly. Source code will be available at https://***/IVIPlab/CTCNet. Copyright © 2022, The Authors. All rights reserved.

关键词： Restoration

Learning robust and discriminative low-rank representations for face recognition with occlusion

学校读者我要写书评

暂无评论

Pattern Recognition 2017年 66卷 129-143页

作者： Guangwei Gao Jian Yang Xiao-Yuan Jing Fumin Shen Wankou Yang Dong Yue Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China Jiangsu Engineering Laboratory of Big Data Analysis and Control for active distribution network Nanjing University of Posts and Telecommunications Nanjing China Fujian Provincial Key Laboratory of Information Processing and Intelligent Control (Minjiang University) Fuzhou China School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing China School of Automation Nanjing University of Posts and Telecommunications Nanjing China School of Computer Science and Engineering University of Electronic Science and Technology of China Chengdu China School of Automation Southeast University Nanjing China

For robust face recognition tasks, we particularly focus on the ubiquitous scenarios where both training and testing images are corrupted due to occlusions. Previous low-rank based methods stacked each error image into a vector and then used L 1 or L 2 norm to measure the error matrix. However, in the stacking step, the structure information of the error image can be lost. Depart from the previous methods, in this paper, we propose a novel method by exploiting the low-rankness of both the data representation and each occlusion-induced error image simultaneously, by which the global structure of data together with the error images can be well captured. In order to learn more discriminative low-rank representations, we formulate our objective such that the learned representations are optimal for classification with the available supervised information and close to an ideal-code regularization term. With strong structure information preserving and discrimination capabilities, the learned robust and discriminative low-rank representation (RDLRR) works very well on face recognition problems, especially with face images corrupted by continuous occlusions. Together with a simple linear classifier, the proposed approach is shown to outperform several other state-of-the-art face recognition methods on databases with a variety of face variations.

关键词： Face recognition Low-rank matrix recovery Nuclear norm

HU-Net: Hessian-incorporated U-Net for CBCT projection-domain metal segmentation with guidance from SAM

学校读者我要写书评

暂无评论

HU-Net: Hessian-incorporated U-Net for CBCT projection-domai...

IEEE Conference on Cybernetics and Intelligent Systems

作者： Chen Jiang Tianling Lyu Gege Ma Yan Xi Yang Chen Wentao Zhu Zhejiang Lab Hangzhou China Hangzhou Institute for Advanced Study of UCAS Hangzhou China Shanghai First-Imaging Tech. Shanghai China Laboratory of Image Science and Technology Southeast University Nanjing China Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing the Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications Southeast University Nanjing China

ISBN: (数字)9798350364194

ISBN: (纸本)9798350364200

Metal artifact is a prevailing factor reducing the image quality of CBCT in minimally invasive spine surgery. In CBCT, conventional metal artifact reduction algorithms pay more attention to the inpainting of the metal traces, but metal segmentation is also challenging. Despite the current success in image segmentation with deep learning, the substantial expense associated with annotating metal traces in the projection domain makes most of these approaches impractical for this task. To address this, we propose a Hessian-incorporated U-Net (HU-Net) for CBCT projection-domain metal segmentation with guidance from SAM. The proposed method has been tested on both digital phantom data and clinical CBCT data. According to the experimental results, our method has demonstrated notable improvement in the segmentation of metal traces across both datasets. This paper presents a feasible approach for metal trace segmentation in the projection domain with deep learning and has the potential to be applied in clinical applications.

关键词： Deep learning Image quality Image segmentation Minimally invasive surgery Mechatronics Metals Phantoms