检索结果-内蒙古大学图书馆

Lip Shape Classification of Sounds for Speech Therapy using SlowFast Networks 13

学校读者我要写书评

暂无评论

Lip Shape Classification of Sounds for Speech Therapy using ...

2023 13th International Workshop on computer Science and Engineering, WCSE 2023

作者： Boonsri, Penpicha Eiamboonsert, Salita Siricharoen, Punnarai Perceptual Intelligent Computing Lab. Department of Computer Engineering Faculty of Engineering Chulalongkorn University Thailand Media Technology King Mongkut's University of Technology Thonburi Thailand

ISBN: (纸本)9789811879500

It is important for patients with Aphasia and Dysarthria to have speech and language therapy to practice breathing exercises, tongue strengthening exercises, and especially speech sounds such as short vowel sounds. To ensure the clarity of the pronunciation sound and the correct position of mouth shape, it is required to be monitored by a therapist. We proposed an automated method using convolutional networks to identify the motion of pronunciation of 9 short vowel sounds which is required for speech exercises in the Thai language. Firstly, videos of vowel sound pronunciation are captured, then preprocessed to crop only the mouth area using Dlib library. The cropped image sequence is then fed into audiovisual SlowFast Networks based on convolutional networks which have Slow and Fast visual pathways to capture spatial and temporal information of a video. We compared our selected model with the transformer-based state-of-the-art model, such as TimeSformer. Our proposed framework using SlowFast networks achieved average accuracy at 97.3% for 9-class video classification of Thai vowel sounds. It shows our proposed framework has a potential for use as a tool for speech sound self-exercises and therapy. © WCSE *** rights reserved.

关键词： Convolution

Concrete Structural Crack Damage Classification Using Nonlinear Dimension Reduction and Broad Learning System

学校读者我要写书评

暂无评论

Concrete Structural Crack Damage Classification Using Nonlin...

2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2024

作者： Wang, Bingshu Lin, Jia Zhuang, Xiaodong Zhang, Guanghui Chen, C.L. Philip School of Software Northwestern Polytechnical University Xi'an China Shenzhen University Guangdong Key Lab. of Intelligent Info. Processing and Shenzhen Key Laboratory of Media Security Shenzhen China School of Computer Science and Technology Shandong University Qingdao China School of Computer Science and Engineering South China University of Technology Guangzhou China

ISBN: (纸本)9781665410205

Concrete structural crack damage classification is of importance for road safety. This paper proposes a new method based on broad neural network for crack damage classification in concrete structures. It includes three stages. Firstly, a pre-trained deep neural network is used to extract the features from crack images. Secondly, principal component analysis is used to project the retrieved features from high dimensions to low dimensions. Thirdly, broad learning system is employed to predict the classification using the low-dimensional features. Experimental results demonstrate that this method reduces the model's training time and improves classification accuracy. © 2024 IEEE.

关键词： Deep neural networks

Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidance

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Arrabi, Ahmad Zhang, Xiaohan Sultani, Waqas Chen, Chen Wshah, Safwan Vermont Artificial Intelligence Lab. Department of Computer Science University of Vermont United States Intelligent Machines Lab. Information Technology University Center for Research in Computer Vision University of Central Florida United States

Aerial imagery analysis is critical for many research fields. However, obtaining frequent high-quality aerial images is not always accessible due to its high effort and cost requirements. One solution is to use the Ground-to-Aerial (G2A) technique to synthesize aerial images from easily collectible ground images. However, G2A is rarely studied, because of its challenges, including but not limited to, the drastic view changes, occlusion, and range of visibility. In this paper, we present a novel Geometric Preserving Ground-to-Aerial (G2A) image synthesis (GPG2A) model that can generate realistic aerial images from ground images. GPG2A consists of two stages. The first stage predicts the Bird’s Eye View (BEV) segmentation (referred to as the BEV layout map) from the ground image. The second stage synthesizes the aerial image from the predicted BEV layout map and text descriptions of the ground image. To train our model, we present a new multi-modal cross-view dataset, namely VIGORv2, built upon VIGOR [66] with newly collected aerial images, maps, and text descriptions. Our extensive experiments illustrate that GPG2A synthesizes better geometry-preserved aerial images than existing models. We also present two applications, data augmentation for cross-view geo-localization and sketch-based region search, to further verify the effectiveness of our GPG2A. The code and dataset are availab.e at https://***/vail-uvm/gpg2a. Copyright © 2024, The Authors. All rights reserved.

关键词： Aerial photography

Purposeful XR: Affordances, Challenges, and Speculations for an Ethical Future

学校读者我要写书评

暂无评论

Purposeful XR: Affordances, Challenges, and Speculations for...

2025 CHI Conference on Human Factors in Computing Systems, CHI EA 2025

作者： Childs, Elizabeth Ghosh, Samir Cmentowski, Sebastian Cuadra, Andrea Ratan, Rabindra IXD Lab. Stanford University StanfordCA United States Social Emotional Technology Lab. University of California Santa Cruz Santa CruzCA United States Eindhoven University of Technology Eindhoven Netherlands Department of Computer Science Olin College of Engineering NeedhamMA United States Department of Media and Information Michigan State University East LansingMI United States

ISBN: (纸本)9798400713958

Education, healthcare, poverty, and equity are just some of the social problems in which XR researchers leverage augmented reality, virtual reality, and mixed reality to create novel solutions. In this workshop proposal, we intend on gathering XR researchers who are interested in making a positive impact with their research, and to use this opportunity to discuss leveraging unique affordances of XR technology and common challenges. In our motivation, we refer to recent human computer interaction gatherings that discuss applications of XR to social benefit. Then, we present a background of research across XR technologies and diverse social problems to illustrate example affordances and challenges. To attend the workshop, attendees will submit a position paper on a particular XR affordances and how it can help address a social problem, and optionally submit a demo artifact as a sample for group discussion. The workshop will include a keynote, discussion based activities and demos, and conclude with a speculative exercise, where attendees will work together to describe XR technology in the context of an ethical future. © 2025 Copyright held by the owner/author(s).

关键词： Ethical technology

Image Segmentation in Foundation Model Era: A Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Tianfei Zhang, Fei Chang, Boyu Wang, Wenguan Yuan, Ye Konukoglu, Ender Cremers, Daniel Department of Computer Science Beijing Institute of Technology China Department of Electronic Engineering Shanghai Jiao Tong University China CCAI Zhejiang University China Computer Vision Lab. ETH Zurich Switzerland The Department of Computer Science Technical University of Munich Germany

Image segmentation is a long-standing challenge in computer vision, studied continuously over several decades, as evidenced by seminal algorithms such as N-Cut, FCN, and MaskFormer. With the advent of foundation models (FMs), contemporary segmentation methodologies have embarked on a new epoch by either adapting FMs (e.g., CLIP, Stable Diffusion, DINO) for image segmentation or developing dedicated segmentation foundation models (e.g., SAM, SAM2). These approaches not only deliver superior segmentation performance, but also herald newfound segmentation capabilities previously unseen in deep learning context. However, current research in image segmentation lacks a detailed analysis of distinct characteristics, challenges, and solutions associated with these advancements. This survey seeks to fill this gap by providing a thorough review of cutting-edge research centered around FM-driven image segmentation. We investigate two basic lines of research – generic image segmentation (i.e., semantic segmentation, instance segmentation, panoptic segmentation), and promptable image segmentation (i.e., interactive segmentation, referring segmentation, few-shot segmentation) – by delineating their respective task settings, background concepts, and key challenges. Furthermore, we provide insights into the emergence of segmentation knowledge from FMs like CLIP, Stable Diffusion, and DINO. An exhaustive overview of over 300 segmentation approaches is provided to encapsulate the breadth of current research efforts. Subsequently, we engage in a discussion of open issues and potential avenues for future research. We envisage that this fresh, comprehensive, and systematic survey catalyzes the evolution of advanced image segmentation systems. © 2024, CC BY.

关键词： Semantic Segmentation

A Feature Reuse Framework with Texture-adaptive Aggregation for Reference-based Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Mei, Xiaoyong Yang, Yi Li, Ming Huang, Changqin Zhang, Kai Lió, Pietro The Key Laboratory of Intelligent Education Technology Application of Zhejiang Province Zhejiang Normal University Jinhua China The Computer Vision Lab. ETH Zurich Switzerland The Department of Computer Science and Technology University of Cambridge United Kingdom

Reference-based super-resolution (RefSR) has gained considerable success in the field of super-resolution with the addition of high-resolution reference images to reconstruct low-resolution (LR) inputs with more high-frequency details, thereby overcoming some limitations of single image super-resolution (SISR). Previous research in the field of RefSR has mostly focused on two crucial aspects. The first is accurate correspondence matching between the LR and the reference (Ref) image. The second is the effective transfer and aggregation of similar texture information from the Ref images. Nonetheless, an important detail of perceptual loss and adversarial loss has been underestimated, which has a certain adverse effect on texture transfer and reconstruction. In this study, we propose a feature reuse framework that guides the step-by-step texture reconstruction process through different stages, reducing the negative impacts of perceptual and adversarial loss. The feature reuse framework can be used for any RefSR model, and several RefSR approaches have improved their performance after being retrained using our framework. Additionally, we introduce a single image feature embedding module and a texture-adaptive aggregation module. The single image feature embedding module assists in reconstructing the features of the LR inputs itself and effectively lowers the possibility of including irrelevant textures. The texture-adaptive aggregation module dynamically perceives and aggregates texture information between the LR inputs and the Ref images using dynamic filters. This enhances the utilization of the reference texture while reducing reference misuse. The source code is availab.e at https://***/Yi-Yang355/FRFSR. © 2023, CC BY.

关键词： Textures

Chinese Emergency Event Extraction Based on Contrastive Learning with Event Semantic Features 19

学校读者我要写书评

暂无评论

Chinese Emergency Event Extraction Based on Contrastive Lear...

19th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2023

作者： Sheng, Xinyi Gu, Jinguang Yan, Youcheng Xu, Fangfang Wuhan University of Science and Technology College of Computer Science and Technology Wuhan430065 China Institute of Big Data Science and Engineering Wuhan University of Science and Technology Wuhan430065 China The Key Lab. of Rich-Media Knowledge Org. and Serv. of Digit. Publ. Content Inst. of Sci. and Tech. Info. of China Beijing100038 China

ISBN: (纸本)9798350304398

Extracting emergency events from a large amount of unstructured information is essential for improving early warning and emergency response. Existing event extraction methods for specialist fields often rely on well-designed syntactic dependencies or external knowledge graphs, which can provide high accuracy in their specialist fields. However, these approaches may have poor generalization capabilities and are difficult to apply directly to the emergency field. In addition, most existing methods ignore the interdependencies between event triggers and event arguments, which may be crucial semantic features for event extraction. To solve these problems, we propose a Chinese emergent event extraction method based on contrastive learning with event semantic features(CNEmergEE). Firstly, the proposed approach automatically expands the pre-trained model's vocabulary on a domain corpus. Then, continuing learning on the same corpus allows the model to complete domain adaptation. Finally, we use contrast learning to enable models to learn semantic representations of events. Experimental results on the CEC (Chinese Emergency Corpus) dataset show that our approach achieves better improvement. © 2023 IEEE.

关键词： Semantics

Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey

学校读者我要写书评

暂无评论

Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Surv...

IEEE computer Society Conference on computer vision and Pattern Recognition Workshops (CVPRW)

作者： Marcos V. Conde Florin-Alexandru Vasluianu Radu Timofte Jianxing Zhang Jia Li Fan Wang Xiaopeng Li Zikun Liu Hyunhee Park Sejun Song Changho Kim Zhijuan Huang Hongyuan Yu Cheng Wan Wending Xiang Jiamin Lin Hang Zhong Qiaosong Zhang Yue Sun Xuanwu Yin Kunlong Zuo Senyan Xu Siyuan Jiang Zhijing Sun Jiaying Zhu Liangyan Li Ke Chen Yunzhe Li Yimo Ning Guanhua Zhao Jun Chen Jinyang Yu Kele Xu Qisheng Xu Yong Dou CAIDAS & IFI Computer Vision Lab. University of Würzburg Samsung Research China - Beijing (SRC-B) Samsung MX(Mobile eXperience) Business Xiaomi Inc. Georgia Institute of Technology University of Science and Technology of China McMaster University Computer Dept. National University of Defense Technology Changsha China

ISBN: (数字)9798350365474

ISBN: (纸本)9798350365481

This paper reviews the NTIRE 2024 RAW Image Super-Resolution Challenge, highlighting the proposed solutions and results. New methods for RAW Super-Resolution could be essential in modern Image Signal Processing (ISP) pipelines, however, this problem is not as explored as in the RGB domain. Th goal of this challenge is to upscale RAW Bayer images by 2x, considering unknown degradations such as noise and blur. In the challenge, a total of 230 participants registered, and 45 submitted results during thee challenge period. The performance of the top-5 sub-missions is reviewed and provided here as a gauge for the current state-of-the-art in RAW Image Super-Resolution.

关键词： Surveys Degradation computer vision Reviews Conferences Superresolution Pipelines Noise Pattern recognition

Unsupervised Pre-training with Language-vision Prompts for Low-Data Instance Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Dingwen Li, Hao He, Diqi Liu, Nian Cheng, Lechao Wang, Jingdong Han, Junwei The Brain and Artificial Intelligence Lab. Northwestern Polytechnical University Shaanxi Xi’an710000 China The Inception Institute of Artificial Intelligence United Kingdom The School of Computer Science and Information Engineering Hefei University of Technology Hefei230601 China Department of Computer Vision Baidu Inc. China

In recent times, following the paradigm of DETR (DEtection TRansformer), query-based end-to-end instance segmentation (QEIS) methods have exhibited superior performance compared to CNN-based models, particularly when trained on large-scale datasets. Nevertheless, the effectiveness of these QEIS methods diminishes significantly when confronted with limited training data. This limitation arises from their reliance on substantial data volumes to effectively train the pivotal queries/kernels that are essential for acquiring localization and shape priors. To address this problem, we propose a novel method for unsupervised pre-training in low-data regimes. Inspired by the recently successful prompting technique, we introduce a new method, Unsupervised Pre-training with Language-vision Prompts (UPLVP), which improves QEIS models’ instance segmentation by bringing language-vision prompts to queries/kernels. Our method consists of three parts: (1) Masks Proposal: Utilizes language-vision models to generate pseudo masks based on unlab.led images. (2) Prompt-Kernel Matching: Converts pseudo masks into prompts and injects the best-matched localization and shape features to their corresponding kernels. (3) Kernel Supervision: Formulates supervision for pre-training at the kernel level to ensure robust learning. With the help of our pre-training method, QEIS models can converge faster and perform better than CNN-based models in low-data regimes. Experimental evaluations conducted on MS COCO, Cityscapes, and CTW1500 datasets indicate that the QEIS models’ performance can be significantly improved when pre-trained with our method. Code will be availab.e at: https://***/lifuguan/UPLVP. Copyright © 2024, The Authors. All rights reserved.

关键词： Large datasets