检索结果-内蒙古大学图书馆

SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Di Zhang, Jing Du, Bo Xu, Minqiang Liu, Lin Tao, Dacheng Zhang, Liangpei School of Computer Science National Engineering Research Center for Multimedia Software Institute of Artificial Intelligence Hubei Key Laboratory of Multimedia and Network Communication Engineering Wuhan University China School of Computer Science Faculty of Engineering The University of Sydney Australia National Engineering Research Center of Speech and Language Information Processing China State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing Wuhan University China

The success of the Segment Anything Model (SAM) demonstrates the significance of data-centric machine learning. However, due to the difficulties and high costs associated with annotating Remote Sensing (RS) images, a large amount of valuable RS data remains unlabeled, particularly at the pixel level. In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS. SAMRS totally possesses 105,090 images and 1,668,241 instances, surpassing existing high-resolution RS segmentation datasets in size by several orders of magnitude. It provides object category, location, and instance information that can be used for semantic segmentation, instance segmentation, and object detection, either individually or in combination. We also provide a comprehensive analysis of SAMRS from various aspects. Moreover, preliminary experiments highlight the importance of conducting segmentation pre-training with SAMRS to address task discrepancies and alleviate the limitations posed by limited training data during fine-tuning. The code and dataset will be available at SAMRS. Copyright © 2023, The Authors. All rights reserved.

关键词： Remote sensing

Eye movement patterns are similar during accurate multiple-target tracking

学校读者我要写书评

暂无评论

Eye movement patterns are similar during accurate multiple-t...

International Conference on Cognitive Infocommunications (CogInfoCom)

作者： Kamyar Bagha Shiva Kamkar Hamid Abrishami Moghaddam Lauri Oksama Jie Li Jukka Hyönä Computer Engineering Department Khatam University Tehran Iran Machine Vision and Medical Image Processing (MVMIP) Laboratory Faculty of Electrical Engineering K.N.Toosi University of Technology Tehran Iran Center for International Scientific Studies and Collaboration (CISSC) Tehran Iran Department of Psychology and Speech-Language Pathology University of Turku Turku Finland Center for Cognition and Brain Disorders Hangzhou Normal University Hangzhou China

ISBN: (数字)9798350378245

ISBN: (纸本)9798350378252

Understanding how the brain works is a base of cognitive info-communication. To this aim we focus on multiple target tracking (MTT) as a key task that involves two important cognitive factors, attention and memory. Humans track multiple objects in their daily life while facing various challenges including occlusion and set-size. Eye movement research has shown that there are within and between subjects’ differences in scanpaths while performing MTT tasks. However, it is unclear if there is a winning scan pattern that would lead to a successful tracking of targets. To answer this question, we used dynamic time warping to compare the similarities between subjects’ scan patterns during an MTT task with different challenges. We studied the effect of set-size, occlusion, and trial response on the similarities. Then a mixed effect analysis was applied on the output to measure whether the findings were statistically significant. Results demonstrated that scan patterns were more similar when MTT task was performed correctly. It suggests that there is a common tracking strategy adopted by the viewers that leads to a correct response. Decoding this strategy has countless applications in the fields including human-computer interaction, brain-modeling and cognitive info-communication.

关键词： Human computer interaction Visualization Target tracking Accuracy Predictive models Communications technology Time measurement Cognition Decoding Cognitive science

Software in the natural world: A computational approach to hierarchical emergence

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Rosas, Fernando E. Geiger, Bernhard C. Luppi, Andrea I. Seth, Anil K. Polani, Daniel Gastpar, Michael Mediano, Pedro A.M. Department of Informatics University of Sussex United Kingdom Sussex Centre for Consciousness Science and Sussex AI University of Sussex United Kingdom Center for Psychedelic Research and Centre for Complexity Science Department of Brain Science Imperial College London United Kingdom Center for Eudaimonia and Human Flourishing University of Oxford United Kingdom Know-Center GmbH Graz Austria Signal Processing and Speech Communication Laboratory Graz University of Technology Graz Austria Montreal Neurological Institute McGill University Canada Department of Computer Science University of Hertfordshire Hatfield United Kingdom School of Computer and Communication Sciences EPFL Lausanne Switzerland Department of Computing Imperial College London United Kingdom Division of Psychology and Language Sciences University College London United Kingdom

Understanding the functional architecture of complex systems is crucial to illuminate their inner workings and enable effective methods for their prediction and control. Recent advances have introduced tools to characterise emergent macroscopic levels;however, while these approaches are successful in identifying when emergence takes place, they are limited in the extent they can determine how it does. Here we address this important limitation by developing a computational approach to emergence, which characterises macroscopic processes in terms of their computational capabilities. Concretely, we articulate a view on emergence based on how software works, which is rooted on a mathematical formalisation of how macroscopic processes can express self-contained informational, interventional, and computational properties. This framework reveals a hierarchy of nested self-contained processes that determines what computations take place at what level, which in turn delineates the functional architecture of a complex system. This approach is illustrated on paradigmatic models from the statistical physics and computational neuroscience literature, which are shown to exhibit macroscopic processes that are akin to software in human-engineered systems. Overall, this framework enables a deeper understanding of the multi-level structure of complex systems, revealing specific ways in which they can be efficiently simulated, predicted, and controlled. Copyright © 2024, The Authors. All rights reserved.

关键词： Statistical Physics

Wake word detection with streaming transformers

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Yiming Lv, Hang Povey, Daniel Xie, Lei Khudanpur, Sanjeev Center for Language and Speech Processing Johns Hopkins University BaltimoreMD United States Center for Language and Speech Processing Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States ASLP@NPU School of Computer Science Northwestern Polytechnical University Xi’an China Xiaomi Corporation Beijing China

Modern wake word detection systems usually rely on neural networks for acoustic modeling. Transformers has recently shown superior performance over LSTM and convolutional networks in various sequence modeling tasks with their better temporal modeling power. However it is not clear whether this advantage still holds for short-range temporal modeling like wake word detection. Besides, the vanilla Transformer is not directly applicable to the task due to its non-streaming nature and the quadratic time and space complexity. In this paper we explore the performance of several variants of chunk-wise streaming Transformers tailored for wake word detection in a recently proposed LF-MMI system, including looking-ahead to the next chunk, gradient stopping, different positional embedding methods and adding same-layer dependency between chunks. Our experiments on the Mobvoi wake word dataset demonstrate that our proposed Transformer model outperforms the baseline convolution network by 25% on average in false rejection rate at the same false alarm rate with a comparable model size, while still maintaining linear complexity w.r.t. the sequence length. Copyright © 2021, The Authors. All rights reserved.

关键词： Wakes

PQLM - MULTILINGUAL DECENTRALIZED PORTABLE QUANTUM language MODEL FOR PRIVACY PROTECTION

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Shuyue Stella Zhang, Xiangyu Zhou, Shu Shu, Hongchao Liang, Ruixing Liu, Hexin Garcia, Leibny Paola Center for Language and Speech Processing Johns Hopkins University United States Human Language Technology Center of Excellence Johns Hopkins University United States Department of Physics Hong Kong University of Science and Technology Hong Kong School of Electrical and Electronic Engineering Nanyang Technological University Singapore

With careful manipulation, malicious agents can reverse engineer private information encoded in pre-trained language models. Security concerns motivate the development of quantum pre-training. In this work, we propose a highly portable quantum language model (PQLM) that can easily transmit information to downstream tasks on classical machines. The framework consists of a cloud PQLM built with random Variational Quantum Classifiers (VQC) and local models for downstream applications. We demonstrate the ad hoc portability of the quantum model by extracting only the word embeddings and effectively applying them to downstream tasks on classical machines. Our PQLM exhibits comparable performance to its classical counterpart on both intrinsic evaluation (loss, perplexity) and extrinsic evaluation (multilingual sentiment analysis accuracy) metrics. We also perform ablation studies on the factors affecting PQLM performance to analyze model stability. Our work establishes a theoretical foundation for a portable quantum pre-trained language model that could be trained on private data and made available for public use with privacy protection guarantees. Copyright © 2022, The Authors. All rights reserved.

关键词： Modeling languages

The Second Multi-Channel Multi-Party Meeting Transcription Challenge (M2MeT 2.0): A Benchmark for Speaker-Attributed ASR

学校读者我要写书评

暂无评论

The Second Multi-Channel Multi-Party Meeting Transcription C...

IEEE Workshop on Automatic speech Recognition and Understanding

作者： Yuhao Liang Mohan Shi Fan Yu Yangze Li Shiliang Zhang Zhihao Du Qian Chen Lei Xie Yanmin Qian Jian Wu Zhuo Chen Kong Aik Lee Zhijie Yan Hui Bu Audio Speech and Language Processing Group (ASLP@NPU) School of Computer Science Northwestern Polytechnical University China Speech Lab of DAMO Academy Alibaba Group China NERC-SLIP University of Science and Technology of China (USTC) China SpeechLab Department of Computer Science and Engineering Shanghai Jiao Tong University China ICT Cluster Singapore Institute of Technology Singapore Institute for Infocomm Research A*STAR Singapore Beijing Shell Shell Technology Co. Ltd. Beijing China

With the success of the first Multi-channel Multi-party Meeting Transcription challenge (M2MeT), the second M2MeT challenge (M2MeT 2.0) held in ASRU2023 particularly aims to tackle the complex task of speaker-attributed ASR (SAASR), which directly addresses the practical and challenging problem of “who spoke what at when” at typical meeting scenario. We particularly established two sub-tracks. The fixed training condition sub-track, where the training data is constrained to predetermined datasets, but participants can use any open-source pre-trained model. The open training condition sub-track, which allows for the use of all available data and models without limitation. In addition, we release a new 10-hour test set for challenge ranking. This paper provides an overview of the dataset, track settings, results, and analysis of submitted systems, as a benchmark to show the current state of speaker-attributed ASR.

关键词：

SPECTRE: Visual speech-Informed Perceptual 3D Facial Expression Reconstruction from Videos

学校读者我要写书评

暂无评论

SPECTRE: Visual Speech-Informed Perceptual 3D Facial Express...

IEEE computer Society Conference on computer Vision and Pattern Recognition Workshops (CVPRW)

作者： Panagiotis P. Filntisis George Retsinas Foivos Paraperas-Papantoniou Athanasios Katsamanis Anastasios Roussos Petros Maragos Institute of Robotics Athena Research Center Maroussi Greece School of Electrical & Computer Engineering National Technical University of Athens Greece Imperial College London UK Institute for Language and Speech Processing Athena R.C. Greece Institute of Computer Science (ICS) Foundation for Research & Technology - Hellas (FORTH) Greece College of Engineering Mathematics and Physical Sciences University of Exeter UK

The recent state of the art on monocular 3D face reconstruction from image data has made some impressive advancements, thanks to the advent of Deep Learning. However, it has mostly focused on input coming from a single RGB image, overlooking the following important factors: a) Nowadays, the vast majority of facial image data of interest do not originate from single images but rather from videos, which contain rich dynamic information. b) Furthermore, these videos typically capture individuals in some form of verbal communication (public talks, teleconferences, audiovisual human-computer interactions, interviews, monologues/dialogues in movies, etc). When existing 3D face reconstruction methods are applied in such videos, the artifacts in the reconstruction of the shape and motion of the mouth area are often severe, since they do not match well with the speech *** overcome the aforementioned limitations, we present the first method for visual speech-informed perceptual reconstruction of 3D mouth expressions. We do this by proposing a "lipreading" loss, which guides the fitting process so that the elicited perception from the 3D reconstructed talking head resembles that of the original video footage. We demonstrate that, interestingly, the lipreading loss is better suited for 3D reconstruction of mouth movements compared to traditional landmark losses, and even direct 3D supervision. Furthermore, the devised method does not rely on any text transcriptions or corresponding audio, rendering it ideal for training in unlabeled datasets. We verify the efficiency of our method through objective evaluations on three large-scale datasets, as well as subjective evaluation with two web-based user studies. Project webpage: https://***/spectre/

关键词：

Unpaired Overwater Image Defogging Using Prior Map Guided Cycle-Consistent Generative Adversarial Network

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Mo, Yaozong Li, Chaofeng Ren, Wenqi Wang, Wenwu Wu, Xiao-Jun Shanghai201306 China School of Cyber Science and Technology Sun Yat-sen University Shenzhen518000 China Center for Vision Speech and Signal Processing Department of Electrical and Electronic Engineering University of Surrey Surrey SurreyGU2 7XH United Kingdom School of Artificial Intelligence and Computer Science Jiangnan University Wuxi 214122 China

Existing image defogging approaches have made significant advancements. But their effectiveness in addressing overwater foggy images remains limited. Current methods are predominantly optimized for land scenes, which causes poor performance when applied to overwater images, since overwater scenes typically contain large expanses of sky and water. In this paper, we propose a prior map guided cycle-consistent generative adversarial network (PG-CycleGAN) for defogging of images with overwater scenes. To promote the recovery of the objects on water in the image, two loss functions are exploited for the network where a prior map is designed to invert the dark channel and the min-max normalization is used to suppress the sky and emphasize objects. However, due to the unpaired training set, the network may learn an under-constrained domain mapping, leading to artifacts and loss of details. Thus, we propose an intuitive Upscaling Inception Module (UIM) and a Long-range Residual Coarse-to-fine framework (LRC) to mitigate this issue. Extensive experimental results demonstrate that PG-CycleGAN is effective in overwater image defogging. © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

Cycleflow: Purify information factors by cycle loss

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sun, Haoran Chen, Chen Li, Lantian Wang, Dong Center for Speech and Language Technologies BNRist Tsinghua University China Department of Computer Science and Technology Tsinghua University China

speechFlow is a powerful factorization model based on information bottleneck (IB), and its effectiveness has been reported by several studies. A potential problem of speechFlow, however, is that if the IB channels are not well designed, the resultant factors cannot be well disentangled. In this study, we propose a CycleFlow model that combines random factor substitution and cycle loss to solve this problem. Experiments on voice conversion tasks demonstrate that this simple technique can effectively reduce mutual information among individual factors, and produce clearly better conversion than the IB-based speechFlow. CycleFlow can also be used as a powerful tool for speech editing. We demonstrate this usage by an emotion perception experiment. Copyright © 2021, The Authors. All rights reserved.

关键词： Factorization