检索结果-内蒙古大学图书馆

SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model

IEEE Journal on Selected Topics in Signal Processing 2025年

作者： Qian, Xinyuan Gao, Jiaran Zhang, Yaodan Zhang, Qiquan Liu, Hexin Garcia, Leibny Paola Li, Haizhou University of Science and Technology Beijing School of Computer and Communication Engineering Beijing100083 China University of New South Wales School of Electrical Engineering and Telecommunications Sydney2052 Australia Nanyang Technological University College of Computing and Data Science Singapore Singapore Johns Hopkins University CLSP and HLT-COE United States Chinese University of HongKong Guangdong Provincial Key Laboratory of Big Data Computing Shenzhen518172 China Shenzhen Research Institute of Big data Shenzhen51872 China

Speech enhancement plays an essential role in various applications, and the integration of visual information has been demonstrated to bring substantial advantages. However, the majority of current research concentrates on the examination of facial and lip movements, which can be compromised or entirely inaccessible in scenarios where occlusions occur or when the camera view is distant. Whereas contextual visual cues from the surrounding environment have been overlooked: for example, when we see a dog bark, our brain has the innate ability to discern and filter out the barking noise. To this end, in this paper, we introduce a novel task, i.e. Scene-aware Audio-Visual Speech Enhancement SAV-SE. To our best knowledge, this is the first proposal to use rich contextual information from synchronized video as auxiliary cues to indicate the type of noise, which eventually improves the speech enhancement performance. Specifically, we propose the VC-S $^{2}$ E method, which incorporates the Conformer and Mamba modules for their complementary strengths. Extensive experiments are conducted on public MUSIC, AVSpeech and AudioSet datasets, where the results demonstrate the superiority of VC-S $^{2}$ E over other competitive methods. We will make the source code publicly available. Project demo page: https://***/ © 2007-2012 IEEE.

关键词： Speech enhancement

来源：评论

学校读者我要写书评

暂无评论

SMGC-SBERT: A Multi-Feature Fusion Chinese Short Text Similarity Computation Model Based on Optimised SBERT

SMGC-SBERT: A Multi-Feature Fusion Chinese Short Text Simila...

引用

2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2024

作者： Zhao, Shuo Gu, Qiliang Zhang, Jianqiang Song, Gongpeng Lu, Qin Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center Jinan China Shandong Engineering Research Center of Big Data Applied Technology Faculty of Computer Science and Technology Jinan China Shandong Provincial Key Laboratory of Computer Networks Shandong Fundamental Research Center for Computer Science Jinan China Shandong Branch of China Mobile Communication Group Design Institute Co. Jinan China

ISBN: (纸本)9781665410205

Chinese short text similarity computation stands as a pivotal task within natural language processing, garnering significant attention. However, existing models grapple with limitations in handling intricate semantic relationships, such as the challenge of discerning subtle semantic nuances in text, inadequacies in effectively integrating diverse levels of semantic information, and the struggle to capture polysemous meanings accurately. In addressing these issues, this paper introduces an innovative Chinese short text similarity computation model, SMGC-SBERT. This model addresses shortcomings of the existing models by employing a multi-module fusion strategy, thereby enabling a more precise measurement of semantic similarity between texts. Primarily, the model incorporates SAT embedding to acquire phrase-level semantic information and leverages the MS-BERT model to encode text, improving the model's comprehension of textual polysemy and obtaining richer semantic representation. Then, the fusion of module features, including multi-branch convolutional networks and mix pooling, enables the extraction of textual features from varied levels, bolstering the model's representational capacity. Additionally, to further reduce overfitting risks while improving accuracy and other performance, a multi-layer feature adjustment network is utilized for short text similarity calculation. The final resultant experimental findings showcase the superiority of the SMGC-SBERT model over other neural network models, demonstrating significant advancements across both Chinese-SNLI and CCKS2018_Task3 Chinese short text datasets. © 2024 IEEE.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

S2AC: ENERGY-BASED REINFORCEMENT LEARNING WITH STEIN SOFT ACTOR CRITIC

arXiv

引用

arXiv 2024年

作者： Messaoud, Safa Mokeddem, Billel Xue, Zhenghai Pang, Linsey An, Bo Chen, Haipeng Chawla, Sanjay Qatar Computing Research Institute Hamad Bin Khalifa University Qatar School of Computer Science and Engineering Nanyang Technological University Singapore SalesForce Skywork AI Singapore Data Science William & Mary

Learning expressive stochastic policies instead of deterministic ones has been proposed to achieve better stability, sample complexity, and robustness. Notably, in Maximum Entropy Reinforcement Learning (MaxEnt RL), the policy is modeled as an expressive Energy-Based Model (EBM) over the Q-values. However, this formulation requires the estimation of the entropy of such EBMs, which is an open problem. To address this, previous MaxEnt RL methods either implicitly estimate the entropy, resulting in high computational complexity and variance (SQL), or follow a variational inference procedure that fits simplified actor distributions (e.g., Gaussian) for tractability (SAC). We propose Stein Soft Actor-Critic (S2AC), a MaxEnt RL algorithm that learns expressive policies without compromising efficiency. Specifically, S2AC uses parameterized Stein Variational Gradient Descent (SVGD) as the underlying policy. We derive a closed-form expression of the entropy of such policies. Our formula is computationally efficient and only depends on first-order derivatives and vector products. Empirical results show that S2AC yields more optimal solutions to the MaxEnt objective than SQL and SAC in the multi-goal environment, and outperforms SAC and SQL on the MuJoCo benchmark. © 2024, CC BY.

关键词： Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

FERI: Feature Enhancement and Relational Interaction for Image-text Matching 30

FERI: Feature Enhancement and Relational Interaction for Ima...

引用

30th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2024

作者： Zhang, Yu Zhang, Jianqiang Song, Gongpeng Lu, Qin Zhao, Shuo Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center Jinan China Shandong Engineering Research Center of Big Data Applied Technology Faculty of Computer Science and Technology Jinan China Shandong Fundamental Research Center for Computer Science Shandong Provincial Key Laboratory of Computer Networks Jinan China Shandong Branch of China Mobile Communication Group Design Institute Co. Jinan China

ISBN: (纸本)9798331515966

Image-text matching is an important problem at the intersection of computer vision and natural language processing. It aims to establish the semantic link between image and text to achieve high-quality semantic alignment between the two modalities. However, the existing methods have the problem that the meaning expressed in the image or the complex narrative in the text cannot be fully understood due to insufficient feature extraction. Moreover, due to the essential modal differences between images and texts, how to effectively and accurately align the semantic contents in images and texts has become the key of research. In order to solve the above problems, this paper proposes a method based on feature enhancement and relationship interaction. When processing images, the proposed method fuses labeled features, region features and location features to represent images. When processing text, a combination of Bi-GRU and self-attention mechanism is used to represent the text. In order to further align the semantic content in images and texts accurately, this paper improves two relational interaction mechanisms by identifying connection relationships and learning association relationships. Thus, the relation enhanced embedding is obtained. Finally, it calculated the similarity of the enhanced embedding to judge the matching degree of the image and text. Extensive experiments on the public datasets Flickr30K and MSCOCO demonstrate the effectiveness of our method. © 2024 IEEE.

关键词： cross-modal retrieval feature enhancement image-text matching relationship interaction

来源：评论

学校读者我要写书评

暂无评论

GoPIM: GCN-Oriented Pipeline Optimization for PIM Accelerators

GoPIM: GCN-Oriented Pipeline Optimization for PIM Accelerato...

引用

IEEE Symposium on High-Performance Computer Architecture

作者： Siling Yang Shuibing He Wenjiong Wang Yanlong Yin Tong Wu Weijian Chen Xuechen Zhang Xian-He Sun Dan Feng The State Key Laboratory of Blockchain and Data Security Zhejiang University Zhejiang Lab Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security Zhejiang Key Laboratory of Big Data Intelligent Computing Washington State University Vancouver Illinois Institute of Technology Huazhong University of Science and Technology Wuhan National Laboratory for Optoelectronics

ISBN: (数字)9798331506476

ISBN: (纸本)9798331506483

Graph convolutional networks (GCNs) are popular for a variety of graph learning tasks. ReRAM-based processing-in-memory (PIM) accelerators are promising to expedite GCN training owing to their in-situ computing capability. However, existing accelerators can be severely underutilized even with pipelines, due to the oversight of the skewed execution times of various GCN stages and the ignorance of skewed degrees of graph vertices. In this work, we propose GOPIM, a GCN-oriented pipeline optimization for PIM accelerators to expedite GCN training. First, GOPIM proposes an ML-based scheme that allocates crossbar resources to the most needed stages to streamline the overall pipeline. Second, GOPIM utilizes a selective vertex updating technique that evenly distributes vertices on crossbars by interleaved mapping. These techniques collectively reduce the overall execution time without losing much accuracy. We also provide a practical architecture design for GOPIM. Our experimental results show that, GoPIM achieves up to 191 × speedup and 16.1 × energy saving, compared to the state-of-the-art work.

关键词： Training Accuracy Graph convolutional networks Pipelines Energy conservation Computer architecture Machine learning Parallel processing Optimization

来源：评论

学校读者我要写书评

暂无评论

Language Model Evolutionary Algorithms for Recommender Systems: Benchmarks and Algorithm Comparisons

arXiv

引用

arXiv 2024年

作者： Liu, Jiao Sun, Zhu Feng, Shanshan Chen, Caishun Ong, Yew-Soon College of Computing & Data Science Nanyang Technological University Singapore Information Systems Technology and Design Singapore University of Technology and Design Singapore Centre for Frontier AI Research A*STAR The Institute of High Performance Computing A*STAR Singapore

In the evolutionary computing community, the remarkable language-handling capabilities and reasoning power of large language models (LLMs) have significantly enhanced the functionality of evolutionary algorithms (EAs), enabling them to tackle optimization problems involving structured language or program code. Although this field is still in its early stages, its impressive potential has led to the development of various LLM-based EAs. To effectively evaluate the performance and practical applicability of these LLM-based EAs, benchmarks with real-world relevance are essential. In this paper, we focus on LLM-based recommender systems (RSs) and introduce a benchmark problem set, named RSBench, specifically designed to assess the performance of LLM-based EAs in recommendation prompt optimization. RSBench emphasizes session-based recommendations, aiming to discover a set of Pareto optimal prompts that guide the recommendation process, providing accurate, diverse, and fair recommendations. We develop three LLM-based EAs based on established EA frameworks and experimentally evaluate their performance using RSBench. Our study offers valuable insights into the application of EAs in LLM-based RSs. Additionally, we explore key components that may influence the overall performance of the RS, providing meaningful guidance for future research on the development of LLM-based EAs in RSs. © 2024, CC BY.

关键词： Algorithmic languages

来源：评论

学校读者我要写书评

暂无评论

Improving VTE Identification through Adaptive NLP Model Selection and Clinical Expert Rule-based Classifier from Radiology Reports

arXiv

引用

arXiv 2023年

作者： Deng, Jamie Wu, Yusen Hayssen, Hilary Englum, Brain Kankaria, Aman Mayorga-Carlin, Minerva Sahoo, Shalini Sorkin, John Lal, Brajesh Yesha, Yelena Nguyen, Phuong Frost Institute for Data Science & Computing University of Miami FL United States School of Medicine University of Maryland MD United States

Rapid and accurate identification of Venous thromboembolism (VTE), a severe cardiovascular condition including deep vein thrombosis (DVT) and pulmonary embolism (PE), is important for effective treatment. Leveraging Natural Language Processing (NLP) on radiology reports, automated methods have shown promising advancements in identifying VTE events from retrospective data cohorts or aiding clinical experts in identifying VTE events from radiology reports. However, effectively training Deep Learning (DL) and the NLP models is challenging due to limited labeled medical text data, the complexity and heterogeneity of radiology reports, and data imbalance. This study proposes novel method combinations of DL methods, along with data augmentation, adaptive pre-trained NLP model selection, and a clinical expert NLP rule-based classifier, to improve the accuracy of VTE identification in unstructured (freetext) radiology reports. Our experimental results demonstrate the model’s efficacy, achieving an impressive 97% accuracy and 97% F1 score in predicting DVT, and an outstanding 98.3% accuracy and 98.4% F1 score in predicting PE. These findings emphasize the model’s robustness and its potential to significantly contribute to VTE research. © 2023, CC BY-NC-ND.

关键词： Natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques 38

Suppress Content Shift: Better Diffusion Features via Off-th...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Yang, Zhiyong Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Computer Science and Tech. University of Chinese Academy of Sciences China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are powerful generative models, and this capability can also be applied to discrimination. The inner activations of a pre-trained diffusion model can serve as features for discriminative tasks, namely, diffusion feature. We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. To be specific, there are content differences between features and the input image, such as the exact shape of a certain object. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. Further empirical study also indicates that its negative impact is not negligible even when content shift is not visually perceivable. Hence, we propose to suppress content shift to enhance the overall quality of diffusion features. Specifically, content shift is related to the information drift during the process of recovering an image from the noisy input, pointing out the possibility of turning off-the-shelf generation techniques into tools for content shift suppression. We further propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology. Despite the simplicity, the proposed approach has achieved superior results on various tasks and datasets, validating its potential as a generic booster for diffusion features. Our code is available at this url. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Towards Better Graph Representation Learning with Parameterized Decomposition & Filtering

arXiv

引用

arXiv 2023年

作者： Yang, Mingqi Feng, Wenjie Shen, Yanming Hooi, Bryan Dalian University of Technology China Institute of Data Science National University of Singapore Singapore School of Computing National University of Singapore Singapore

Proposing an effective and flexible matrix to represent a graph is a fundamental challenge that has been explored from multiple perspectives, e.g., filtering in Graph Fourier Transforms. In this work, we develop a novel and general framework which unifies many existing GNN models from the view of parameterized decomposition and filtering, and show how it helps to enhance the flexibility of GNNs while alleviating the smoothness and amplification issues of existing models. Essentially, we show that the extensively studied spectral graph convolutions with learnable polynomial filters are constrained variants of this formulation, and releasing these constraints enables our model to express the desired decomposition and filtering simultaneously. Based on this generalized framework, we develop models that are simple in implementation but achieve significant improvements and computational efficiency on a variety of graph learning tasks. Code is available at https://***/qslim/PDF. Copyright © 2023, The Authors. All rights reserved.

关键词： Fourier transforms

来源：评论

学校读者我要写书评

暂无评论

A fully coupled regularized mortar-type finite element approach for embedding one-dimensional fibers into three-dimensional fluid flow

arXiv

引用

arXiv 2023年

作者： Hagmeyer, Nora Mayr, Matthias Popp, Alexander Institute for Mathematics and Computer-Based Simulation University of the Bundeswehr Munich Germany Data Science & Computing Lab University of the Bundeswehr Munich Germany

[Summary]The present article proposes a partitioned Dirichlet-Neumann algorithm, that allows to address unique challenges arising from a novel mixed-dimensional coupling of very slender fibers embedded in fluid flow using a regularized mortar-type finite element discretization. The fibers are modeled via one-dimensional (1D) partial differential equations based on geometrically exact nonlinear beam theory, while the flow is described by the three-dimensional (3D) incompressible Navier-Stokes equations. The arising truly mixed-dimensional 1D-3D coupling scheme constitutes a novel approximate model and numerical strategy, that naturally necessitates specifically tailored solution schemes to ensure an accurate and efficient computational treatment. In particular, we present a strongly coupled partitioned solution algorithm based on a Quasi-Newton method for applications involving fibers with high slenderness ratios that usually present a challenge with regard to the well-known added mass effect. The influence of all employed algorithmic and numerical parameters, namely the applied acceleration technique, the employed constraint regularization parameter as well as shape functions, on efficiency and results of the solution procedure is studied through appropriate examples. Finally, the convergence of the two-way coupled mixed-dimensional problem solution under uniform mesh refinement is demonstrated, a comparison to a 3D reference solution is performed, and the method’s capabilities in capturing flow phenomena at large geometric scale separation is illustrated by the example of a submersed vegetation canopy. © 2023, CC BY-SA.

关键词： Fibers

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：