检索结果-内蒙古大学图书馆

MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cui, Wanqing Bi, Keping Guo, Jiafeng Cheng, Xueqi CAS Key Lab of Network Data Science and Technology Institute of Computing Technology Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China

Since commonsense information has been recorded significantly less frequently than its existence, language models pre-trained by text generation have difficulty to learn sufficient commonsense knowledge. Several studies have leveraged text retrieval to augment the models' commonsense ability. Unlike text, images capture commonsense information inherently but little effort has been paid to effectively utilize them. In this work, we propose a novel Multi-mOdal REtrieval (MORE) augmentation framework, to leverage both text and images to enhance the commonsense ability of language models. Extensive experiments on the Common-Gen task have demonstrated the efficacy of MORE based on the pre-trained models of both single and multiple modalities. © 2024, CC BY-NC-SA.

关键词： Computational linguistics

On the physics of nested Markov models: a generalized probabilistic theory perspective

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Xingjian Wang, Yuhao QICI Quantum Information and Computation Initiative School of Computing and Data Science University of Hong Kong Hong Kong Institute for Interdisciplinary Information Sciences Tsinghua University China

Determining potential probability distributions with a given causal graph is vital for causality studies. To bypass the difficulty in characterizing latent variables in a Bayesian network, the nested Markov model provides an elegant algebraic approach by listing exactly all the equality constraints on the observed variables. However, this algebraically motivated causal model comprises distributions outside Bayesian networks, and its physical interpretation remains vague. In this work, we inspect the nested Markov model through the lens of generalized probabilistic theory, an axiomatic framework to describe general physical theories. We prove that all the equality constraints defining the nested Markov model hold valid theory-independently. Yet, we show this model generally contains distributions not implementable even within such relaxed physical theories subjected to merely the relativity principles and mild probabilistic rules. To interpret the origin of such a gap, we establish a new causal model that defines valid distributions as projected from a high-dimensional Bell-type causal structure. The new model unveils inequality constraints induced by relativity principles, or equivalently high-dimensional conditional independences, which are absent in the nested Markov model. Nevertheless, we also notice that the restrictions on states and measurements introduced by the generalized probabilistic theory framework can pose additional inequality constraints beyond the new causal model. As a by-product, we discover a new causal structure exhibiting strict gaps between the distribution sets of a Bayesian network, generalized probabilistic theories, and the nested Markov model. We anticipate our results will enlighten further explorations on the unification of algebraic and physical perspectives of causality. © 2024, CC BY.

关键词： Bayesian networks

GoPIM: GCN-Oriented Pipeline Optimization for PIM Accelerators

学校读者我要写书评

暂无评论

GoPIM: GCN-Oriented Pipeline Optimization for PIM Accelerato...

IEEE Symposium on High-Performance Computer Architecture

作者： Siling Yang Shuibing He Wenjiong Wang Yanlong Yin Tong Wu Weijian Chen Xuechen Zhang Xian-He Sun Dan Feng The State Key Laboratory of Blockchain and Data Security Zhejiang University Zhejiang Lab Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security Zhejiang Key Laboratory of Big Data Intelligent Computing Washington State University Vancouver Illinois Institute of Technology Huazhong University of Science and Technology Wuhan National Laboratory for Optoelectronics

ISBN: (数字)9798331506476

ISBN: (纸本)9798331506483

Graph convolutional networks (GCNs) are popular for a variety of graph learning tasks. ReRAM-based processing-in-memory (PIM) accelerators are promising to expedite GCN training owing to their in-situ computing capability. However, existing accelerators can be severely underutilized even with pipelines, due to the oversight of the skewed execution times of various GCN stages and the ignorance of skewed degrees of graph vertices. In this work, we propose GOPIM, a GCN-oriented pipeline optimization for PIM accelerators to expedite GCN training. First, GOPIM proposes an ML-based scheme that allocates crossbar resources to the most needed stages to streamline the overall pipeline. Second, GOPIM utilizes a selective vertex updating technique that evenly distributes vertices on crossbars by interleaved mapping. These techniques collectively reduce the overall execution time without losing much accuracy. We also provide a practical architecture design for GOPIM. Our experimental results show that, GoPIM achieves up to 191 × speedup and 16.1 × energy saving, compared to the state-of-the-art work.

关键词： Training Accuracy Graph convolutional networks Pipelines Energy conservation Computer architecture Machine learning Parallel processing Optimization

SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model

学校读者我要写书评

暂无评论

IEEE Journal on Selected Topics in Signal Processing 2025年

作者： Qian, Xinyuan Gao, Jiaran Zhang, Yaodan Zhang, Qiquan Liu, Hexin Garcia, Leibny Paola Li, Haizhou University of Science and Technology Beijing School of Computer and Communication Engineering Beijing100083 China University of New South Wales School of Electrical Engineering and Telecommunications Sydney2052 Australia Nanyang Technological University College of Computing and Data Science Singapore Singapore Johns Hopkins University CLSP and HLT-COE United States Chinese University of HongKong Guangdong Provincial Key Laboratory of Big Data Computing Shenzhen518172 China Shenzhen Research Institute of Big data Shenzhen51872 China

Speech enhancement plays an essential role in various applications, and the integration of visual information has been demonstrated to bring substantial advantages. However, the majority of current research concentrates on the examination of facial and lip movements, which can be compromised or entirely inaccessible in scenarios where occlusions occur or when the camera view is distant. Whereas contextual visual cues from the surrounding environment have been overlooked: for example, when we see a dog bark, our brain has the innate ability to discern and filter out the barking noise. To this end, in this paper, we introduce a novel task, i.e. Scene-aware Audio-Visual Speech Enhancement SAV-SE. To our best knowledge, this is the first proposal to use rich contextual information from synchronized video as auxiliary cues to indicate the type of noise, which eventually improves the speech enhancement performance. Specifically, we propose the VC-S $^{2}$ E method, which incorporates the Conformer and Mamba modules for their complementary strengths. Extensive experiments are conducted on public MUSIC, AVSpeech and AudioSet datasets, where the results demonstrate the superiority of VC-S $^{2}$ E over other competitive methods. We will make the source code publicly available. Project demo page: https://***/ © 2007-2012 IEEE.

关键词： Speech enhancement

Improving VTE Identification through Language Models from Radiology Reports: A Comparative Study of Mamba, Phi-3 Mini, and BERT

学校读者我要写书评

暂无评论

Improving VTE Identification through Language Models from Ra...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Jamie Deng Yusen Wu Yelena Yesha Phuong Nguyen Department of Computer Science University of Miami Miami FL USA Frost Institute for Data Science & Computing University of Miami Miami FL USA Department of Radiology University of Miami Miami FL USA

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

Venous thromboembolism (VTE) is a critical cardio-vascular condition, encompassing deep vein thrombosis (DVT) and pulmonary embolism (PE). Accurate and timely identification of VTE is essential for effective medical care. This study builds upon our previous work, which addressed VTE detection using deep learning methods for DVT and a hybrid approach combining deep learning and rule-based classification for PE. Our earlier approaches, while effective, had two major limitations: they were complex and required expert involvement for feature engineering of the rule set. To overcome these challenges, we utilize the Mamba architecture-based classifier. This model achieves remarkable results, with a 97% accuracy and F1 score on the DVT dataset and a 98% accuracy and F1 score on the PE dataset. In contrast to the previous hybrid method on PE identification, the Mamba classifier eliminates the need for hand-engineered rules, significantly reducing model complexity while maintaining comparable performance. Additionally, we evaluated a lightweight Large Language Model (LLM), Phi-3 Mini, in detecting VTE. While this model delivers competitive results, outperforming the baseline BERT models, it proves to be computationally intensive due to its larger parameter set. Our evaluation shows that the Mamba-based model demonstrates superior performance and efficiency in VTE identification, offering an effective solution to the limitations of previous approaches.

关键词： Deep learning Training Accuracy Computational modeling Biological system modeling Computer architecture Feature extraction Complexity theory Thrombosis Biomedical imaging

FERI: Feature Enhancement and Relational Interaction for Image-text Matching 30

学校读者我要写书评

暂无评论

FERI: Feature Enhancement and Relational Interaction for Ima...

30th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2024

作者： Zhang, Yu Zhang, Jianqiang Song, Gongpeng Lu, Qin Zhao, Shuo Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center Jinan China Shandong Engineering Research Center of Big Data Applied Technology Faculty of Computer Science and Technology Jinan China Shandong Fundamental Research Center for Computer Science Shandong Provincial Key Laboratory of Computer Networks Jinan China Shandong Branch of China Mobile Communication Group Design Institute Co. Jinan China

ISBN: (纸本)9798331515966

Image-text matching is an important problem at the intersection of computer vision and natural language processing. It aims to establish the semantic link between image and text to achieve high-quality semantic alignment between the two modalities. However, the existing methods have the problem that the meaning expressed in the image or the complex narrative in the text cannot be fully understood due to insufficient feature extraction. Moreover, due to the essential modal differences between images and texts, how to effectively and accurately align the semantic contents in images and texts has become the key of research. In order to solve the above problems, this paper proposes a method based on feature enhancement and relationship interaction. When processing images, the proposed method fuses labeled features, region features and location features to represent images. When processing text, a combination of Bi-GRU and self-attention mechanism is used to represent the text. In order to further align the semantic content in images and texts accurately, this paper improves two relational interaction mechanisms by identifying connection relationships and learning association relationships. Thus, the relation enhanced embedding is obtained. Finally, it calculated the similarity of the enhanced embedding to judge the matching degree of the image and text. Extensive experiments on the public datasets Flickr30K and MSCOCO demonstrate the effectiveness of our method. © 2024 IEEE.

关键词： cross-modal retrieval feature enhancement image-text matching relationship interaction

Improving VTE Identification through Adaptive NLP Model Selection and Clinical Expert Rule-based Classifier from Radiology Reports

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Deng, Jamie Wu, Yusen Hayssen, Hilary Englum, Brain Kankaria, Aman Mayorga-Carlin, Minerva Sahoo, Shalini Sorkin, John Lal, Brajesh Yesha, Yelena Nguyen, Phuong Frost Institute for Data Science & Computing University of Miami FL United States School of Medicine University of Maryland MD United States

Rapid and accurate identification of Venous thromboembolism (VTE), a severe cardiovascular condition including deep vein thrombosis (DVT) and pulmonary embolism (PE), is important for effective treatment. Leveraging Natural Language Processing (NLP) on radiology reports, automated methods have shown promising advancements in identifying VTE events from retrospective data cohorts or aiding clinical experts in identifying VTE events from radiology reports. However, effectively training Deep Learning (DL) and the NLP models is challenging due to limited labeled medical text data, the complexity and heterogeneity of radiology reports, and data imbalance. This study proposes novel method combinations of DL methods, along with data augmentation, adaptive pre-trained NLP model selection, and a clinical expert NLP rule-based classifier, to improve the accuracy of VTE identification in unstructured (freetext) radiology reports. Our experimental results demonstrate the model’s efficacy, achieving an impressive 97% accuracy and 97% F1 score in predicting DVT, and an outstanding 98.3% accuracy and 98.4% F1 score in predicting PE. These findings emphasize the model’s robustness and its potential to significantly contribute to VTE research. © 2023, CC BY-NC-ND.

关键词： Natural language processing systems

S2AC: ENERGY-BASED REINFORCEMENT LEARNING WITH STEIN SOFT ACTOR CRITIC

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Messaoud, Safa Mokeddem, Billel Xue, Zhenghai Pang, Linsey An, Bo Chen, Haipeng Chawla, Sanjay Qatar Computing Research Institute Hamad Bin Khalifa University Qatar School of Computer Science and Engineering Nanyang Technological University Singapore SalesForce Skywork AI Singapore Data Science William & Mary

Learning expressive stochastic policies instead of deterministic ones has been proposed to achieve better stability, sample complexity, and robustness. Notably, in Maximum Entropy Reinforcement Learning (MaxEnt RL), the policy is modeled as an expressive Energy-Based Model (EBM) over the Q-values. However, this formulation requires the estimation of the entropy of such EBMs, which is an open problem. To address this, previous MaxEnt RL methods either implicitly estimate the entropy, resulting in high computational complexity and variance (SQL), or follow a variational inference procedure that fits simplified actor distributions (e.g., Gaussian) for tractability (SAC). We propose Stein Soft Actor-Critic (S2AC), a MaxEnt RL algorithm that learns expressive policies without compromising efficiency. Specifically, S2AC uses parameterized Stein Variational Gradient Descent (SVGD) as the underlying policy. We derive a closed-form expression of the entropy of such policies. Our formula is computationally efficient and only depends on first-order derivatives and vector products. Empirical results show that S2AC yields more optimal solutions to the MaxEnt objective than SQL and SAC in the multi-goal environment, and outperforms SAC and SQL on the MuJoCo benchmark. © 2024, CC BY.

关键词： Reinforcement learning

Leveraging SDN: Empirical Evaluation of Privacy Enriched data Communication over VANET Using Software Defined Network Assistance 9

学校读者我要写书评

暂无评论

Leveraging SDN: Empirical Evaluation of Privacy Enriched Dat...

9th International Conference on science, Technology, Engineering and Mathematics, ICONSTEM 2024

作者： Ratheesh, R. Vijayalakshmi, S. Arunsundar, B. Swarna, M. Saranya, G. Agni College of Technology Department of Ece Chennai India R.M.K. Engineering College Department of Ece Chennai India School of Computing College of Engineering And Technology Srm Institute of Science And Technology Department of Data Science And Business Systems Kattankulathur India Dr. M.G.R. Educational And Research Institute University Department of Ece Chennai India Sri Krishna College of Engineering And Technology Department of Electronics And Communication Engineering Coimbatore Kuniyamuthur India

ISBN: (纸本)9798350365092

In the realm of modern transportation systems, Vehicular Ad Hoc Networks (VANETs) play a pivotal role in enabling efficient communication among vehicles and infrastructure elements for enhanced road safety and traffic management. However, the inherent vulnerabilities in VANETs, such as privacy breaches and security threats, necessitate robust frameworks to mitigate risks and ensure secure data transmission. This paper presents a comprehensive architecture for Privacy-Enriched data Communication over VANETs using Software-Defined Network (SDN) assistance. The proposed framework integrates key components including vehicles, Roadside Units (RSUs), and SDN controllers to facilitate secure and privacy-aware communication. Vehicles act as data sources, employing pseudonyms and encryption mechanisms to protect user privacy. RSUs serve as access points, collecting and relaying data while enforcing access control policies. SDN controllers provide centralized management, dynamically allocating resources and enforcing privacy policies through encryption, pseudonymization, and access control mechanisms. Communication protocols such as IEEE 802.11p and LTE enable vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication, while the OpenFlow protocol facilitates communication between SDN controllers and network devices. Additionally, trust management mechanisms evaluate the trustworthiness of network entities based on behavioral analysis, reliability metrics, and security considerations, ensuring the integrity and resilience of the network. By amalgamating SDN-based management with privacy enhancement mechanisms and trust management protocols, the proposed framework offers a robust solution for addressing privacy and security challenges in VANETs, thereby fostering safer and more secure transportation ecosystems. © 2024 IEEE.

关键词： Vehicular ad hoc networks