检索结果-内蒙古大学图书馆

ChemDFM-X: towards large multimodal model for chemistry

science China(Information sciences) 2024年第12期67卷 99-100页

作者： Zihan ZHAO Bo CHEN Jingpiao LI Lu CHEN Liyang WEN Pengyu WANG Zichen ZHU Danyang ZHANG Yansi LI Zhongyang Dai Xin CHEN Kai YU X-LANCE Lab Department of Computer Science and EngineeringMoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Suzhou Laboratory

Chemistry, as a naturally multimodal discipline, plays a crucial role in various vital fields such as pharmaceutical research and material manufacturing. Therefore, research on artificial intelligence(ai) for chemistry has garnered increasing attention. Despite the rapid development, most of the chemical ai models today mainly focus on single tasks with unimodal input [1].

关键词：

来源：评论

学校读者我要写书评

暂无评论

Towards Practical Edge Inference Attacks Against Graph Neural Networks 48

Towards Practical Edge Inference Attacks Against Graph Neura...

引用

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Li, Kailai Sun, Jiawei Chen, Ruoxin Ding, Wei Yu, Kexue Li, Jie Wu, Chentao Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence AI Institute Department of Computer Science and Engineering China

ISBN: (纸本)9781728163277

Graph Neural Networks (GNNs) have demonstrated superior performance in numerous real-world applications. Despite their success, recent studies have shown that GNNs are vulnerable under edge inference attacks aimed to infer the connectivity of a given pair of nodes. However, existing methods primarily focus on the scenario when properties of target nodes are revealed. In this paper, we propose an edge inference attack in a more realistic and practical setting. In our threat model, the adversary cannot obtain properties of target nodes but can inject a single probing node and query the target GNN for its prediction. By connecting the probing and target nodes, the adversary can infer the connectivity of the target node pair based on the prediction of the probing node. Extensive experiments show that our attack performs comparably to ones that require properties of target nodes. And when given such auxiliary knowledge, our attack outperforms state-of-the-art methods. © 2023 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding

Fast and High-Quality Auto-Regressive Speech Synthesis via S...

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Li, Bohan Wang, Hankun Zhang, Situo Guo, Yiwei Yu, Kai MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China

ISBN: (纸本)9798350368741

The auto-regressive (AR) architecture, exemplified by models such as GPT, is extensively utilized in modern Text-to-Speech (TTS) systems. However, it often leads to considerable inference delays, primarily due to the challenges associated with next-token prediction in long speech sequences. In this work, we introduce VADUSA, one of the first approaches to accelerate AR-based TTS through speculative decoding. Our findings demonstrate that VADUSA not only delivers a significant reduction in inference time but also enhances TTS quality by employing draft heads to predict future speech tokens in an auto-regressive manner. Additionally, the incorporation of a tolerance mechanism during the sampling process further boosts performance, yielding approximately a 3× speedup in AR TTS. Moreover, our approach exhibits strong generalization across diverse datasets and various speech token types. © 2025 IEEE.

关键词： inference speedup speculative decoding text-to-speech

来源：评论

学校读者我要写书评

暂无评论

ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation

ROME: Robustifying Memory-Efficient NAS via Topology Disenta...

引用

International Conference on computer Vision (ICCV)

作者： Xiaoxing Wang Xiangxiang Chu Yuda Fan Zhexi Zhang Bo Zhang Xiaokang Yang Junchi Yan Dep. of Computer Science and Engineering & MoE Key Lab of AI Shanghai Jiao Tong University Meituan

Albeit being a prevalent architecture searching approach, differentiable architecture search (DARTS) is largely hindered by its substantial memory cost since the entire supernet resides in the memory. This is where the single-path DARTS comes in, which only chooses a single-path submodel at each step. While being memory-friendly, it also comes with low computational costs. Nonetheless, we discover a critical issue of single-path DARTS that has not been primarily noticed. Namely, it also suffers from severe performance collapse since too many parameter-free operations like skip connections are derived, just like DARTS does. In this paper, we propose a new algorithm called RObustifying Memory-Efficient NAS (ROME) to give a cure. First, we disentangle the topology search from the operation search to make searching and evaluation consistent. We then adopt Gumbel-Top2 reparameterization and gradient accumulation to robustify the unwieldy bi-level optimization. We verify ROME extensively across 15 benchmarks to demonstrate its effectiveness and robustness.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Leveraging Hallucinations to Reduce Manual Prompt dep.ndency in Promptable Segmentation 38

Leveraging Hallucinations to Reduce Manual Prompt Dependency...

引用

38th Conference on Neural Information Processing Systems, NeurIPS 2024

作者： Hu, Jian Lin, Jiayi Yan, Junchi Gong, Shaogang School of Electronic Engineering and Computer Science Queen Mary University of London United Kingdom Dept. of CSE School of AI Moe Key Lab of AI Shanghai Jiao Tong University China

Promptable segmentation typically requires instance-specific manual prompts to guide the segmentation of each desired object. To minimize such a need, task-generic promptable segmentation has been introduced, which employs a single task-generic prompt to segment various images of different objects in the same task. Current methods use Multimodal Large Language Models (MLLMs) to reason detailed instance-specific prompts from a task-generic prompt for improving segmentation accuracy. The effectiveness of this segmentation heavily dep.nds on the precision of these derived prompts. However, MLLMs often suffer hallucinations during reasoning, resulting in inaccurate prompting. While existing methods focus on eliminating hallucinations to improve a model, we argue that MLLM hallucinations can reveal valuable contextual insights when leveraged correctly, as they represent pre-trained large-scale knowledge beyond individual images. In this paper, we utilize hallucinations to mine task-related information from images and verify its accuracy for enhancing precision of the generated prompts. Specifically, we introduce an iterative Prompt-Mask Cycle generation framework (ProMaC) with a prompt generator and a mask generator. The prompt generator uses a multi-scale chain of thought prompting, initially exploring hallucinations for extracting extended contextual knowledge on a test image. These hallucinations are then reduced to formulate precise instance-specific prompts, directing the mask generator to produce masks that are consistent with task semantics by mask semantic alignment. The generated masks iteratively induce the prompt generator to focus more on task-relevant image areas and reduce irrelevant hallucinations, resulting jointly in better prompts and masks. Experiments on 5 benchmarks demonstrate the effectiveness of ProMaC. Code given in https://***/ProMaC/. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Attack Named Entity Recognition by Entity Boundary Interference 30

Attack Named Entity Recognition by Entity Boundary Interfere...

引用

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Yang, Yifei Wu, Hongqiu Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

ISBN: (纸本)9782493814104

Named Entity Recognition (NER) is a cornerstone natural language processing task while its robustness has been given little attention. This paper rethinks the principles of the conventional text attack, as they can easily violate the label consistency between the original and adversarial NER samples. This is due to the fine-grained nature of NER, as even minor word changes in the sentence can result in the emergence or mutation of any entity, producing invalid adversarial samples. To this end, we propose a novel one-word modification NER attack based on a key insight, NER models are always vulnerable to the boundary position of an entity to make their decision. We thus strategically insert a new boundary into the sentence and trigger the victim model to make a wrong recognition either on this boundary word or on other words in the sentence. We call this attack Virtual Boundary Attack (ViBA), which is shown to be remarkably effective when attacking both English and Chinese models with a 70%-90% attack success rate on state-of-the-art language models, and also significantly faster than previous methods. We share the code in https://***/yangyifei729/ViBA. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

Exploiting Persistent CPU Cache for Scalable Persistent Hash Index 40

Exploiting Persistent CPU Cache for Scalable Persistent Hash...

引用

40th IEEE International Conference on Data engineering, ICDE 2024

作者： Zhang, Bowen Zheng, Shengan Nie, Liangxu Qi, Zhenlin Huang, Linpeng Mei, Hong Shanghai Jiao Tong University Department of Computer Science and Engineering China AI Institute Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence China

ISBN: (纸本)9798350317152

Byte-addressable persistent memory (PM) has been widely studied in the past few years. Recently, the emerging eADR technology further incorporates CPU cache into the persistence domain. The persistent CPU cache is promising to optimize the write performance of PM-based storage systems and facilitate the design of concurrent crash-consistent data structures. In this paper, we propose Spash, a highly scalable persistent hash index for PM systems with persistent CPU cache. Spash fully exploits the benefits of persistent CPU cache to implement a durable linearizable index with low PM access overhead and high concurrency. Spash employs a fine-grained extendible hash architecture and a metadata-free segment design to minimize the number of PM accesses. Moreover, Spash adopts adaptive in-place updates and compacted-flush insertions, which dramatically conserve scarce PM write bandwidth by absorbing a large amount of PM write in the persistent CPU cache. Furthermore, Spash proposes a two-phase concurrency protocol and a collaborative staged doubling mechanism, which leverage the persistent CPU cache and hardware transactional memory to achieve lock-free concurrency and durable linearizability. Spash outperforms the other state-of-the-art persistent hash indexes in YCSB workloads by up to 19.6×. © 2024 IEEE.

关键词： Concurrency control

来源：评论

学校读者我要写书评

暂无评论

Efficient Text-Only Domain Adaptation For CTC-Based ASR

Efficient Text-Only Domain Adaptation For CTC-Based ASR

引用

2023 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2023

作者： Chen, Chang Gong, Xun Qian, Yanmin Ai Institute Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence Department of Computer Science and Engineering Shanghai China

ISBN: (纸本)9798350306897

For connectionist temporal classification (CTC) based speech recognition (ASR) models, text-only domain adaptation still faces several challenges. In this study, we propose an efficient text-only domain adaptation method for CTC-based models. We introduce the assistant textual adapter (ATA) to learn textual features and transform them into the latent space of the acoustic encoder. With the help of the ATA module, the adaptation is achieved by fine-tuning the top layers of the acoustic encoder with the target domain text. Meanwhile, further improvement can be obtained by the integration with shallow fusion (SF). Adapted from LibriSpeech, experiments show that the proposed method can achieve averaged 29.7% relative WER reduction (WERR) compared with the un-adapted baseline on WSJ, and 10.5% WERR compared to SF as well. Moreover, it also shows 15.4∼37.1% WERR for 10 GigaSpeech target domains test sets compared to the un-adapted baseline, and also 6.5% WERR on average compared with SF. © 2023 IEEE.

关键词： Speech recognition

来源：评论

学校读者我要写书评

暂无评论

Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models 31

Converging to a Lingua Franca: Evolution of Linguistic Regio...

引用

31st International Conference on Computational Linguistics, COLING 2025

作者： Zeng, Hongchuan Han, Senyu Chen, Lu Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China Suzhou Laboratory Suzhou China

ISBN: (纸本)9798891761964

Large language models (LLMs) have demonstrated remarkable performance, particularly in multilingual contexts. While recent studies suggest that LLMs can transfer skills learned in one language to others, the internal mechanisms behind this ability remain unclear. We observed that the neuron activation patterns of LLMs exhibit similarities when processing the same language, revealing the existence and location of key linguistic regions. Additionally, we found that neuron activation patterns are similar when processing sentences with the same semantic meaning in different languages. This indicates that LLMs map semantically identical inputs from different languages into a "Lingua Franca", a common semantic latent space that allows for consistent processing across languages. This semantic alignment becomes more pronounced with training and increased model size, resulting in a more language-agnostic activation pattern. Moreover, we found that key linguistic neurons are concentrated in the first and last layers of LLMs, becoming denser in the first layers as training progresses. Experiments on BLOOM and LLaMA2 support these findings, highlighting the structural evolution of multilingual LLMs during training and scaling up. This paper provides insights into the internal workings of LLMs, offering a foundation for future improvements in their cross-lingual capabilities. The codes are available at: https://***/X-LANCE/LinguaFranca. © 2025 Association for Computational Linguistics.

关键词： Neurons

来源：评论

学校读者我要写书评

暂无评论

FAT-HuBERT: Front-End Adaptive Training of Hidden-Unit BERT For Distortion-Invariant Robust Speech Recognition

FAT-HuBERT: Front-End Adaptive Training of Hidden-Unit BERT ...

引用

2023 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2023

作者： Yang, Dongning Wang, Wei Qian, Yanmin Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence Ai Institute Department of Computer Science and Engineering Shanghai China

ISBN: (纸本)9798350306897

Advancements in monaural speech enhancement (SE) techniques have greatly improved the perceptual quality of speech. However, integrating these techniques into automatic speech recognition (ASR) systems has not yielded the expected performance gains, primarily due to the introduction of distortions during the SE process. In this paper, we propose a novel approach called FAT-HuBERT, which leverages distortion-invariant self-supervised learning (SSL) to enhance the robustness of ASR. To address the distortions introduced by the SE frontends, we introduce layer-wise fusion modules that incorporate features extracted from both observed noisy signals and enhanced signals. During training, the SE frontend is randomly selected from a pool of models. We evaluate the performance of FAT-HuBERT on simulated noisy speech generated from LIBRISPEECH as well as real-world noisy speech from the CHIME-4 1-channel dataset. The experimental results demonstrate a significant relative reduction in word error rate (WER). © 2023 IEEE.

关键词： Speech enhancement

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：