检索结果-内蒙古大学图书馆

Generative Biomedical Event Extraction With constrained decoding Strategy

IEEE-ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2024年第6期21卷 2471-2484页

作者： Su, Fangfang Teng, Chong Li, Fei Li, Bobo Zhou, Jun Ji, Donghong Hangzhou Dianzi Univ Sch Cyberspace Hangzhou 310018 Zhejiang Peoples R China Wuhan Univ Sch Cyber Sci & Engn Key Lab Aerosp Informat Secur & Trusted Comp Minist Educ Wuhan 430072 Peoples R China

Currently, biomedical event extraction has received considerable attention in various fields, including natural language processing, bioinformatics, and computational biomedicine. This has led to the emergence of numerous machine learning and deep learning models that have been proposed and applied to tackle this complex task. While existing models typically adopt an extraction-based approach, which requires breaking down the extraction of biomedical events into multiple subtasks for sequential processing, making it prone to cascading errors. This paper presents a novel approach by constructing a biomedical event generation model based on the framework of the pre-trained language model T5. We employ a sequence-to-sequence generation paradigm to obtain events, the model utilizes constrained decoding algorithm to guide sequence generation, and a curriculum learning algorithm for efficient model learning. To demonstrate the effectiveness of our model, we evaluate it on two public benchmark datasets, Genia 2011 and Genia 2013. Our model achieves superior performance, illustrating the effectiveness of generative modeling of biomedical events.

关键词： Biological system modeling decoding Regulation Computational modeling Semantics Vectors Transformers Proteins Protein engineering Adaptation models Biomedical event extraction constrained decoding generative models

来源：评论

学校读者我要写书评

暂无评论

Generative Sentiment Analysis via Latent Category Distribution and constrained decoding 33rd

Generative Sentiment Analysis via Latent Category Distributi...

引用

33rd International Conference on Artificial Neural Networks and Machine Learning (ICANN)

作者： Zhou, Jun Yu, Dongyang Aziz, Kamran Su, Fangfang Zhang, Qing Li, Fei Ji, Donghong Wuhan Univ Sch Cyber Sci & Engn Key Lab Aerosp Informat Secur & Trusted Comp Minist Educ Wuhan Peoples R China Beijing Rigour Technol Co Ltd Beijing Peoples R China Hangzhou Dianzi Univ Sch Cyberspace Hangzhou Peoples R China North China Power Engn Co Ltd China Power Engn Consulting Grp Beijing Peoples R China

ISBN: (纸本)9783031723490;9783031723506

Fine-grained sentiment analysis involves extracting and organizing sentiment elements from textual data. However, existing approaches often overlook issues of category semantic inclusion and overlap, as well as inherent structural patterns within the target sequence. This study introduces a generative sentiment analysis model. To address the challenges related to category semantic inclusion and overlap, a latent category distribution variable is introduced. By reconstructing the input of a variational autoencoder, the model learns the intensity of the relationship between categories and text, thereby improving sequence generation. Additionally, a trie data structure and constrained decoding strategy are utilized to exploit structural patterns, which in turn reduces the search space and regularizes the generation process. Experimental results on the Restaurant-ACOS and Laptop-ACOS datasets demonstrate a significant performance improvement compared to baseline models. Ablation experiments further confirm the effectiveness of latent category distribution and constrained decoding strategy.

关键词： Sentiment analysis Text generation Latent distribution constrained decoding

来源：评论

学校读者我要写书评

暂无评论

Aligning Documentation and Q&A Forum through constrained decoding with Weak Supervision 39

Aligning Documentation and Q&A Forum through Constrained Dec...

引用

39th IEEE International Conference on Software Maintenance and Evolution (ICSME)

作者： Pudari, Rohith Zhou, Shiyuan Ahmed, Iftekhar Dai, Zhuyun Zhou, Shurui Univ Toronto Toronto ON Canada Univ Calif Irvine Irvine CA 92717 USA Google Mountain View CA 94043 USA

ISBN: (纸本)9798350327830

Stack Overflow (SO) is a widely used question-andanswer (Q&A) forum dedicated to software development. It plays a supplementary role to official documentation (DOC for short) by offering practical examples and resolving uncertainties. However, the process of simultaneously consulting both the documentation and SO posts can be challenging and time-consuming due to their disconnected nature. In this study, we propose DOSA, a novel approach to automatically align SO and DOC, which inject domain-specific knowledge about the DOC structure into large language models (LLMs) through weak supervision and constrained decoding, thereby enhancing knowledge retrieval and streamlining task completion during the software development procedure. Our preliminary experiments find that DOSA outperforms various widely-used baselines, showing the promise of using generative retrieval models to perform low-resource software engineering tasks.

关键词： Stack Overflow Natural language processing constrained decoding Weak Supervision

来源：评论

学校读者我要写书评

暂无评论

Improving Phonetic Transcriptions of Children's Speech by Pronunciation Modelling with constrained CTC-decoding 23

Improving Phonetic Transcriptions of Children's Speech by Pr...

引用

Interspeech Conference

作者： Rumberg, Lars Gebauer, Christopher Ehlert, Hanna Luedtke, Ulrike Ostermann, Joern Leibniz Univ Hannover Inst Informat Verarbeitung L3S Hannover Germany Leibniz Univ Hannover Inst Sonderpadag Hannover Germany

Language sample analysis (LSA) is a powerful tool for both therapeutic applications and research of child speech and language development. Nevertheless, it is not routinely used, due to the high cost of manual transcription and analysis. Assistance by automatic speech recognition for children has the potential to enable a wide-spread use of LSA. However, the development of modern speech recognition systems heavily relies on large scale datasets. Therefore, it faces the same obstacle of high cost for transcription as LSA itself. In this paper, we study how cheaply transcribed child speech, i. e., limited to an orthographic transcription, can be improved on a phonetic level by leveraging a CTC based automatic speech recognition model, trained on a small phonetically transcribed dataset. We constrain the CTC decoding by modeling variation of the pronunciation given the orthographic transcription using weighted finite state automata. Our experiments show that the transcription is improved in terms of phone error rate by relative 14% when applying our method. Additionally, we show how the improved transcript can in turn be leveraged to improve the training of a new model.

关键词： speech recognition constrained decoding child speech finite state automaton

来源：评论

学校读者我要写书评

暂无评论

ReZG: Retrieval-augmented zero-shot counter narrative generation for hate speech

引用

NEUROCOMPUTING 2025年 620卷

作者： Jiang, Shuyu Tang, Wenyi Chen, Xingshu Tang, Rui Wang, Haizhou Wang, Wenxian Sichuan Univ Sch Cyber Sci & Engn Chengdu 610065 Peoples R China Sichuan Univ Key Lab Data Protect & Intelligent Management Minist Educ Chengdu 610065 Peoples R China Sichuan Univ Cyber Sci Res Inst Chengdu 610065 Peoples R China

The proliferation of hate speech (HS) on social media poses a serious threat to societal security. Automatic counter narrative (CN) generation, as an active strategy for HS intervention, has garnered increasing attention in recent years. Existing methods for automatically generating CNs mainly rely on re-training or fine-tuning pre-trained language models (PLMs) on human-curated CN corpora. Unfortunately, the annotation speed of CN corpora cannot keep up with the growth of HS targets, while generating specific and effective CNs for unseen targets remains a significant challenge for the model. To tackle this issue, we propose Retrieval-Augmented Zero-shot Generation (ReZG) to generate CNs with high-specificity for unseen targets. Specifically, we propose a multi-dimensional hierarchical retrieval method that integrates stance, semantics, and fitness, extending the retrieval metric from single dimension to multiple dimensions suitable for the knowledge that refutes HS. Then, we implement an energy-based constrained decoding mechanism that enables PLMs to use differentiable knowledge preservation, countering, and fluency constraint functions instead of in-target CNs as control signals for generation, thereby achieving zero-shot CN generation. With the above techniques, ReZG can integrate external knowledge flexibly and improve the specificity of CNs. Experimental results show that ReZG exhibits stronger generalization capabilities and outperforms strong baselines with significant improvements of 2.0%+ in the relevance and 4.5%+ in the countering success rate metrics.

关键词： constrained text generation Dialog constrained decoding Hate speech Pre-trained language model Retrieval augmentation

来源：评论

学校读者我要写书评

暂无评论

UniSAr: a unified structure-aware autoregressive language model for text-to-SQL semantic parsing

引用

INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS 2023年第12期14卷 4361-4376页

作者： Dou, Longxu Gao, Yan Pan, Mingyang Wang, Dingzirui Che, Wanxiang Lou, Jian-Guang Zhan, Dechen Harbin Inst Technol Harbin 150001 Heilongjiang Peoples R China Microsoft Res Asia Beijing 100089 Peoples R China

Existing text-to-SQL semantic parsers are typically designed for particular settings such as handling queries that span multiple tables, domains, or turns which makes them ineffective when applied to different settings. We present UniSAr (Unified Structure-Aware Autoregressive Language Model), which benefits from directly using an off-the-shelf language model architecture and demonstrates consistently high performance under different settings. Specifically, UniSAr extends existing autoregressive language models to incorporate two non-invasive extensions to make them structure-aware: (1) adding structure mark to encode database schema, conversation context, and their relationships;(2) constrained decoding to decode well-structured SQL for a given database schema. On seven well-known text-to-SQL datasets covering multi-domain, multi-table, and multi-turn, UniSAr demonstrates highly comparable or better performance to the most advanced specifically-designed text-to-SQL models.

关键词： Text-to-SQL Semantic parsing Natural language interfaces to databases Natural language processing constrained decoding

来源：评论

学校读者我要写书评

暂无评论

Using Grammar Masking to Ensure Syntactic Validity in LLM-based Modeling Tasks 24

Using Grammar Masking to Ensure Syntactic Validity in LLM-ba...

引用

ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings (MODELS)

作者： Netz, Lukas Reimer, Jan Rumpe, Bernhard Chair Software Engn Aachen Nrw Germany

ISBN: (纸本)9798400706226

We present and evaluate a method called grammar masking, which is used to guide large language models (LLMs) toward producing syntactically correct models for a given context-free grammar. Prompt engineering methods such as few-shot learning or priming can be used to improve the chances of an LLM producing correct syntax, but the more complex the grammar, the more time-consuming and less promising these methods become. Previous work is focused primarily on the usage of either language model training or prompt engineering. In this work, a method is presented that restricts the output to a given grammar using constrained decoding to ensure the output adheres to a valid syntax. We use several domain-specific languages (DSLs) built with MontiCore and task multiple LLMs to produce models with and without constrained decoding. A corresponding parser is used to confirm the syntactic correctness of each model. We show that grammar masking can dramatically improve the modeling capabilities of several LLMs, reducing the need for well-refined prompting while increasing the chance of producing correct models.

关键词： LLM MDSE Guidance CFG constrained decoding

来源：评论

学校读者我要写书评

暂无评论

A Fuzzy Training Framework for Controllable Sequence-to-Sequence Generation

引用

IEEE ACCESS 2022年 10卷 92467-92480页

作者： Li, Jiajia Wang, Ping Li, Zuchao Liu, Xi Utiyama, Masao Sumita, Eiichiro Zhao, Hai Ai, Haojun Hankou Univ Mus Sch Wuhan 430212 Peoples R China Wuhan Univ Wuhan 430072 Peoples R China Wuhan Conservatory Mus Wuhan 430060 Peoples R China Natl Inst Informat & Commun Technol Koganei Tokyo 1848795 Japan Shanghai Jiao Tong Univ Shanghai 200240 Peoples R China

The generation of music lyrics by artificial intelligence (AI) is frequently modeled as a language-targeted sequence-to-sequence generation task. Formally, if we convert the melody into a word sequence, we can consider the lyrics generation task to be a machine translation task. Traditional machine translation tasks involve translating between cross-lingual word sequences, whereas music lyrics generation tasks involve translating between music and natural language word sequences. The theme or key words of the generated lyrics are usually limited to meet the needs of the users when they are generated. This requirement can be thought of as a restricted translation problem. In this paper, we propose a fuzzy training framework that allows a model to simultaneously support both unrestricted and restricted translation by adopting an additional auxiliary training process without constraining the decoding process. This maintains the benefits of restricted translation but greatly reduces the extra time overhead of constrained decoding, thus improving its practicality. The experimental results show that our framework is well suited to the Chinese lyrics generation and restricted machine translation tasks, and that it can also generate language sequence under the condition of given restricted words without training multiple models, thereby achieving the goal of green AI.

关键词： Artificial intelligence decoding Machine translation Training data Music Natural languages Computational modeling Time factors Fuzzy systems Task analysis Music lyrics generation controllable generation music understanding constrained decoding fuzzy training

来源：评论

学校读者我要写书评

暂无评论

Type-constrained Code Generation with Language Models

引用

Proceedings of the ACM on Programming Languages 2025年第PLDI期9卷 601-626页

作者： Niels Mündler Jingxuan He Hao Wang Koushik Sen Dawn Song Martin Vechev ETH Zurich Zurich Switzerland University of California at Berkeley Berkeley USA

Large language models (LLMs) have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedure does not model formal aspects of code. Although constrained decoding is a promising approach to alleviate this issue, it has only been applied to handle either domain-specific languages or syntactic features of general-purpose programming languages. However, LLMs frequently generate code with typing errors, which are beyond the domain of syntax and generally hard to adequately constrain. To address this challenge, we introduce a type-constrained decoding approach that leverages type systems to guide code generation. For this purpose, we develop novel prefix automata and a search over inhabitable types, forming a sound approach to enforce well-typedness on LLM-generated code. We formalize our approach on a foundational simply-typed language and extend it to TypeScript to demonstrate practicality. Our evaluation on the HumanEval and MBPP datasets shows that our approach reduces compilation errors by more than half and significantly increases functional correctness in code synthesis, translation, and repair tasks across LLMs of various sizes and model families, including state-of-the-art open-weight models with more than 30B parameters. The results demonstrate the generality and effectiveness of our approach in constraining LLM code generation with formal rules of type systems.

关键词： Code Generation constrained decoding Language Model Program Repair Program Synthesis Program Translation Type System

来源：评论

学校读者我要写书评

暂无评论

Table-to-Dialog: Building Dialog Assistants to Chat With People on Behalf of You

引用

IEEE ACCESS 2020年 8卷 102313-102320页

作者： E, Haihong Zhan, Zecheng Song, Meina Beijing Univ Posts & Telecommun Sch Comp Sci Beijing 100876 Peoples R China

Artificial Intelligence (AI) personal assistant has attracted much attention from both academia and industry. Almost all existing AI personal assistants serve as service terminals to chat with human users for certain tasks. We are instead interested in building AI personal assistants for a different yet important dialog scenario, where they chat with people to fulfill specific tasks on behalf of their human users. As the personal assistants are playing a requester role, instead of a service terminal role, the conversation goal becomes delivering or requesting information according to specific user requests precisely and efficiently. The challenge for the conversation policy is that all user requests must be delivered precisely, while the challenge for the response generation is that it's generally expected for machine generated responses to cover multiple information slots, either requesting or delivering, to make the conversation efficient. In this paper, we present Table-to-Dialogue, a novel approach to address the above challenges when building a requester role AI personal assistant. We employ an encoder-decoder network to learn explicit conversation policy, which generates the corresponding information slots based on the conversation context and the user request table. We further integrate a novel Multi-Slot constrained Bi-directional Decoder (MS-CBD) into the above encoder-decoder network, to generate machine response according to the multiple slot values and their intermediate representations from the policy decoder. Different from the existing single direction text decoder approaches, MS-CBD leverage the bi-directional context of the response when generating it to enhance the semantic coherence. The experiments shows that our approach significantly outperform the state-of-the-art conversation approaches on automatic and human evaluation metrics.

关键词： Dialogue system sequence generation deep learning constrained decoding table-to-text natural language process

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：