检索结果-内蒙古大学图书馆

Investigating alignment interpretability for low-resource NMT

MACHINE TRANSLATION 2020年第4期34卷 305-323页

作者： Boito, Marcely Zanon Villavicencio, Aline Besacier, Laurent Univ Grenoble Alpes Lab Informat Grenoble Grenoble France Univ Sheffield Dept Comp Sci Sheffield S Yorkshire England Univ Fed Rio Grande do Sul Inst Informat Porto Alegre RS Brazil

The attention mechanism in Neural Machine Translation (NMT) models added flexibility to translation systems, and the possibility to visualize soft-alignments between source and target representations. While there is much debate about the relationship between attention and the yielded output for neural models (Jain and Wallace 2019;Serrano and Smith 2019;Wiegreffe and Pinter 2019;Vashishth et al. 2019), in this paper we propose a different assessment, investigating soft-alignment interpretability in low-resource scenarios. We experimented with different architectures (RNN (Bahdanau et al. 2015), 2D-CNN (Elbayad et al. 2018), and Transformer (Vaswani et al. 2017)), comparing them with regards to their ability to produce directly exploitable alignments. For evaluating exploitability, we replicated the Unsupervised Word Segmentation (UWS) task from Godard et al. (2018). There, source words are translated into unsegmented phone sequences. Posterior to training, the resulting soft-alignments are used for producing segmentation over the target side. Our results showed that a RNN-based NMT model produced the most exploitable alignments in this scenario. We then investigated methods for increasing its UWS scores by comparing the following methodologies: monolingual pre-training, input representation augmentation (hybrid model), and explicit word length optimization during training. We reached the best results by using the hybrid model, which uses an intermediate monolingual-rooted segmentation from a non-parametric Bayesian model (Goldwater 2007) to enrich the input representation before training.

关键词： Low-resource languages Attention mechanism Sequence-to-sequence models Unsupervised word segmentation computational language documentation Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Empirical Evaluation of Sequence-to-Sequence Models for Word Discovery in Low-resource Settings 20

Empirical Evaluation of Sequence-to-Sequence Models for Word...

引用

Interspeech Conference

作者： Boito, Marcely Zanon Villavicencio, Aline Besacier, Laurent Univ Grenoble Alpes UGA Lab Informat Grenoble Grenoble France Univ Essex Sch Comp Sci & Elect Engn Colchester Essex England Univ Fed Rio Grande do Sul Inst Informat Porto Alegre RS Brazil

Since Bahdanau et al. [1] first introduced attention for neural machine translation, most sequence-to-sequence models made use of attention mechanisms [2, 3, 4]. While they produce soft-alignment matrices that could be interpreted as alignment between target and source languages, we lack metrics to quantify their quality, being unclear which approach produces the best alignments. This paper presents an empirical evaluation of 3 of the main sequence-to-sequence models for word discovery from unsegmented phoneme sequences: CNN, RNN and Transformer-based. This task consists in aligning word sequences in a source language with phoneme sequences in a target language, inferring from it word segmentation on the target side [5]. Evaluating word segmentation quality can be seen as an extrinsic evaluation of the soft-alignment matrices produced during training. Our experiments in a low-resource scenario on Mboshi and English languages (both aligned to French) show that RNNs surprisingly outperform CNNs and Transformer for this task. Our results are confirmed by an intrinsic evaluation of alignment quality through the use Average Normalized Entropy (ANE). Lastly, we improve our best word discovery model by using an alignment entropy confidence measure that accumulates ANE over all the occurrences of a given alignment pair in the collection.

关键词： sequence-to-sequence models soft-alignment matrices word discovery low-resource languages computational language documentation

来源：评论

学校读者我要写书评

暂无评论

Unsupervised Word Segmentation from Speech with Attention 19

Unsupervised Word Segmentation from Speech with Attention

引用

19th Annual Conference of the International-Speech-Communication-Association (INTERSPEECH 2018)

作者： Godard, Pierre Boito, Marcely Zanon Ondel, Lucas Berard, Alexandre Yvon, Francois Villavicencio, Aline Besacier, Laurent Univ Paris Saclay CNRS LIMSI Orsay France UGA CNRS INRIA LIGG INP Grenoble France BUT Brno Czech Republic Univ Essex CSEE Colchester Essex England Univ Lille CRISTAL Lille France

ISBN: (纸本)9781510872219

We present a first attempt to perform attentional word segmentation directly from the speech signal, with the final goal to automatically identify lexical units in a low-resource, unwritten language (UL). Our methodology assumes a pairing between recordings in the UL with translations in a well-resourced language. It uses Acoustic Unit Discovery (AUD) to convert speech into a sequence of pseudo-phones that is segmented using neural soft-alignments produced by a neural machine translation model. Evaluation uses an actual Bantu UL, Mboshi;comparisons to monolingual and bilingual baselines illustrate the potential of attentional word segmentation for language documentation.

关键词： computational language documentation encoder-decoder models attentional models unsupervised word segmentation

来源：评论

学校读者我要写书评

暂无评论

UNWRITTEN languageS DEMAND ATTENTION TOO! WORD DISCOVERY WITH ENCODER-DECODER MODELS

UNWRITTEN LANGUAGES DEMAND ATTENTION TOO! WORD DISCOVERY WIT...

引用

IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

作者： Boito, Marcely Zanon Berard, Alexandre Villavicencio, Aline Besacier, Laurent Univ Grenoble Alpes Lab Informat Grenoble Grenoble France Univ Fed Rio Grande do Sul Inst Informat Porto Alegre RS Brazil

ISBN: (纸本)9781509047888

Word discovery is the task of extracting words from un-segmented text. In this paper we examine to what extent neural networks can be applied to this task in a realistic unwritten language scenario, where only small corpora and limited annotations are available. We investigate two scenarios: one with no supervision and another with limited supervision with access to the most frequent words. Obtained results show that it is possible to retrieve at least 27% of the gold standard vocabulary by training an encoder-decoder neural machine translation system with only 5,157 sentences. This result is close to those obtained with a task-specific Bayesian nonparametric model. Moreover, our approach has the advantage of generating translation alignments, which could be used to create a bilingual lexicon. As a future perspective, this approach is also well suited to work directly from speech.

关键词： Word Discovery computational language documentation Neural Machine Translation Attention models

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：