Our physical world is constantly evolving over time, rendering challenges for pre-trained language models to understand and reason over the temporal contexts of texts. Existing work focuses on strengthening the direct...
详细信息
ISBN:
(纸本)9798891760608
Our physical world is constantly evolving over time, rendering challenges for pre-trained language models to understand and reason over the temporal contexts of texts. Existing work focuses on strengthening the direct association between a piece of text and its time-stamp. However, the knowledge-time association is usually insufficient for the downstream tasks that require reasoning over temporal dependencies between knowledge. In this work, we make use of the underlying nature of time, all temporally-scoped sentences are strung together through a one-dimensional time axis, and suggest creating a graph structure based on the relative placements of events along the time axis. Inspired by the graph view, we propose REMEMO (Relative Time Modeling), which explicitly connects all temporally-scoped facts by modeling the time relations between any two sentences. Experimental results show that REMEMO outperforms the baseline T5 on multiple temporal question answering datasets under various settings. Further analysis suggests that REMEMO is especially good at modeling long-range complex temporal dependencies. We release our code and pretrained checkpoints at https://***/DAMO-NLP-SG/RemeMo.
The performance of Large language Models (LLMs) is substantially influenced by the pretraining corpus, which consists of vast quantities of unsupervised data processed by the models. Despite its critical role in model...
详细信息
State-of-the-art Large language Models (LLMs) are accredited with an increasing number of different capabilities, ranging from reading comprehension over advanced mathematical and reasoning skills to possessing scient...
详细信息
Subject-verb agreement in the presence of an attractor noun located between the main noun and the verb elicits complex behavior: judgments of grammaticality are modulated by the grammatical features of the attractor. ...
详细信息
ISBN:
(纸本)9798891760608
Subject-verb agreement in the presence of an attractor noun located between the main noun and the verb elicits complex behavior: judgments of grammaticality are modulated by the grammatical features of the attractor. For example, in the sentence "The girl near the boys likes climbing", the attractor (boys) disagrees in grammatical number with the verb (likes), creating a locally implausible transition probability. Here, we parametrically modulate the distance between the attractor and the verb while keeping the length of the sentence equal. We evaluate the performance of both humans and two artificial neural network models: both make more mistakes when the attractor is closer to the verb, but neural networks get close to the chance level while humans are mostly able to overcome the attractor interference. Additionally, we report a linear effect of attractor distance on reaction times. We hypothesize that a possible reason for the proximity effect is the calculation of transition probabilities between adjacent words. Nevertheless, classical models of attraction such as the cue-based model might suffice to explain this phenomenon, thus paving the way for new research. Data and analyses available at https://***/d4g6k
Pretrained language models (PLMs) have been shown to encode binary gender information of text authors, raising the risk of skewed representations and downstream harms. This effect is yet to be examined for transgender...
详细信息
Machine translation eliminates the obstacles caused by linguistic disparities around the world. The automatic translation of naturallanguages using machine translation methods breaks communication barriers and brings...
详细信息
Machine translation eliminates the obstacles caused by linguistic disparities around the world. The automatic translation of naturallanguages using machine translation methods breaks communication barriers and brings people closer together, regardless of language differences. Over the years, neural-based automatic naturallanguage translation has achieved tremendous success. Despite its massive success, the neural-based approach is corpus-based, meaning that prediction accuracy depends on the input data volume. Recent years have witnessed an enormous growth in research into machine translation of various Indian languages. However, several Indian languages are still under investigation because their resources are inefficient for machine translation methods. In our experiment, we used a variety of neural-based techniques to assess the translation performance of the Nyishi-to-English corpora based on sentence lengths, a language with extremely limited online and offline resources. The experiment aims to identify the model's performance based on the lengths of sentences with limited resources. We used the BLEU score up to 4-gram precision, Chrf, sacreBLEU and TER to judge the quality of prediction. Finally, we use the human evaluation method to evaluate the prediction error. We use BPE tokenization to handle the rare word difficulties of low-resource tonal languages. With BPE, all models with sentence lengths of 1-10 words and a BLEU score perform flawlessly.
By grounding naturallanguage inference in code (and vice versa), researchers aim to create programming assistants that explain their work, are "coachable" and can surface any gaps in their reasoning. Can we...
详细信息
ISBN:
(纸本)9798891760615
By grounding naturallanguage inference in code (and vice versa), researchers aim to create programming assistants that explain their work, are "coachable" and can surface any gaps in their reasoning. Can we deduce automatically interesting properties of programs from their syntax and common-sense annotations alone, without resorting to static analysis? How much of program logic and behaviour can be captured in naturallanguage? To stimulate research in this direction and attempt to answer these questions we propose HTL, a dataset and protocol for annotating programs with naturallanguage predicates at a finer granularity than code comments and without relying on internal compiler representations. The dataset is available at the following address: https://***/10.5281/zenodo.7893113
Large language models (LLMs) are primarily evaluated by overall performance on various text understanding and generation tasks. However, such a paradigm fails to comprehensively differentiate the fine-grained language...
详细信息
Despite their success at many naturallanguageprocessing (NLP) tasks, large language models (LLMs) still struggle to effectively leverage knowledge for knowledge-intensive tasks, manifesting limitations such as gener...
详细信息
Gender-fair language, an evolving German linguistic variation, fosters inclusion by addressing all genders or using neutral forms. Nevertheless, there is a significant lack of resources to assess the impact of this li...
详细信息
暂无评论