Major Depressive Disorder (MDD) is a prevalent mental disorder, affecting a significant number of individuals, with estimates reaching 300 million cases worldwide. Currently, the diagnosis of this condition relies hea...
详细信息
This paper describes a Plastic Waste Hotspot Detection System which has been developed in an international collaborative research project to realize an 'Environmental AI-Human Actions integration' with marine ...
详细信息
Air quality significantly impacts human health and economic conditions, making precise and timely assessment crucial in urban areas. Existing studies often fail to predict pollution accurately in smaller areas due to ...
详细信息
Scientific research increasingly relies on distributed computational resources, storage systems, networks, and instruments, ranging from HPC and cloud systems to edge devices. Event-driven architecture (EDA) benefits ...
详细信息
We tackle the problem of enumerating set-theoretic solutions to the Yang-Baxter equation. This equation originates from statistical and quantum mechanics, but also has applications in knot theory, cryptography, quantu...
详细信息
This study investigates the performance of Vision Transformer (ViT) variants - the Shifted Window Transformers (SWIN), Distillation with No Labels (DINO), and data-efficient Image Transformers (DeIT) - in image captio...
详细信息
Cancer is one of the leading causes of death worldwide. Pathogenic viruses are estimated to be responsible for 15% of all human cancers globally and pose significant threats to public health. Viruses integrate their g...
详细信息
Pre-training large transformer-based language models on gigantic corpora and later repurposing them as base models for finetuning on downstream tasks has proven instrumental to the recent advances in computational lin...
详细信息
Pre-training large transformer-based language models on gigantic corpora and later repurposing them as base models for finetuning on downstream tasks has proven instrumental to the recent advances in computational linguistics. However, the prohibitively high cost associated with pretraining often hampers the regular updates of base models to incorporate the latest linguistic developments. To address this issue, we present an innovative approach for efficiently producing more powerful and up-to-date versions of RobBERT, our series of cutting-edge Dutch language models, by leveraging existing language models designed for high-resource languages. Unlike the prior versions of RobBERT, which relied on the training methodology of RoBERTa but required a fresh weight initialization, our two RobBERT-2023 models (base and large) are entirely initialized using the RoBERTa-family of models. To initialize an embedding table tailored to the newly devised Dutch tokenizer, we rely on a token translation strategy introduced by Remy et al. (2023). Along with our RobBERT-2023 release, we deliver a freshly pre-trained Dutch tokenizer using the latest version of the Dutch OSCAR corpus. This corpus incorporates new high-frequency terms, such as those related to the COVID-19 pandemic, cryptocurrencies, and the ongoing energy crisis, while mitigating the inclusion of previously over-represented terms from adult-oriented content. To assess the value of RobBERT-2023, we evaluate its performance using the same benchmarks employed for the state-of-the-art RobBERT-2022 model, as well as the newly-released Dutch Model Benchmark. Our experimental results demonstrate that RobBERT-2023 not only surpasses its predecessor in various aspects but also achieves these enhancements at a significantly reduced training cost. This work represents a significant step forward in keeping Dutch language models up-to-date and demonstrates the potential of model conversion techniques for reducing the environmental
Camouflaged object detection (COD) aims to identify target objects in complex scenes with extremely high similarity to their surroundings, and has significant applications in military, medical, and other fields. This ...
详细信息
Assessing the performance of machine translation systems is of critical value, especially to languages with lower resource availability. Due to the large evaluation effort required by the translation task, studies oft...
详细信息
暂无评论