检索结果-内蒙古大学图书馆

Non-autoregressive Streaming Transformer for Simultaneous Translation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ma, Zhengrui Zhang, Shaolei Guo, Shoutao Shao, Chenze Zhang, Min Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China School of Future Science and Engineering Soochow University China

Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while maintaining low latency often leads to a tendency for aggressive anticipation. We argue that such issue stems from the autoregressive architecture upon which most existing SiMT models are built. To address those issues, we propose non-autoregressive streaming Transformer (NAST) which comprises a unidirectional encoder and a non-autoregressive decoder with intra-chunk parallelism. We enable NAST to generate the blank token or repetitive tokens to adjust its READ/WRITE strategy flexibly, and train it to maximize the non-monotonic latent alignment with an alignment-based latency loss. Experiments on various SiMT benchmarks demonstrate that NAST outperforms previous strong autoregressive SiMT baselines. Source code is publicly available at https://***/ictnlp/NAST. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine translation

Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hu, Beizhe Sheng, Qiang Cao, Juan Shi, Yuhui Li, Yang Wang, Danding Qi, Peng Key Lab of Intelligent Information Processing Chinese Academy of Sciences Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China National University of Singapore Singapore

Detecting fake news requires both a delicate sense of diverse clues and a profound understanding of the real-world background, which remains challenging for detectors based on small language models (SLMs) due to their knowledge and capability limitations. Recent advances in large language models (LLMs) have shown remarkable performance in various tasks, but whether and how LLMs could help with fake news detection remains underexplored. In this paper, we investigate the potential of LLMs in fake news detection. First, we conduct an empirical study and find that a sophisticated LLM such as GPT 3.5 could generally expose fake news and provide desirable multi-perspective rationales but still underperforms the basic SLM, fine-tuned BERT. Our subsequent analysis attributes such a gap to the LLM’s inability to select and integrate rationales properly to conclude. Based on these findings, we propose that current LLMs may not substitute fine-tuned SLMs in fake news detection but can be a good advisor for SLMs by providing multi-perspective instructive rationales. To instantiate this proposal, we design an adaptive rationale guidance network for fake news detection (ARG), in which SLMs selectively acquire insights on news analysis from the LLMs’ rationales. We further derive a rationale-free version of ARG by distillation, namely ARG-D, which services cost-sensitive scenarios without querying LLMs. Experiments on two real-world datasets demonstrate that ARG and ARG-D outperform three types of baseline methods, including SLM-based, LLM-based, and combinations of small and large language models. © 2023, CC BY-NC-SA.

关键词： Distillation

Beyond MLE: Convex Learning for Text Generation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shao, Chenze Ma, Zhengrui Zhang, Min Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China School of Future Science and Engineering Soochow University China

Maximum likelihood estimation (MLE) is a statistical method used to estimate the parameters of a probability distribution that best explain the observed data. In the context of text generation, MLE is often used to train generative language models, which can then be used to generate new text. However, we argue that MLE is not always necessary and optimal, especially for closed-ended text generation tasks like machine translation. In these tasks, the goal of model is to generate the most appropriate response, which does not necessarily require it to estimate the entire data distribution with MLE. To this end, we propose a novel class of training objectives based on convex functions, which enables text generation models to focus on highly probable outputs without having to estimate the entire data distribution. We investigate the theoretical properties of the optimal predicted distribution when applying convex functions to the loss, demonstrating that convex functions can sharpen the optimal distribution, thereby enabling the model to better capture outputs with high probabilities. Experiments on various text generation tasks and models show the effectiveness of our approach. It enables autoregressive models to bridge the gap between greedy and beam search, and facilitates the learning of non-autoregressive models with a maximum improvement of 9+ BLEU points. Moreover, our approach also exhibits significant impact on large language models (LLMs), substantially enhancing their generative capability on various tasks. Source code is available at https://***/ictnlp/Convex-Learning. Copyright © 2023, The Authors. All rights reserved.

关键词： Maximum likelihood estimation

SSE-SAM: Balancing Head and Tail Classes Gradually Through Stage-Wise SAM 39

学校读者我要写书评

暂无评论

SSE-SAM: Balancing Head and Tail Classes Gradually Through S...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Lyu, Xingyu Xu, Qianqian Yang, Zhiyong Lyu, Shaojie Huang, Qingming Key Lab. of Intelligent Information Processing Institute of Computing Tech. CAS China School of Computer Science and Tech. University of Chinese Academy of Sciences China Tencent Corporate China BDKM University of Chinese Academy of Sciences China

ISBN: (纸本)157735897X

Real-world datasets often exhibit a long-tailed distribution, where vast majority of classes known as tail classes have only few samples. Traditional methods tend to overfit on these tail classes. Recently, a new approach called Imbalanced SAM (ImbSAM) is proposed to leverage the generalization benefits of Sharpness-Aware Minimization (SAM) for long-tailed distributions. The main strategy is to merely enhance the smoothness of the loss function for tail classes. However, we argue that improving generalization in long-tail scenarios requires a careful balance between head and tail classes. We show that neither SAM nor ImbSAM alone can fully achieve this balance. For SAM, we prove that although it enhances the model's generalization ability by escaping saddle point in the overall loss landscape, it does not effectively address this for tail-class losses. Conversely, while ImbSAM is more effective at avoiding saddle points in tail classes, the head classes are trained insufficiently, resulting in significant performance drops. Based on these insights, we propose Stage-wise Saddle Escaping SAM (SSE-SAM), which uses complementary strengths of ImbSAM and SAM in a phased approach. Initially, SSE-SAM follows the majority sample to avoid saddle points of the head-class loss. During the later phase, it focuses on tail-classes to help them escape saddle points. Our experiments confirm that SSE-SAM has better ability in escaping saddles both on head and tail classes, and shows performance improvements. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：

Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Shao, Chenze Ma, Zhengrui Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China

Non-autoregressive models achieve significant decoding speedup in neural machine translation but lack the ability to capture sequential dependency. Directed Acyclic Transformer (DA-Transformer) was recently proposed to model sequential dependency with a directed acyclic graph. Consequently, it has to apply a sequential decision process at inference time, which harms the global translation accuracy. In this paper, we present a Viterbi decoding framework for DA-Transformer, which guarantees to find the joint optimal solution for the translation and decoding path under any length constraint. Experimental results demonstrate that our approach consistently improves the performance of DA-Transformer while maintaining a similar decoding speedup. Copyright © 2022, The Authors. All rights reserved.

关键词： Decoding

Improving Fake News Detection of Influential Domain via Domain- and Instance-Level Transfer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Nan, Qiong Wang, Danding Zhu, Yongchun Sheng, Qiang Shi, Yuhui Cao, Juan Li, Jintao Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China

Both real and fake news in various domains, such as politics, health, and entertainment are spread via online social media every day, necessitating fake news detection for multiple domains. Among them, fake news in specific domains like politics and health has more serious potential negative impacts on the real world (e.g., the infodemic led by COVID-19 misinformation). Previous studies focus on multi-domain fake news detection, by equally mining and modeling the correlation between domains. However, these multi-domain methods suffer from a seesaw problem: the performance of some domains is often improved at the cost of hurting the performance of other domains, which could lead to an unsatisfying performance in specific domains. To address this issue, we propose a Domain- and Instance-level Transfer Framework for Fake News Detection (DITFEND), which could improve the performance of specific target domains. To transfer coarse-grained domain-level knowledge, we train a general model with data of all domains from the meta-learning perspective. To transfer fine-grained instance-level knowledge and adapt the general model to a target domain, we train a language model on the target domain to evaluate the transferability of each data instance in source domains and re-weigh each instance’s contribution. Offline experiments on two datasets demonstrate the effectiveness of DITFEND. Online experiments show that DITFEND brings additional improvements over the base models in a real-world scenario. Copyright © 2022, The Authors. All rights reserved.

关键词： Social networking (online)

An Efficient Authentication and Key Agreement Scheme for CAV Internal Applications 19th

学校读者我要写书评

暂无评论

An Efficient Authentication and Key Agreement Scheme for C...

19th EAI International Conference on Collaborative computing: Networking, Applications and Worksharing, CollaborateCom 2023

作者： Li, Yang Zhang, Qingyang Cao, Wenwen Cui, Jie Zhong, Hong Anhui Province Key Laboratory of Cyberspace Security Situation Awareness and Evaluation Hefei230037 China School of Computer Science and Technology Anhui University Hefei230039 China Anhui Engineering Laboratory of IoT Security Technologies Anhui University Hefei230039 China Institute of Physical Science and Information Technology Anhui University Hefei230039 China Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education Anhui University Hefei230039 China

ISBN: (纸本)9783031545276

The data of applications in connected and autonomous vehicles are important, which is usually collected by service providers to improve their services, such as object detection model. But, wireless communication is susceptible to various kinds of attacks. Thus, the data of the application module needs to be securely shared to the corresponding service provider. However, current schemes are with limited performance while a service provider collects multiple application data at the same time. By adopting signcryption and chaotic map, an efficient authentication and key agreement scheme is proposed, while batch authentication is achieved for efficient message authentication of multiple applications, and the efficient revocation is realized based on Chinese remainder theorem under the assistance of trusted execution environment supported vehicle computing/communication unit. The formal security proof shows that the scheme is secure under the random oracle model, and the experiment results shows that the scheme is more efficient than related schemes and can meet the requirements of CAV. © ICST institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2024.

关键词： Authentication

Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Shao, Chenze Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China

Non-autoregressive translation (NAT) models are typically trained with the cross-entropy loss, which forces the model outputs to be aligned verbatim with the target sentence and will highly penalize small shifts in word positions. Latent alignment models relax the explicit alignment by marginalizing out all monotonic latent alignments with the CTC loss. However, they cannot handle non-monotonic alignments, which is non-negligible as there is typically global word reordering in machine translation. In this work, we explore non-monotonic latent alignments for NAT. We extend the alignment space to non-monotonic alignments to allow for the global word reordering and further consider all alignments that overlap with the target sentence. We non-monotonically match the alignments to the target sentence and train the latent alignment model to maximize the F1 score of non-monotonic matching. Extensive experiments on major WMT benchmarks show that our method substantially improves the translation performance of CTC-based models. Our best model achieves 30.06 BLEU on WMT14 En-De with only one-iteration decoding, closing the gap between non-autoregressive and autoregressive models.2 Copyright © 2022, The Authors. All rights reserved.

关键词： Alignment

A Wideband Circularly Polarized Folded Transmitarray Antenna With Linearly Polarized Feed 7

学校读者我要写书评

暂无评论

A Wideband Circularly Polarized Folded Transmitarray Antenna...

7th IEEE International Conference on Electronic information and Communication Technology, ICEICT 2024

作者： Sun, Xiangdong Guo, Xulong Li, Yin Zhang, Yongliang Li, Yingsong Inner Mongolia University School of Electronic Information Engineering Hohhot010021 China Inner Mongolia University Inner Mongolia Branch of National Computer Network Emergency Response Technical Team Coordination Center of China Hohhot010020 China Peng Cheng Laboratory Shenzhen518000 China Inner Mongolia University School of Transportation Institute Hohhot010021 China Ministry of Education Anhui University Key Laboratory of Intelligent Computing and Signal Processing China

ISBN: (纸本)9798350384437

A low-profile, high-gain, wideband circularly polarized folded transmitarray antenna (CPFTA) is proposed, and a polarization-selective linear-to-circular polarization (LP-CP) conversion element is introduced. The CPFTA consists of a linearly polarized (LP) magnetoelectric dipole antenna and a circularly polarized (CP) helical patch antenna. The CPFTA demonstrates excellent performance with a 3dB gain bandwidth of 29.5% and a 3dB axial ratio (AR) bandwidth of 40.9%. © 2024 IEEE.

关键词： Circular polarization