检索结果-内蒙古大学图书馆

Global receptive field transformer decoder method on quantum surface code data and syndrome error correction

Chinese Physics B 2025年第3期34卷 267-276页

作者： Ao-Qing Li Ce-Wen Tian Xiao-Xuan Xu Hong-Yang Ma Jun-Qing Liang School of Information and Control Engineering Qingdao University of TechnologyQingdao 266033China School of Sciences Qingdao University of TechnologyQingdao 266033China

Quantum computing has the potential to solve complex problems that are inefficiently handled by classical ***,the high sensitivity of qubits to environmental interference and the high error rates in current quantum devices exceed the error correction thresholds required for effective algorithm ***,quantum error correction technology is crucial to achieving reliable quantum *** this work,we study a topological surface code with a two-dimensional lattice structure that protects quantum information by introducing redundancy across multiple qubits and using syndrome qubits to detect and correct ***,errors can occur not only in data qubits but also in syndrome qubits,and different types of errors may generate the same syndromes,complicating the decoding task and creating a need for more efficient decoding *** address this challenge,we used a transformer decoder based on an attention *** mapping the surface code lattice,the decoder performs a self-attention process on all input syndromes,thereby obtaining a global receptive *** performance of the decoder was evaluated under a phenomenological error *** results demonstrate that the decoder achieved a decoding accuracy of 93.8%.Additionally,we obtained decoding thresholds of 5%and 6.05%at maximum code distances of 7 and 9,*** results indicate that the decoder used demonstrates a certain capability in correcting noise errors in surface codes.

关键词： quantum error correction surface code transformer decoder

来源：评论

学校读者我要写书评

暂无评论

A speech recognition method with enhanced transformer decoder

引用

EURASIP JOURNAL ON AUDIO SPEECH AND MUSIC PROCESSING 2025年第1期2025卷 1-12页

作者： Hu, Hengbo Niu, Tong He, Zhenhua Zhengzhou Xinda Inst Adv Technol Res & Dev Dept 1 Intelligent Speech Technol Team Lianhua St Zhengzhou 450000 Henan Peoples R China Univ informat Engn Sch Informat Syst Engn Sci Ave Zhengzhou 450000 Henan Peoples R China

Addressing the issue that the transformer decoder struggles to capture local features for monotonic alignment in speech recognition, and simultaneously incorporating language model cold fusion training into the decoder, an enhanced decoder-based speech recognition model is investigated. The enhanced decoder separates and combines the two attention mechanisms in the transformer decoder into cross-attention layers and a self-attention language model module. The cross-attention layers are utilized to capture local features more efficiently from the encoder output, and the self-attention language model module is used to pre-train with additional domain-related text, followed by cold fusion training. Experimental results on the Mandarin Aishell-1 dataset demonstrate that when the encoder is a Conformer, the enhanced decoder achieves a 16.1% reduction in character error rate compared to the transformer decoder. Furthermore, when the language model is pre-trained with suitable text data, the performance of the cold fusion-trained model is further enhanced.

关键词： Cross-attention transformer decoder Language model cold fusion

来源：评论

学校读者我要写书评

暂无评论

TNFORMER: SINGLE-PASS MULTILINGUAL TEXT NORMALIZATION WITH A transformer decoder MODEL 49

TNFORMER: SINGLE-PASS MULTILINGUAL TEXT NORMALIZATION WITH A...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Shen, Binbin Wang, Jie Meng, Meng Wang, Yujun Xiaomi Inc Beijing Peoples R China

ISBN: (纸本)9798350344868;9798350344851

Text Normalization (TN) is a pivotal pre-processing procedure in speech synthesis systems, which converts diverse forms of text into a canonical form suitable for correct synthesis. This work introduces a novel model, TNFormer, which innovatively transforms the TN task into a next token prediction problem, leveraging the structure of GPT with only transformer decoders for efficient, single-pass TN. The strength of TNFormer lies not only in its ability to identify Non-Standard Words that require normalization but also in its aptitude for context-driven normalization in a single pass. Though not exclusively designed for multilingual contexts, TNFormer naturally supports different languages and multi-lingual mixtures, demonstrating impressive performance on English and Chinese datasets. The development of TNFormer represents a notable advancement in text normalization task.

关键词： Text normalization Speech synthesis Multilingual transformer decoder

来源：评论

学校读者我要写书评

暂无评论

Unifying Convolution and transformer decoder for Textile Fiber Identification

引用

Journal of Donghua University(English Edition) 2023年第4期40卷 357-363页

作者：许罗力李粉英常姗 College of Computer Science and Technology Donghua UniversityShanghai 201620China Silicon Engineer Group ZEKU Technology(Shanghai)Shanghai 201203China

At present,convolutional neural networks(CNNs)and transformers surpass humans in many situations(such as face recognition and object classification),but do not work well in identifying fibers in textile surface ***,this paper proposes an architecture named FiberCT which takes advantages of the feature extraction capability of CNNs and the long-range modeling capability of transformer decoders to adaptively extract multiple types of fiber ***,the convolution module extracts fiber features from the input textile surface ***,these features are sent into the transformer decoder module where label embeddings are compared with the features of each type of fibers through multi-head cross-attention and the desired features are pooled ***,an asymmetric loss further purifies the extracted fiber *** show that FiberCT can more effectively extract the representations of various types of fibers and improve fiber identification accuracy than state-of-the-art multi-label classification approaches.

关键词： non-destructive textile fiber identification transformer decoder asymmetric loss

来源：评论

学校读者我要写书评

暂无评论

A transformer decoder-based Generative Adversarial Model with TrajLoss Function for Privacy-Preserving Trajectory Publishing 5

A Transformer decoder-based Generative Adversarial Model wit...

引用

5th International Conference on Machine Learning and Natural Language Processing (MLNLP)

作者： Cao, Xiaoqian Yu, Juan Han, Jianmin Yao, Xin Lu, Jianfeng Peng, Hao Zhejiang Normal Univ Coll Math & Comp Sci Jinhua Zhejiang Peoples R China

ISBN: (纸本)9781450399067

Synthetic trajectories is a promising method for privacy preserving trajectory publishing. With the significant success of deep generative models, researchers attempt to generate synthetic trajectories with deep generative models, and the LSTM-TrajGAN is a typical model. However, the LSTM-TrajGAN suffers from the drawback of sub-optimal privacy and utility trade-off. In this paper, we propose TD-TrajGANv, a transformer decoder-based generative adversarial model with TrajLoss function to generate synthetic trajectories for privacy preserving trajectory publishing. In particular, we design the GAN generator with transformer decoder to capture sequential features and maintain high utility of synthetic trajectories. Furthermore,we design a new generator loss function (TrajLoss) to avoid too similar between real and generated trajectories, thereby reducing disclosure of sensitive information by synthetic trajectories and improving privacy level. We compare the performance of the proposed method with the state-of-the-art methods LSTM-TrajGAN and its variant LSTM-TrajGANv in terms of both privacy and utility on three real-world datasets. Experimental results show that the proposed method could guarantee the utility of trajectories and protect the privacy of trajectories more effectively than the LSTM-TrajGAN and LSTM-TrajGANv.

关键词： Additional Key Words and Phrases:deep learning trajectory privacy transformer decoder trajectory utility GAN

来源：评论

学校读者我要写书评

暂无评论

基于DCNv2和transformer decoder的隧道衬砌裂缝高效检测模型研究

引用

图学学报 2024年第5期45卷 1050-1061页

作者：孙己龙刘勇周黎伟路鑫侯小龙王亚琼王志丰陕西省交通运输工程质量监测鉴定站陕西西安710075 长安大学公路学院陕西西安710064 长安大学材料科学与工程学院陕西西安710061 西安公路研究院有限公司陕西西安710065

为解决因衬砌裂缝性状随机、分布密集、标注框分辨率低所导致的现有模型识别精度低、检测速度慢及参数量庞大等问题,以第2版可变形卷积网络(DCNv2)和端到端变换器解码器(transformer decoder)为基础对YOLOv8网络框架进行改进,提出了面... 详细信息

为解决因衬砌裂缝性状随机、分布密集、标注框分辨率低所导致的现有模型识别精度低、检测速度慢及参数量庞大等问题,以第2版可变形卷积网络(DCNv2)和端到端变换器解码器(transformer decoder)为基础对YOLOv8网络框架进行改进,提出了面向衬砌裂缝的检测模型DTD-YOLOv8。首先,通过引入DCNv2对YOLOv8主干卷积网络C2f进行融合以实现模型对裂缝形变特征的准确快速感知,同时采用transformer decoder对YOLOv8检测头进行替换以实现端到端框架内完整目标检测流程,从而消除因Anchor-free处理模式所带来的计算消耗。采用自建裂缝数据集对SSD,Faster-RCNN,RT-DETR,YOLOv3,YOLOv5,YOLOv8和DTD-YOLOv8的7种检测模型进行对比验证。结果表明:改进模型F1分数和mAP@50值分别为87.05%和89.58%;其中F1分数相较其他6种模型分别提高了14.16%,7.68%,1.55%,41.36%,8.20%和7.40%;mAP@50分别提高了28.84%,15.47%,1.33%,47.65%,10.14%和10.84%。改进模型参数量仅为RT-DETR的三分之一,检测单张图片的速度为16.01 ms,FPS为65.46帧每秒,对比其他模型检测速度得到提升。该模型在面向运营隧道裂缝检测任务需求时能够表现出高效的性能。

关键词：隧道工程目标检测第2版可变形卷积网络 transformer decoder 衬砌裂缝

来源：评论

学校读者我要写书评

暂无评论

Enhancing high-vocabulary image annotation with a novel attention-based pooling

引用

VISUAL COMPUTER 2025年第5期41卷 3537-3551页

作者： Salar, Ali Ahmadi, Ali KN Toosi Univ Technol Fac Comp Engn Tehran Iran

Given an image, we aim to assign a set of semantic labels to its visual content automatically. This is generally known as automatic image annotation (AIA). Images contain objects that can vary in size and position, with some only taking up a small region of the entire picture. The rise in the number of object classes also heightens this variety. Despite the achievement of promising results, the majority of the current methods have limited efficacy in the detection of small-scale objects. To make more effective use of spatial data compared to the global pooling method, we propose a modified transformer decoder layer that decreases computational complexity without sacrificing model performance. The study has conducted multiple experiments on four datasets, including three high-vocabulary small-scale datasets (Corel 5 k, IAPR TC-12, and Esp Game) and one large-scale dataset (Visual Genome) with a vocabulary list of 500 words. In comparison with existing state-of-the-art models, our approach achieves comparable results in F1-score, N+, and mean average precision (mAP) on small- and large-scale datasets.

关键词： Image annotation Multi-label classification Large-scale categories transformer decoder Cross-attention

来源：评论

学校读者我要写书评

暂无评论

An attentive convolutional transformer-based network for road safety

引用

JOURNAL OF SUPERCOMPUTING 2023年第14期79卷 16351-16377页

作者： Jayanthan, K. S. Domnic, S. Natl Inst Technol Dept Comp Applicat Tiruchirappalli 620005 Tamil Nadu India

The sharp surge in the number of vehicles on the road leads to numerous traffic violations (Road Transport and Highways in Ministry of road transport and highways, 2021. ). The detection of traffic violations in a dynamic environment is a complex task. This paper focuses on detecting a particular traffic violation, such as riding a motorcycle without a helmet. We propose to address the problem as an object detection task. In this paper, we propose a novel convolutional encoder-transformer decoder architecture (CETD) for the object detection task. The proposed architecture comprises two primary modules: a convolutional neural network (CNN)-based convolution encoder that extracts high-level features from input images and a transformer-based decoder that utilizes attention mechanisms to identify relevant components, such as helmets or missing helmets, in the image. This architecture is designed to achieve accurate object detection and localization in images by combining advanced feature extraction techniques with state-of-the-art attention mechanisms. Layer normalization module of the proposed architecture acts as an intermediate bias stabilizer for the encoder-decoder network. The design also includes a standard backbone feature extractor and fused backbone feature extractor. The model gives strong confidence in detecting occluded objects compared to other state-of-the-art models. The detector works in an end-to-end fashion with fewer handcrafted features. We have studied the applicability of the model with a miniature version of the COCO dataset. The model gives a competitive performance with models like Faster region-based convolutional neural network (Faster R-CNN) and Mask region-based convolutional neural network on this dataset. The proposed model is also fine-tuned on the traffic data with occlusion for helmet detection. The model's performance on helmet detection from traffic data is comparable with the state-of-the-art real-time detectors such as EspiNet, a m

关键词： Object detection Convolutional encoder Layer normalization transformer decoder Helmet detection Traffic data

来源：评论

学校读者我要写书评

暂无评论

A lightweight siamese transformer for few-shot semantic segmentation

引用

NEURAL COMPUTING & APPLICATIONS 2024年第13期36卷 7455-7469页

作者： Zhu, Hegui Zhou, Yange Jiang, Cong Yang, Lianping Jiang, Wuming Wang, Zhimu Northeastern Univ Coll Sci Shenyang 110819 Peoples R China Northeastern Univ Sch Comp Sci & Engn Shenyang 110819 Peoples R China Beijing EyeCool Technol Co Ltd Beijing 100085 Peoples R China Queens Univ Sch Comp Kingston ON K7L 2N8 Canada

Few-shot semantic segmentation (FSS) is a challenging task that aims to segment new classes in query images with a few annotated support samples. One inherent challenge in FSS is the intra-class variation resulting from the limited availability of support samples and the diversity of query data. Current methods frequently employ prototype-support techniques to tackle this issue. However, it is important to note that a single support prototype shares limited commonalities with query features, and increases the difficulty of accurate segmentation. In this paper, we propose a lightweight and effective framework named Siamese transformer (SiaT) with a mere 0.68M learnable parameters to enhance commonalities between prototypes and query features. The SiaT framework consists of two key modules: the Siamese transformer Module (STM) and the Query Activation Module (QAM). The STM integrates two shared transformer decoders for information propagation to generate two enhanced prototypes. One transformer decoder facilitates the propagation of target-related information from the query to the original support prototype, while the other propagates foreground information from the support to the initial query prototype. The QAM utilizes the two enhanced prototypes from the STM as support information, engaging with query features through channel-wise allocation and concatenation, termed query activation. Moreover, SiaT showcases competitive performance on the widely used benchmarks PASCAL-5i\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5<^>i$$\end{document} and COCO-20i\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$20<^>i$$

关键词： Few-shot segmentation Siamese transformer Prototype learning transformer decoder

来源：评论

学校读者我要写书评

暂无评论

transformer with sparse self-attention mechanism for image captioning

引用

ELECTRONICS LETTERS 2020年第15期56卷 764-+页

作者： Wang, Duofeng Hu, Haifeng Chen, Dihu Sun Yat Sen Univ Sch Elect & Informat Engn Guangzhou 510006 Guangdong Peoples R China

Recently, transformer has been applied to the image caption model, in which the convolutional neural network and the transformer encoder act as the image encoder of the model, and the transformer decoder acts as the decoder of the model. However, transformer may suffer from the interference of non-critical objects of a scene and meet with difficulty to fully capture image information due to its self-attention mechanism's dense characteristics. In this Letter, in order to address this issue, the authors propose a novel transformer model with decreasing attention gates and attention fusion module. Specifically, they firstly use attention gate to force transformer to overcome the interference of non-critical objects and capture objects information more efficiently via truncating all the attention weights that smaller than gate threshold. Secondly, through inheriting attentional matrix from the previous layer of each network layer, the attention fusion module enables each network layer to consider other objects without losing the most critical ones. Their method is evaluated using the benchmark Microsoft COCO dataset and achieves better performance compared to the state-of-the-art methods.

关键词： computer vision image coding decoding neural nets learning (artificial intelligence) object detection image colour analysis matrix algebra image resolution image retrieval sparse self-attention mechanism image captioning image caption model convolutional neural network transformer encoder act image encoder transformer decoder noncritical objects image information transformer model attention gate attention fusion module capture objects information attention weights attentional matrix network layer

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：