检索结果-内蒙古大学图书馆

IEEE Transactions on Intelligent Vehicles 2024年 1-14页

作者： Lan, Zhengxing Liu, Lingshan Fan, Bo Lv, Yisheng Ren, Yilong Cui, Zhiyong School of Transportation Science and Engineering Beihang University Beijing China Beijing Key Laboratory of Traffic Engineering College of Metropolitan Transportation Beijing University of Technology Beijing China State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences Beijing China School of Transportation Science and Engineering State Key Lab of Intelligent Transportation System Beihang University Beijing China

Predicting the future trajectories of dynamic traffic actors is a cornerstone task in autonomous driving. Though existing notable efforts have resulted in impressive performance improvements, a gap persists in scene cognitive and understanding of complex traffic semantics. This paper proposes Traj-LLM, the first to investigate the potential of using pre-trained Large Language Models (LLMs) without explicit prompt engineering to generate future motions from vehicular past trajectories and traffic scene semantics. Traj-LLM starts with sparse context joint encoding to dissect the agent and scene features into a form that LLMs understand. On this basis, we creatively explore LLMs' strong understanding capability to capture a spectrum of high-level scene knowledge and interactive information. To emulate the human-like lane focus cognitive function and enhance Traj-LLM's scene comprehension, we introduce lane-aware probabilistic learning powered by the Mamba module. Finally, a multi-modal Laplace decoder is designed to achieve scene-compliant predictions. Extensive experiments manifest that Traj-LLM, fueled by prior knowledge and understanding prowess of LLMs, together with lane-aware probability learning, transcends the state-of-the-art methods across most evaluation metrics. Moreover, the few-shot analysis serves to substantiate Traj-LLM's performance, as even with merely 50% of the dataset, it surpasses the majority of benchmarks relying on complete data utilization. This study explores endowing the trajectory prediction task with advanced capabilities inherent in LLMs, furnishing a more universal and adaptable solution for forecasting agent movements in a new way. IEEE

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Adversarial Training-Aided Time-Varying Channel Prediction for TDD/FDD systems

引用

China Communications 2023年第6期20卷 100-115页

作者： Zhen Zhang Yuxiang Zhang Jianhua Zhang Feifei Gao State Key Lab of Networking and Switching Technology Beijing University of Posts and TelecommunicationsBeijing 100876China Institute for Artificial Intelligence Tsinghua UniversityBeijing 100084China

In this paper, a time-varying channel prediction method based on conditional generative adversarial network(CPcGAN) is proposed for time division duplexing/frequency division duplexing(TDD/FDD) systems. CPc GAN utilizes a discriminator to calculate the divergence between the predicted downlink channel state information(CSI) and the real sample distributions under a conditional constraint that is previous uplink CSI. The generator of CPcGAN learns the function relationship between the conditional constraint and the predicted downlink CSI and reduces the divergence between predicted CSI and real *** capability of CPcGAN fitting data distribution can capture the time-varying and multipath characteristics of the channel well. Considering the propagation characteristics of real channel, we further develop a channel prediction error indicator to determine whether the generator reaches the best state. Simulations show that the CPcGAN can obtain higher prediction accuracy and lower system bit error rate than the existing methods under the same user speeds.

关键词： channel prediction time-varying channel conditional generative adversarial network multipath channel deep learning

来源：评论

学校读者我要写书评

暂无评论

MixCon: A Hybrid Architecture for Efficient and Adaptive Sequence Modeling 27

MixCon: A Hybrid Architecture for Efficient and Adaptive Seq...

引用

27th European Conference on Artificial intelligence, ECAI 2024

作者： Xu, Xin Lin, Zhouchen State Key Lab of General Artificial Intelligence School of Intelligence Science and Technology Peking University China Institute for Artificial Intelligence Peking University China Guangdong Guangzhou China

ISBN: (纸本)9781643685489

Sequence modeling is a critical task in various domains such as natural language processing, speech recognition, and time series analysis. The existing models still face challenges in capturing long-range dependencies and efficiently modeling sequences. This paper proposes a novel hybrid sequence modeling architecture called MixCon to address these challenges. The MixCon (Mixture of Conba) architecture combines a Transformer layer based on attention mechanism, a Conba layer, and a Mixture of Experts (MoE) module. We apply this idea to the design of the attention mechanism, achieving significant improvements in computational efficiency. Additionally, the MixCon architecture integrates feedback and adaptive control mechanism inspired by control theory, providing a new perspective and approach to sequence modeling. The experimental results demonstrate MixCon's superior throughput, outperforming Mixtral by 4.5 times and Jamba by 1.5 times when processing lengthy sequences of up to 128K tokens on a single A800 80GB GPU. Moreover, MixCon achieves top-tier scores on academic benchmarks, exemplified by its outstanding performance with a score of 87.9% on HellaSwag and 83.4% on WinoGrande, showcasing its capability to excel in complex sequence modeling tasks. © 2024 The Authors.

关键词： Adaptive control systems

来源：评论

学校读者我要写书评

暂无评论

SpikingMiniLM: energy-efficient spiking transformer for natural language understanding

引用

Science China(Information Sciences) 2024年第10期67卷 115-128页

作者： Jiayu ZHANG Jiangrong SHEN Zeke WANG Qinghai GUO Rui YAN Gang PAN Huajin TANG College of Computer Science and Technology Zhejiang University The State Key Lab of Brain-Machine Intelligence Zhejiang University Collaborative Innovation Center of Artificial Intelligence Zhejiang University Advanced Computing and Storage Laboratory Huawei Technologies Co. Ltd. College of Computer Science and Technology Zhejiang University of Technology MOE Frontier Science Center for Brain Science and Brain-Machine Integration Zhejiang University

In the era of large-scale pretrained models, artificial neural networks（ANNs） have excelled in natural language understanding（NLU） tasks. However, their success often necessitates substantial computational resources and energy consumption. To address this, we explore the potential of spiking neural networks（SNNs） in NLU——a promising avenue with demonstrated advantages, including reduced power consumption and improved efficiency due to their event-driven characteristics. We propose the SpikingMiniLM,a novel spiking Transformer model tailored for natural language understanding. We first introduce a multi-step encoding method to convert text embeddings into spike trains. Subsequently, we redesign the attention mechanism and residual connections to make our model operate on the pure spike-based paradigm without any normalization technique. To facilitate stable and fast convergence, we propose a general parameter initialization method grounded in the stable firing rate principle. Furthermore, we apply an ANN-to-SNN knowledge distillation to overcome the challenges of pretraining SNNs. Our approach achieves a macro-average score of 75.5 on the dev sets of the GLUE benchmark, retaining 98% of the performance exhibited by the teacher model MiniLMv2. Our smaller model also achieves similar performance to BERTMINIwith fewer parameters and much lower energy consumption, underscoring its competitiveness and resource efficiency in NLU tasks.

关键词： spiking neural networks natural language understanding spiking Transformer spike-based attention multi-step encoding ANN-to-SNN distillation

来源：评论

学校读者我要写书评

暂无评论

Squeezing More Past Knowledge for Online Class-Incremental Continual Learning

引用

IEEE/CAA Journal of Automatica Sinica 2023年第3期10卷 722-736页

作者： Da Yu Mingyi Zhang Mantian Li Fusheng Zha Junge Zhang Lining Sun Kaiqi Huang the State Key Laboratory of Robotics and System Harbin Institute of Technology(HIT)Harbin 150080China Center for Research on Intelligent System and Engineering Institute of AutomationChinese Academy of Sciences(CASIA)Beijing 100190 School of Artificial Intelligence University of Chinese Academy of SciencesBeijing 100049China Center for Research on Intelligent System and Engineering Institute of AutomationChinese Academy of Sciences(CASIA)Beijing 100190the School of Artificial IntelligenceUniversity of Chinese Academy of SciencesBeijing 100049 the CAS Center for Excellence in Brain Science and Intelligence Technology Shanghai 200031China IEEE

Continual learning(CL)studies the problem of learning to accumulate knowledge over time from a stream of data.A crucial challenge is that neural networks suffer from performance degradation on previously seen data,known as catastrophic forgetting,due to allowing parameter *** this work,we consider a more practical online class-incremental CL setting,where the model learns new samples in an online manner and may continuously experience new ***,prior knowledge is unavailable during training and *** works usually explore sample usages from a single dimension,which ignores a lot of valuable supervisory *** better tackle the setting,we propose a novel replay-based CL method,which leverages multi-level representations produced by the intermediate process of training samples for replay and strengthens supervision to consolidate previous ***,besides the previous raw samples,we store the corresponding logits and features in the ***,to imitate the prediction of the past model,we construct extra constraints by leveraging multi-level information stored in the *** the same number of samples for replay,our method can use more past knowledge to prevent *** conduct extensive evaluations on several popular CL datasets,and experiments show that our method consistently outperforms state-of-the-art methods with various sizes of episodic *** further provide a detailed analysis of these results and demonstrate that our method is more viable in practical scenarios.

关键词： Catastrophic forgetting class-incremental learning continual learning(CL) experience replay

来源：评论

学校读者我要写书评

暂无评论

Decomposing Visual and Semantic Correlations for Both Fully Supervised and Few-Shot Image Classification

IEEE Transactions on Artificial Intelligence

引用

IEEE Transactions on Artificial intelligence 2024年第4期5卷 1658-1668页

作者： Zhang, Chunjie Zheng, Xiaolong Beijing Jiaotong University Institute of Information Science Beijing100044 China Beijing Jiaotong University Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing100044 China University of Chinese Academy of Sciences State Key Laboratory of Multimodal Artificial Intelligence Systems The State of Key Laboratory of Management and Control for Complex System Institute of Automation Chinese Academy of Sciences School of Artificial Intelligence Beijing100190 China

Most image classification methods are designed to either boost the classification accuracies with abundant supervision, or cope with the shortage of supervision information. This is often achieved by using the visual and semantic information of other sources. However, these methods use the visual and semantic information within the same class independently, leaving their intrinsic correlations unconsidered. Objects and disturbing components as well as noise exist on the same image. Besides, semantic representations also contain noisy information. To solve the problems mentioned above, we propose a novel method for both fully supervised image classification and few-shot image classification by decomposing visual and semantic correlations. We jointly explore the intrinsic correlations of visual and semantic information of images within the same class. For each class, we decompose its visual and semantic correlations using low-rank and sparse constraint respectively. The decomposed low-rank parts character the intrinsic correlations of images can be used in a linear transformation way. Using the decomposed parts of each class, Classification can be achieved by reconstruction error minimization. We conduct experiments on several datasets for both fully supervised image classification and few-shot image classification. Experimental results well show the effectiveness of the proposed method. © 2020 IEEE.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

STUDY ON THE PROPERTIES OF COPPER-CHROMIUM ALLOY CONTACTS AFTER ELEMENTAL DOPING

STUDY ON THE PROPERTIES OF COPPER-CHROMIUM ALLOY CONTACTS AF...

引用

2024 China HVDC and Power Electronics Annual Meeting of Chinese Society of Electrical Engineering, DCPE 2024

作者： Liu, Tingting Fu, Xinye Li, Zhe Feng, Dawei School of Electrical Engineering State Key Lab of Reliability and Intelligence of Electrical Equipment Hebei University of Technology Tianjin China

ISBN: (纸本)9781837242689

With the continuous expansion and complexity of the load scale of the power grid, the stability of the power grid is facing increasingly severe challenges. To adapt to this trend, the voltage level of the grid is gradually increased, which puts forward new requirements for the physical properties of the contact in electric equipment. Therefore, this paper study the physical properties of CuCr alloy contacts with different doping elements. Firstly, according to the melting point of CuCr alloy after doping with the third element, the three doping elements W, Os, Co was screened out. Then, their doping effects on the physical properties of Cu0.5Cr0.5 were studied. The melting points and elastic properties of Cu0.5Cr0.46875W0.03125, Cu0.5Cr0.46875Os0.03125 and Cu0.5Cr0.40625Co0.09375 are superior to those of Cu0.5Cr0.5, and high melting point indicates increased resistance to sputter erosion caused by electric arcs. However, the structural stability of Cu0.5Cr0.40625Co0.09375 are obviously inferior to Cu0.5Cr0.5, which is unsatisfactory and eliminated. The comprehensive properties of Cu0.5Cr0.46875W0.03125 and Cu0.5Cr0.46875Os0.03125 are improved compared with Cu0.5Cr0.5, which are promising doped alloys. © The Institution of Engineering & technology 2024.

关键词： ABLATION RESISTANCE CONTACT CuCr ALLOY FIRST PRINCIPLES

来源：评论

学校读者我要写书评

暂无评论

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcu...

引用

2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024

作者： Yuan, Yu Zhao, Lili Zhang, Kai Zheng, Guangting Liu, Qi State Key Lab of Cognitive Intelligence University of Science and Technology of China China School of Computer Science and Technology University of Science and Technology of China China

ISBN: (纸本)9798891761643

Large Language Models (LLMs) have shown remarkable capabilities in various natural language processing tasks. However, LLMs may rely on dataset biases as shortcuts for prediction, which can significantly impair their robustness and generalization capabilities. This paper presents Shortcut Suite, a comprehensive test suite designed to evaluate the impact of shortcuts on LLMs' performance, incorporating six shortcut types, five evaluation metrics, and four prompting strategies. Our extensive experiments yield several key findings: 1) LLMs demonstrate varying reliance on shortcuts for downstream tasks, significantly impairing their performance. 2) Larger LLMs are more likely to utilize shortcuts under zero-shot and few-shot in-context learning prompts. 3) Chain-of-thought prompting notably reduces shortcut reliance and outperforms other prompting strategies, while few-shot prompts generally underperform compared to zero-shot prompts. 4) LLMs often exhibit overconfidence in their predictions, especially when dealing with datasets that contain shortcuts. 5) LLMs generally have a lower explanation quality in shortcut-laden datasets, with errors falling into three types: distraction, disguised comprehension, and logical fallacy. Our findings offer new insights for evaluating robustness and generalization in LLMs and suggest potential directions for mitigating the reliance on shortcuts. The code is available at https://***/yyhappier/***. © 2024 Association for Computational Linguistics.

关键词： Zero-shot learning

来源：评论

学校读者我要写书评

暂无评论

Microcosmic mechanism and correction of moisture effect on furfural diffusion and equilibrium in oil‐paper insulation

引用

High Voltage 2023年第4期8卷 772-780页

作者： Dawei Feng Ge Chen Yuandi Lin Ruijin Liao Dayong Yuan State Key Lab of Reliability and Intelligence of Electrical Equipment School of Electrical EngineeringHebei University of TechnologyTianjinChina The MOE Key Laboratory of Special Machine and High Voltage Apparatus School of Electrical EngineeringShenyang University of TechnologyShenyangChina State Grid Jiangsu Electric Power Company Research Institute NanjingChina State Key Laboratory of Power Transmission Equipment&System Security and New Technology School of Electrical EngineeringChongqing UniversityChongqingChina TBEA Shenyang Transformer Group Co. LtdShenyangChina

Furfural content in oil is considered as an effective parameter to evaluate the ageing state of oil-immersed *** previous thermal ageing experiments at 130℃,we found that moisture increment in oil-paper system makes more furfural distribute in ***,the microcosmic mechanism of moisture effect on furfural diffusion from paper to oil is still unknown,and the furfural equilibrium law at transformers'typical operating temperatures is also unclear,resulting in the unreliability of the existed furfural evaluation *** elucidate the above problems,molecular dynamics simulation and furfural equilibrium experiments at 60℃ were carried out at different moisture *** results show that high moisture content makes part of furfural diffuse from paper to oil *** moisture content increases from 1% to 2%,mass fraction of furfural in oil may increase from 32.98% to 49.83%,leading to furfural content in oil increases from 0.76 mg/L to 1.06 mg/L when the degree of polymerisation of insulation paper is about 524,which increased by 39.5%.The increase in moisture content makes cellulose form hydrogen bonds with H_(2)O preferentially,decreasing the hydrogen bonds interaction between cellulose and ***,partial furfural falls off cellulose and diffuses into oil,leading to an increase in furfural content in *** eliminate the moisture interference,a corrected furfural assessment model based on the moisture equilibrium curve has been constructed,which can accurately evaluate the ageing state of insulation paper regardless of the moisture influence.

关键词： equilibrium insulation diffusion

来源：评论

学校读者我要写书评

暂无评论

U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers 38

U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers

引用

38th Conference on Neural Information Processing systems, NeurIPS 2024

作者： Tian, Yuchuan Tu, Zhijun Chen, Hanting Hu, Jie Xu, Chao Wang, Yunhe State Key Lab of General AI School of Intelligence Science and Technology Peking University China Huawei Noah's Ark Lab Canada

Diffusion Transformers (DiTs) introduce the transformer architecture to diffusion tasks for latent-space image generation. With an isotropic architecture that chains a series of transformer blocks, DiTs demonstrate competitive performance and good scalability;but meanwhile, the abandonment of U-Net by DiTs and their following improvements is worth rethinking. To this end, we conduct a simple toy experiment by comparing a U-Net architectured DiT with an isotropic one. It turns out that the U-Net architecture only gain a slight advantage amid the U-Net inductive bias, indicating potential redundancies within the U-Net-style DiT. Inspired by the discovery that U-Net backbone features are low-frequency-dominated, we perform token downsampling on the query-key-value tuple for self-attention that bring further improvements despite a considerable amount of reduction in computation. Based on self-attention with downsampled tokens, we propose a series of U-shaped DiTs (U-DiTs) in the paper and conduct extensive experiments to demonstrate the extraordinary performance of U-DiT models. The proposed U-DiT could outperform DiT-XL/2 with only 1/6 of its computation cost. Codes are available at https://***/YuchuanTian/U-DiT. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：