检索结果-内蒙古大学图书馆

BWIN: A BILATERAL WARPING METHOD FOR VIDEO FRAME INTERPOLATION

学校读者我要写书评

暂无评论

BWIN: A BILATERAL WARPING METHOD FOR VIDEO FRAME INTERPOLATI...

2021 IEEE International Conference on Multimedia and Expo, ICME 2021

作者： Xue, Fanyong Li, Jie Liu, Jiannan Wu, Chentao Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

ISBN: (纸本)9781665438643

Flow-based video frame interpolation approaches typically adopt forward or backward warping to approximate the intermediate frames. And a synthesis network is used to refine the interpolation results. Optical flows indicate motion between two input frames, but both forward and backward warping only utilize the first frame. In this work, we propose bilateral warping to make full use of optical flows. Specifically, the proposed bilateral warping yields intermediate candidates from not only the first frame but also the second frame. Our model first applies bilateral warping on the input frames and contextual features. Then, we add skip connections from the input frames and contextual features to the synthesis network. Finally, the synthesis network generates the interpolation results by integrating the original and warped representations. The experimental results on a wide variety of datasets demonstrate the superiority of the proposed approach over the state-of-the-art video frame interpolation methods. © 2021 IEEE

关键词： Optical flows

Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zeng, Hongchuan Han, Senyu Chen, Lu Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China Suzhou Laboratory Suzhou China

Large language models (LLMs) have demonstrated remarkable performance, particularly in multilingual contexts. While recent studies suggest that LLMs can transfer skills learned in one language to others, the internal mechanisms behind this ability remain unclear. We observed that the neuron activation patterns of LLMs exhibit similarities when processing the same language, revealing the existence and location of key linguistic regions. Additionally, we found that neuron activation patterns are similar when processing sentences with the same semantic meaning in different languages. This indicates that LLMs map semantically identical inputs from different languages into a "Lingua Franca", a common semantic latent space that allows for consistent processing across languages. This semantic alignment becomes more pronounced with training and increased model size, resulting in a more language-agnostic activation pattern. Moreover, we found that key linguistic neurons are concentrated in the first and last layers of LLMs, becoming denser in the first layers as training progresses. Experiments on BLOOM and LLaMA2 support these findings, highlighting the structural evolution of multilingual LLMs during training and scaling up. This paper provides insights into the internal workings of LLMs, offering a foundation for future improvements in their cross-lingual capabilities. The codes are available at: https://***/X-LANCE/LinguaFranca. © 2024, CC BY.

关键词： Neurons

PS4: A Low Power SNN Accelerator with Spike Speculative Scheme

学校读者我要写书评

暂无评论

PS4: A Low Power SNN Accelerator with Spike Speculative Sche...

IEEE International Conference on computer Design: VLSI in computers and Processors, (ICCD)

作者： Zongwu Wang Fangxin Liu Xin Tang Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai Qi Zhi Institute MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

ISBN: (数字)9798350380408

ISBN: (纸本)9798350380415

Spiking neural networks (SNNs) offer computational and energy efficiency advantages over traditional artificial neural networks (ANNs) due to their event-driven representations. Unlike ANNs, which use continuous activation values, SNNs transmit information through spikes-binary events represented as either 0 or 1. This characteristic allows SNNs to replace weight multiplications in ANNs with additions, resulting in a more energy-efficient and less computationally demanding implementation. However, current SNN algorithms prioritize high accuracy and large sparsity by employing complex neuron models with sparse spike generation. This approach tends to compromise energy efficiency and increase latency. On the other hand, existing SNN hardware designs struggle to jointly exploit high parallel processing dataflows and the inherent large sparsity of (dynamic) spikes, due to the unpredictable sparsity patterns and time-dep.ndency of these spikes. To address these issues, this paper proposes PS4, an algorithm-hardware co-design framework. PS4 exploits the inherently rich sparsity in SNN spike activity using a spatial architecture for high energy efficiency and low latency without compromising accuracy. The key insight of PS4 is selectively merging multiple time steps into single-shot computations based on output (latent) sparsity, enabling speculative fast forwarding by skipping iterative spatiotemporal computations across multiple time steps where nothing happens (i.e., output spike is zero). PS4 incorporates lightweight popcount-based circuits to efficiently handle merged time steps, maximizing spike sparsity utilization and hardware parallelism. Thus, the unexplored sparsity of output spikes can be efficiently exploited to achieve highly parallel and energy-efficient computation, with very low overheads. This enables a hardware-efficient PS4 sparsity-exploiting design, which can be efficiently integrated into existing hardware accelerators like systolic arrays. Evaluatio

关键词： Accuracy Heuristic algorithms computer architecture Spiking neural networks Parallel processing Energy efficiency Systolic arrays Computational efficiency Spatiotemporal phenomena Iterative methods

Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zeng, Hongchuan Xu, Hongshen Chen, Lu Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China Suzhou Laboratory Suzhou China

Large Language Models (LLMs) have ushered in a new era in Natural Language Processing, but their massive size demands effective compression techniques for practicality. Although numerous model compression techniques have been investigated, they typically rely on a calibration set that overlooks the multilingual context and results in significant accuracy degradation for low-resource languages. This paper introduces Multilingual Brain Surgeon (MBS), a novel calibration data sampling method for multilingual LLMs compression. MBS overcomes the English-centric limitations of existing methods by sampling calibration data from various languages proportionally to the language distribution of the model training datasets. Our experiments, conducted on the BLOOM multilingual LLM, demonstrate that MBS improves the performance of existing English-centric compression methods, especially for low-resource languages. We also uncover the dynamics of language interaction during compression, revealing that the larger the proportion of a language in the training set and the more similar the language is to the calibration language, the better performance the language retains after compression. In conclusion, MBS presents an innovative approach to compressing multilingual LLMs, addressing the performance disparities and improving the language inclusivity of existing compression techniques. The codes are available at: https://***/XLANCE/MBS. © 2024, CC BY.

关键词： Calibration

CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Hanchong Cao, Ruisheng Xu, Hongshen Chen, Lu Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China Suzhou Laboratory Suzhou China

Recently, Large Language Models (LLMs) have been demonstrated to possess impressive capabilities in a variety of domains and tasks. We investigate the issue of prompt design in the multi-turn text-to-SQL task and attempt to enhance the LLMs' reasoning capacity when generating SQL queries. In the conversational context, the current SQL query can be modified from the preceding SQL query with only a few operations due to the context dep.ndency. We introduce our method called CoE-SQL1 which can prompt LLMs to generate the SQL query based on the previously generated SQL query with an edition chain. We also conduct extensive ablation studies to determine the optimal configuration of our approach. Our approach outperforms different in-context learning baselines stably and achieves state-of-the-art performances on two benchmarks SParC and CoSQL using LLMs, which is also competitive to the SOTA fine-tuned models. © 2024, CC BY-SA.

关键词：

HOLES: Boosting Large Language Models Efficiency with Hardware-Friendly Lossless Encoding

学校读者我要写书评

暂无评论

HOLES: Boosting Large Language Models Efficiency with Hardwa...

IEEE International Conference on computer Design: VLSI in computers and Processors, (ICCD)

作者： Fangxin Liu Ning Yang Zhiyan Song Zongwu Wang Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai Qi Zhi Institute MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

ISBN: (数字)9798350380408

ISBN: (纸本)9798350380415

Transformer-based large language models (LLMs) have demonstrated remarkable success; however, their increasing model size poses a challenge due to the widening gap between model size and hardware capacity. To address this, model com-pression techniques have been proposed, but existing compression methods struggle to effectively handle the significant parameter variations (activations and weights) within the model. More-over, current outlier-aware encoding solutions for compression introduce complex logic, leading to limited compression benefits and hardware efficiency. In this context, we present HOLES, a novel algorithm/architecture co-designed solution that utilizes variable-length data representation and metadata for local pa-rameter value processing, offering low hardware overhead and high-performance gains. Our key insight is that only a tiny fraction of activations are outliers demanding high precision representation. This observation enables us to exploit the sparsity of high significant bits within data representation, allowing us to employ an identifier bit to assign the appropriate encoding length, thereby eliminating redundant bit-length footprints. This reduction in data representation based on data characteristics enables a serialized structured data encoding scheme that seam-lessly integrates with existing hardware accelerators, such as systolic arrays. We evaluate HOLES-based accelerators against some existing encoding-based accelerator, and our results demon-strate significant improvements. The HOLES-based accelerator achieves up to 3.98 × speedup and 70.0% energy reduction, while maintaining superior model accuracy.

关键词： Accuracy Quantization (signal) Large language models Data visualization Metadata Transformers Encoding Systolic arrays Inference algorithms Hardware acceleration

BARTENDER: A simple baseline model for task-level heterogeneous federated learning

学校读者我要写书评

暂无评论

BARTENDER: A simple baseline model for task-level heterogene...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Yuwen Yang Yuxiang Lu Suizhi Huang Shalayiding Sirejiding Chang Liu Muyang Yi Zhaozhi Xie Yue Ding Hongtao Lu Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350390155

ISBN: (纸本)9798350390162

This study presents the Task-level Heterogeneous Federated Learning (TH-FL), a novel paradigm that fuses the principles of Federated Learning (FL) and Multi-Task Learning (MTL). In the TH-FL scenario, each client can learn an indefinite number of tasks, which may vary in type and originate from distinct domains. We introduce a unique baseline model, BARTENDER, that integrates a Conditional Prompt (CP) module. This module encodes task-specific and domain-specific information, enabling the model to generate tailored outputs based on the encoding inputs. This innovative strategy not only minimizes the communication costs associated with FL but also enhances model generalization across a variety of task types. Through extensive experiments, we establish that the BARTENDER model surpasses traditional multi-decoder architecture models across diverse scenarios. We also explore the influence of the parameter decoupling strategy on model training and outline the assumptions necessary for achieving a $O\left( {1/\sqrt T } \right)$ convergence speed in the TH-FL scenario.

关键词： Training Costs Federated learning Fuses Multitasking Encoding Convergence

COMPASS: SRAM-Based Computing-in-Memory SNN Accelerator with Adaptive Spike Speculation

学校读者我要写书评

暂无评论

COMPASS: SRAM-Based Computing-in-Memory SNN Accelerator with...

IEEE/ACM International Symposium on Microarchitecture (MICRO)

作者： Zongwu Wang Fangxin Liu Ning Yang Shiyuan Huang Haomin Li Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai Qi Zhi Institute MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

ISBN: (数字)9798350350579

ISBN: (纸本)9798350350586

Brain-inspired spiking neural networks (SNNs) are considered energy-efficient alternatives to conventional deep neural networks (DNNs). By adopting event-driven information processing, SNNs can significantly reduce the computational demands associated with DNNs, while still achieving comparable performance. However, current SNNs primarily prioritize high accuracy by constructing complex neuron models that generate sparse spikes. Unfortunately, this approach results in low energy efficiency and high latency, posing a significant challenge for dep.oying SNNs at the edge. Furthermore, the dominant computation in SNNs, which involves spike-wise Accumulate-Compare operations, is well-suited for Computing-in-Memory (CIM) architectures. However, exploiting high parallel processing and spike sparsity in CIM-based SNN accelerators is challenging due to the irregularity and time dep.ndency of spikes. To address these limitations, the paper proposes COMPASS, a SRAM-based CIM architecture for efficient SNNs. We first introduce an efficient method to exploit irregular sparsity for both input spikes (explicit) and output spikes (implicit). This is achieved through a speculation mechanism that exploit dynamic spike patterns, enabling lean hardware for sparsity utilization. Additionally, the CIM architecture is carefully modified to facilitate dynamic spike pattern generation and exploitation with minimal overhead. Moreover, we design an adaptive dataflow with temporal spike representation tailored for input/output spikes, reducing memory footprint and enabling parallel execution. Comprehensive evaluation results demonstrate that COMPASS can achieve 26.7x end-to-end speedup over recent SNN accelerators hardware implementation with up to 386.7x less energy per inference.

关键词： Accuracy Neurons computer architecture Spiking neural networks Parallel processing In-memory computing Energy efficiency Hardware Spatiotemporal phenomena Compass

Alignment for Efficient Tool Calling of Large Language Models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Xu, Hongshen Wang, Zihan Zhu, Zichen Pan, Lei Chen, Xingyu Chen, Lu Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China AISpeech Co. Ltd. Suzhou China

Recent advancements in tool learning have enabled large language models (LLMs) to integrate external tools, enhancing their task performance by expanding their knowledge boundaries. However, relying on tools often introduces trade-offs between performance, speed, and cost, with LLMs sometimes exhibiting overreliance and overconfidence in tool usage. This paper addresses the challenge of aligning LLMs with their knowledge boundaries to make more intelligent decisions about tool invocation. We propose a multi-objective alignment framework that combines probabilistic knowledge boundary estimation with dynamic decision-making, allowing LLMs to better assess when to invoke tools based on their confidence. Our framework includes two methods for knowledge boundary estimation—consistency-based and absolute estimation—and two training strategies for integrating these estimates into the model’s decision-making process. Experimental results on various tool invocation scenarios demonstrate the effectiveness of our framework, showing significant improvements in tool efficiency by reducing unnecessary tool usage. Copyright © 2025, The Authors. All rights reserved.

关键词： Adversarial machine learning