检索结果-内蒙古大学图书馆

On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Park, Nohil Park, Joonsuk Yoo, Kang Min Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Artificial Intelligence Institute Seoul National University Korea Republic of NAVER Cloud Korea Republic of NAVER AI Lab Korea Republic of University of Richmond United States

An exciting advancement in the field of multilingual models is the emergence of autoregressive models with zero- and few-shot capabilities, a phenomenon widely reported in large-scale language models. To further improve model adaptation to cross-lingual tasks, another trend is to further fine-tune the language models with either full fine-tuning or parameter-efficient tuning. However, the interaction between parameter-efficient fine-tuning (PEFT) and cross-lingual tasks in multilingual autoregressive models has yet to be studied. Specifically, we lack an understanding of the role of linguistic distributions in multilingual models in the effectiveness of token-based prompt tuning. To address this question, we conduct experiments comparing prompt tuning and fine-tuning on the decoder-based multilingual model, XGLM, with four cross-lingual tasks (XNLI, PAWS-X, POS, NER). According to our study, prompt tuning achieves on par or better performance over fine-tuning across all languages while updating at most 0.13% of the model parameters. Moreover, we empirically show that prompt tuning is more effective in enhancing the performance of low-resource languages than fine-tuning. Our further analysis shows that the phenomenon is related to the tokenization scheme of the multilingual model. © 2023, CC BY.

关键词： Decoding

Photonic Crystals with Split Ring Unit Cells for Subwavelength Light Confinement

学校读者我要写书评

暂无评论

Photonic Crystals with Split Ring Unit Cells for Subwaveleng...

Conference on Lasers and Electro-Optics (CLEO)

作者： Kellen P. Arnold Sami I. Halimi Joshua A. Allen Shuren Hu Sharon M. Weiss Interdisciplinary Materials Science Program Vanderbilt University Nashville TN USA Department of Electrical Engineering and Computer Science Vanderbilt University Nashville TN USA

We report a split ring photonic crystal that demonstrates an order of magnitude larger peak energy density compared to traditional photonic crystals. The split ring offers highly focused optical energy in an accessibl... 详细信息

ISBN: (数字)9781957171050

ISBN: (纸本)9781665466660

关键词： Optical resonators Photonic crystals Ring lasers Electrooptical waveguides Photonics

CPO: Change Robust Panorama to Point Cloud Localization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Kim, Junho Jang, Hojun Choi, Changwoon Kim, Young Min Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence and INMC Seoul National University Korea Republic of

We present CPO, a fast and robust algorithm that localizes a 2D panorama with respect to a 3D point cloud of a scene possibly containing changes. To robustly handle scene changes, our approach deviates from conventional feature point matching, and focuses on the spatial context provided from panorama images. Specifically, we propose efficient color histogram generation and subsequent robust localization using score maps. By utilizing the unique equivariance of spherical projections, we propose very fast color histogram generation for a large number of camera poses without explicitly rendering images for all candidate poses. We accumulate the regional consistency of the panorama and point cloud as 2D/3D score maps, and use them to weigh the input color values to further increase robustness. The weighted color distribution quickly finds good initial poses and achieves stable convergence for gradient-based optimization. CPO is lightweight and achieves effective localization in all tested scenarios, showing stable performance despite scene changes, repetitive structures, or featureless regions, which are typical challenges for visual localization with perspective cameras. Code is available at https://***/82magnolia/panoramic-localization/. © 2022, CC BY.

关键词： Cameras

Ev-NeRF: Event Based Neural Radiance Field

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hwang, Inwoo Kim, Junho Kim, Young Min Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence INMC Seoul National University Korea Republic of

We present Ev-NeRF, a Neural Radiance Field derived from event data. While event cameras can measure subtle brightness changes in high frame rates, the measurements in low lighting or extreme motion suffer from significant domain discrepancy with complex noise. As a result, the performance of event-based vision tasks does not transfer to challenging environments, where the event cameras are expected to thrive over normal cameras. We find that the multi-view consistency of NeRF provides a powerful self-supervision signal for eliminating spurious measurements and extracting the consistent underlying structure despite highly noisy input. Instead of posed images of the original NeRF, the input to Ev-NeRF is the event measurements accompanied by the movements of the sensors. Using the loss function that reflects the measurement model of the sensor, Ev-NeRF creates an integrated neural volume that summarizes the unstructured and sparse data points captured for about 2-4 seconds. The generated neural volume can also produce intensity images from novel views with reasonable depth estimates, which can serve as a high-quality input to various vision-based tasks. Our results show that Ev-NeRF achieves competitive performance for intensity image reconstruction under extreme noise and high-dynamic-range imaging. © 2022, CC BY.

关键词： Cameras

On Task-Relevant Loss Functions in Meta-Reinforcement Learning and Online LQR

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shin, Jaeuk Kim, Giho Lee, Howon Han, Joonho Yang, Insoon The Department of Electrical and Computer Engineering Automation and Systems Research Institute Seoul National University Seoul08826 Korea Republic of Interdisciplinary Program in Artificial Intelligence ASRI Seoul National University Seoul08826 Korea Republic of

Designing a competent meta-reinforcement learning (meta-RL) algorithm in terms of data usage remains a central challenge to be tackled for its successful real-world applications. In this paper, we propose a sample-efficient meta-RL algorithm that learns a model of the system or environment at hand in a task-directed manner. As opposed to the standard model-based approaches to meta-RL, our method exploits the value information in order to rapidly capture the decision-critical part of the environment. The key component of our method is the loss function for learning the task inference module and the system model that systematically couples the model discrepancy and the value estimate, thereby facilitating the learning of the policy and the task inference module with a significantly smaller amount of data compared to the existing meta-RL algorithms. The idea is also extended to a non-meta-RL setting, namely an online linear quadratic regulator (LQR) problem, where our method can be simplified to reveal the essence of the strategy. The proposed method is evaluated in high-dimensional robotic control and online LQR problems, empirically verifying its effectiveness in extracting information indispensable for solving the tasks from observations in a sample efficient manner. Copyright © 2023, The Authors. All rights reserved.

关键词： Reinforcement learning

ENTROPY IS NOT ENOUGH FOR TEST-TIME ADAPTATION: FROM THE PERSPECTIVE OF DISENTANGLED FACTORS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lee, Jonghyun Jung, Dahuin Lee, Saehyung Park, Junsung Shin, Juhyeon Hwang, Uiwon Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of School of Computer Science and Engineering Soongsil University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Division of Digital Healthcare Yonsei University Korea Republic of AIIS ASRI INMC and ISRC Seoul National University Korea Republic of

Test-time adaptation (TTA) fine-tunes pre-trained deep neural networks for unseen test data. The primary challenge of TTA is limited access to the entire test dataset during online updates, causing error accumulation. To mitigate it, TTA methods have utilized the model output’s entropy as a confidence metric that aims to determine which samples have a lower likelihood of causing error. Through experimental studies, however, we observed the unreliability of entropy as a confidence metric for TTA under biased scenarios and theoretically revealed that it stems from the neglect of the influence of latent disentangled factors of data on predictions. Building upon these findings, we introduce a novel TTA method named Destroy Your Object (DeYO), which leverages a newly proposed confidence metric named Pseudo-Label Probability Difference (PLPD). PLPD quantifies the influence of the shape of an object on prediction by measuring the difference between predictions before and after applying an object-destructive transformation. DeYO consists of sample selection and sample weighting, which employ entropy and PLPD concurrently. For robust adaptation, DeYO prioritizes samples that dominantly incorporate shape information when making predictions. Our extensive experiments demonstrate the consistent superiority of DeYO over baseline methods across various scenarios, including biased and wild. Project page is publicly available at https://***/DeYO/. Copyright © 2024, The Authors. All rights reserved.

关键词： Entropy

On the Impact of Knowledge Distillation for Model Interpretability

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Han, Hyeongrok Kim, Siwon Choi, Hyun-Soo Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Seoul Korea Republic of Department of Computer Science and Engineering Seoul National University of Science and Technology Seoul Korea Republic of ZIOVISION Inc. Chuncheon Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Seoul Korea Republic of

Several recent studies have elucidated why knowledge distillation (KD) improves model performance. However, few have researched the other advantages of KD in addition to its improving model performance. In this study, we have attempted to show that KD enhances the interpretability as well as the accuracy of models. We measured the number of concept detectors identified in network dissection for a quantitative comparison of model interpretability. We attributed the improvement in interpretability to the class-similarity information transferred from the teacher to student models. First, we confirmed the transfer of class-similarity information from the teacher to student model via logit distillation. Then, we analyzed how class-similarity information affects model interpretability in terms of its presence or absence and degree of similarity information. We conducted various quantitative and qualitative experiments and examined the results on different datasets, different KD methods, and according to different measures of interpretability. Our research showed that KD models by large models could be used more reliably in various fields. © 2023, CC BY-NC-SA.

关键词： Distillation

Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ahn, Hongjoon Hyeon, Jinu Oh, Youngmin Hwang, Bosun Moon, Taesup Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Samsung Advanced Institute of Technology Korea Republic of ASRI INMC IPAI/AIIS Seoul National University Korea Republic of

We argue that one of the main obstacles for developing effective Continual Reinforcement Learning (CRL) algorithms is the negative transfer issue occurring when the new task to learn arrives. Through comprehensive experimental validation, we demonstrate that such issue frequently exists in CRL and cannot be effectively addressed by several recent work on mitigating plasticity loss of RL agents. To that end, we develop Reset & Distill (R&D), a simple yet highly effective method, to overcome the negative transfer problem in CRL. R&D combines a strategy of resetting the agent’s online actor and critic networks to learn a new task and an offline learning step for distilling the knowledge from the online actor and previous expert’s action probabilities. We carried out extensive experiments on long sequence of Meta-World tasks and show that our method consistently outperforms recent baselines, achieving significantly higher success rates across a range of tasks. Our findings highlight the importance of considering negative transfer in CRL and emphasize the need for robust strategies like R&D to mitigate its detrimental effects. © 2024, CC BY.

关键词： Reinforcement learning

Advancements in UAV-based Integrated Sensing and Communication: A Comprehensive Survey

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ahmed, Manzoor Nasir, Ali Arshad Masood, Mudassir Memon, Kamran Ali Qureshi, Khurram Karim Khan, Feroz Khan, Wali Ullah Xu, Fang Han, Zhu School of Computer and Information Science Institute for AI Industrial Technology Research Hubei Engineering University Xiaogan432000 China Interdisciplinary Research Center for Communication Systems and Sensing King Fahd University of Petroleum and Minerals Dhahran31261 Saudi Arabia Department of Electrical Engineering King Fahd University of Petroleum and Minerals Dhahran31261 Saudi Arabia School of Electronic Engineering Beijing University of Posts and Telecommunications Beijing China University of Luxembourg Luxembourg City1855 Luxembourg Department of Electrical and Computer Engineering The University of Houston HoustonTX77004 United States Department of Computer Science and Engineering Kyung Hee University Seoul446-701 Korea Republic of

Unmanned aerial vehicle (UAV)-based integrated sensing and communication (ISAC) systems are poised to revolutionize next-generation wireless networks by enabling simultaneous sensing and communication (S&C). This survey comprehensively reviews UAV-ISAC systems, highlighting foundational concepts, key advancements, and future research directions. We explore recent advancements in UAV-based ISAC systems from various perspectives and objectives, including advanced channel estimation (CE), beam tracking, and system throughput optimization under joint sensing and communication S&C constraints. Additionally, we examine weighted sum rate (WSR) and sensing trade-offs, delay and age of information (AoI) minimization, energy efficiency (EE), and security enhancement. These applications highlight the potential of UAV-based ISAC systems to improve spectrum utilization, enhance communication reliability, reduce latency, and optimize energy consumption across diverse domains, including smart cities, disaster relief, and defense operations. The survey also features summary tables for comparative analysis of existing methodologies, emphasizing performance, limitations, and effectiveness in addressing various challenges. By synthesizing recent advancements and identifying open research challenges, this survey aims to be a valuable resource for developing efficient, adaptive, and secure UAV-based ISAC systems. Copyright © 2025, The Authors. All rights reserved.

关键词： Unmanned aerial vehicles (UAV)