检索结果-内蒙古大学图书馆

Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yu, Sangwon Song, Jongyoon Hwang, Bongkyu Kang, Hoyoung Cho, Sooah Choi, Junhwa Joe, Seongho Lee, Taehee Gwon, Youngjune L. Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Samsung SDS Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of

A binary decision task, like yes-no questions or answer verification, reflects a significant real-world scenario such as where users look for confirmation about the correctness of their decisions on specific issues. In this work, we observe that language models exhibit a negative bias in the binary decisions of complex reasoning tasks. Based on our observations and the rationale about attention-based model dynamics, we propose a negative attention score (NAS) to systematically and quantitatively formulate negative bias. Based on NAS, we identify attention heads that attend to negative tokens provided in the instructions as answer candidate of binary decisions, regardless of the question in the prompt, and validate their association with the negative bias. Additionally, we propose the negative attention score alignment (NASA) method, which is a parameter-efficient fine-tuning technique to address the extracted negatively biased attention heads. Experimental results from various domains of reasoning tasks and large model search space demonstrate that NASA significantly reduces the gap between precision and recall caused by negative bias while preserving their generalization abilities. Our codes are available at https://***/ysw1021/NASA. Copyright © 2024, The Authors. All rights reserved.

关键词： NASA

On the Powerfulness of Textual Outlier Exposure for Visual OoD Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Park, Sangha Mok, Jisoo Jung, Dahuin Lee, Saehyung Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of

Successful detection of Out-of-Distribution (OoD) data is becoming increasingly important to ensure safe deployment of neural networks. One of the main challenges in OoD detection is that neural networks output overconfident predictions on OoD data, make it difficult to determine OoD-ness of data solely based on their predictions. Outlier exposure addresses this issue by introducing an additional loss that encourages low-confidence predictions on OoD data during training. While outlier exposure has shown promising potential in improving OoD detection performance, all previous studies on outlier exposure have been limited to utilizing visual outliers. Drawing inspiration from the recent advancements in vision-language pre-training, this paper venture out to the uncharted territory of textual outlier exposure. First, we uncover the benefits of using textual outliers by replacing real or virtual outliers in the image-domain with textual equivalents. Then, we propose various ways of generating preferable textual outliers. Our extensive experiments demonstrate that generated textual outliers achieve competitive performance on large-scale OoD and hard OoD benchmarks. Furthermore, we conduct empirical analyses of textual outliers to provide primary criteria for designing advantageous textual outliers: near-distribution, descriptiveness, and inclusion of visual semantics. © 2023, CC BY-NC-SA.

关键词： Forecasting

Development of LiFx/Al2O3/ITO Multilayer Antireflection Coatings for Reflectance Reduction in Silicon Heterojunction Solar Cells

学校读者我要写书评

暂无评论

Physica Status Solidi (A) Applications and Materials Science 2025年第0期

作者： Aida, Maha Nur Khokhar, Muhammad Quddamah Yousuf, Hasnain Chu, Mengmeng Rahman, Rafi Ur Alamgeer, Khan Bae, Junhan Jang, Seokjin Yi, Junsin Department of Future Energy Engineering Sungkyunkwan University Gyeonggi-Do Suwon16419 Korea Republic of Department of Electrical and Computer Engineering Sungkyunkwan University Gyeonggi-Do Suwon16419 Korea Republic of Interdisciplinary Program in Photovoltaic System Engineering Sungkyunkwan University Gyeonggi-Do Suwon16419 Korea Republic of

This study focuses on the optimization of antireflection coatings (ARCs) to enhance the performance of silicon heterojunction (SHJ) solar cells. SHJ solar cells face a significant challenge in achieving their theoretical efficiency limits due to light reflection at the air–cell interface. This reflection reduces photon absorption, which limits photocurrent generation and hinders the widespread adoption of SHJ technology. To address this, the optimization of ARCs using single, double, and triple-layer designs with a graded refractive index material as the active layer is investigated. The optimization process is simulated by calculating the layer thicknesses using OPAL2 software and validated through experimental deposition. The results demonstrate that a triple-layer ARC structure consisting of 80/50/95 nm of indium tin oxide (ITO), aluminum oxide (Al2O3), and lithium fluoride (LiFx) significantly improves SHJ performance. This configuration leads to a 1.04 mA cm−2 increase in short-circuit current density (Jsc), reaching 39.85 mA cm−2, and an efficiency of 21.62%, compared to a single-layer ARC. This study presents a promising solution to improve SHJ solar cell efficiency by reducing reflection losses and optimizing optical properties, contributing to the advancement of SHJ technology and facilitating the development of more efficient and cost-effective photovoltaic systems for sustainable energy transition. © 2025 Wiley-VCH GmbH.

关键词： Antireflection coatings

Neural Diffeomorphic Non-uniform B-spline Flows

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hong, Seongmin Chun, Se Young Department of Electrical and Computer Engineering Seoul National University Korea Republic of INMC Interdisciplinary Program in AI Seoul National University Korea Republic of

Normalizing flows have been successfully modeling a complex probability distribution as an invertible transformation of a simple base distribution. However, there are often applications that require more than invertibility. For instance, the computation of energies and forces in physics requires the second derivatives of the transformation to be well-defined and continuous. Smooth normalizing flows employ infinitely differentiable transformation, but with the price of slow non-analytic inverse transforms. In this work, we propose diffeomorphic non-uniform B-spline flows that are at least twice continuously differentiable while bi-Lipschitz continuous, enabling efficient parametrization while retaining analytic inverse transforms based on a sufficient condition for diffeomorphism. Firstly, we investigate the sufficient condition for Ck−2-diffeomorphic non-uniform kth-order B-spline transformations. Then, we derive an analytic inverse transformation of the non-uniform cubic B-spline transformation for neural diffeomorphic non-uniform B-spline flows. Lastly, we performed experiments on solving the force matching problem in Boltzmann generators, demonstrating that our C2-diffeomorphic non-uniform B-spline flows yielded solutions better than previous spline flows and faster than smooth normalizing flows. Our source code is publicly available at https://***/smhongok/Non-uniform-B-spline-Flow. © 2023, CC BY-NC-ND.

关键词： Interpolation

FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networks 22

学校读者我要写书评

暂无评论

FedClassAvg: Local Representation Learning for Personalized ...

Proceedings of the 51st International Conference on Parallel Processing

作者： Jaehee Jang Heoneok Ha Dahuin Jung Sungroh Yoon Department of Electrical and Computer Engineering Seoul National University Republic of Korea Department of Electrical and Computer Engineering and Interdisciplinary Program in Artificial Intelligence Seoul National University Republic of Korea and AIIS ASRI and INMC Seoul National University Republic of Korea

ISBN: (纸本)9781450397339

Personalized federated learning is aimed at allowing numerous clients to train personalized models while participating in collaborative training in a communication-efficient manner without exchanging private data. However, many personalized federated learning algorithms assume that clients have the same neural network architecture, and those for heterogeneous models remain understudied. In this study, we propose a novel personalized federated learning method called federated classifier averaging (FedClassAvg). Deep neural networks for supervised learning tasks consist of feature extractor and classifier layers. FedClassAvg aggregates classifier weights as an agreement on decision boundaries on feature spaces so that clients with not independently and identically distributed (non-iid) data can learn about scarce labels. In addition, local feature representation learning is applied to stabilize the decision boundaries and improve the local feature extraction capabilities for clients. While the existing methods require the collection of auxiliary data or model weights to generate a counterpart, FedClassAvg only requires clients to communicate with a couple of fully connected layers, which is highly communication-efficient. Moreover, FedClassAvg does not require extra optimization problems such as knowledge transfer, which requires intensive computation overhead. We evaluated FedClassAvg through extensive experiments and demonstrated it outperforms the current state-of-the-art algorithms on heterogeneous personalized federated learning tasks.

关键词： Communication Efficient Resource Constraint Representation Learning Neural Networks Model Heterogeneity Federated Learning

Unsupervised Homography Estimation on Multimodal Image Pair via Alternating Optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Song, Sanghyeob Lew, Jaihyun Jang, Hyemi Yoon, Sungroh Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Department of Electrical and Computer Engineering Seoul National University Korea Republic of Samsung Electro-Mechanics Korea Republic of

Estimating the homography between two images is crucial for mid- or high-level vision tasks, such as image stitching and fusion. However, using supervised learning methods is often challenging or costly due to the difficulty of collecting ground-truth data. In response, unsupervised learning approaches have emerged. Most early methods, though, assume that the given image pairs are from the same camera or have minor lighting differences. Consequently, while these methods perform effectively under such conditions, they generally fail when input image pairs come from different domains, referred to as multimodal image pairs. To address these limitations, we propose AltO, an unsupervised learning framework for estimating homography in multimodal image pairs. Our method employs a two-phase alternating optimization framework, similar to Expectation-Maximization (EM), where one phase reduces the geometry gap and the other addresses the modality gap. To handle these gaps, we use Barlow Twins loss for the modality gap and propose an extended version, Geometry Barlow Twins, for the geometry gap. As a result, we demonstrate that our method, AltO, can be trained on multimodal datasets without any ground-truth data. It not only outperforms other unsupervised methods but is also compatible with various architectures of homography estimators. The source code can be found at: https://***/songsang7/AltO Copyright © 2024, The Authors. All rights reserved.

关键词： Geometry

2D Polynomial Regression Modeling for Predictive Analysis of Glucose Concentration in NP-Model Fractal Microwave Sensor Using Frequency and Transmission Coefficient Disparities

学校读者我要写书评

暂无评论

2D Polynomial Regression Modeling for Predictive Analysis of...

International electrical engineering Congress (iEECON)

作者： Nutyada Nantapanich Pisit Mahatthanajatuphat Norakamon Wongsin Wilaiporn Lee Department of Electronics and Telecommunication Engineering Faculty of Engineering Rajamangala University of Technology Thanyaburi Pathum Thani Thailand Regular Program in Mathematics-Science Yothinburana School Bangkok Thailand Department of Electrical and Computer Engineering Faculty of Engineering King Mongkut's University of Technology North Bangkok Bangkok Thailand

ISBN: (数字)9798331543952

ISBN: (纸本)9798331543969

The article introduces a two-dimensional polynomial regression model for the predictive analysis of glucose concentration in a fractal microwave sensor NP model, utilizing frequency and transmission coefficient differences to examine their correlation with glucose concentration levels. The sensor employed in this investigation is fabricated on an economical FR-4 substrate to minimize costs. The sensor structure is developed with the fractal structure reduction method established by the NP model to operate at a frequency of roughly 2.97 GHz. The CST Studio software is utilized to analyze the findings from the simulation of the suggested sensor structure. Furthermore, the proposed sensor is integrated into a test apparatus comprising a sensor platform and a liquid tube constructed from PPA material. The liquid under examination is a glucose and water mixture at varying concentrations. It was analyzed using a VNA instrument to evaluate its frequency response and the magnitude of the alteration in the transmission coefficient. The values are utilized to create a two-dimensional polynomial regression model. The experiments demonstrated that the proposed sensor model possesses an Adjusted R² value of 0.99499 and an average error of approximately 3.57% for glucose concentration analysis throughout the range of 5% to 30%, with sensitivities of 0.00879 dB/% for insertion loss and 0.092 MHz/% for frequency variation, facilitating accurate glucose concentration prediction analysis.

关键词： Analytical models Adaptation models Correlation Liquids Predictive models Microwave sensors Polynomials Fractals Glucose Predictive analytics

Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Hwang, Inwoo Kim, Hyeonwoo Kim, Young Min Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence and INMC Seoul National University Korea Republic of

We propose Text2Scene, a method to automatically create realistic textures for virtual scenes composed of multiple objects. Guided by a reference image and text descriptions, our pipeline adds detailed texture on labeled 3D geometries in the room such that the generated colors respect the hierarchical structure or semantic parts that are often composed of similar materials. Instead of applying flat stylization on the entire scene at a single step, we obtain weak semantic cues from geometric segmentation, which are further clarified by assigning initial colors to segmented parts. Then we add texture details for individual objects such that their projections on image space exhibit feature embedding aligned with the embedding of the input. The decomposition makes the entire pipeline tractable to a moderate amount of computation resources and memory. As our framework utilizes the existing resources of image and text embedding, it does not require dedicated datasets with high-quality textures designed by skillful artists. To the best of our knowledge, it is the first practical and scalable approach that can create detailed and realistic textures of the desired style that maintain structural context for scenes with multiple objects. © 2023, CC BY.

关键词： Textures

2D Polynomial Regression Modeling for Predictive Analysis of Glucose Concentration in NP-Model Fractal Microwave Sensor Using Frequency and Transmission Coefficient Disparities

学校读者我要写书评

暂无评论

2D Polynomial Regression Modeling for Predictive Analysis of...

13th International electrical engineering Congress, iEECON 2025

作者： Nantapanich, Nutyada Mahatthanajatuphat, Pisit Wongsin, Norakamon Lee, Wilaiporn Rajamangala University of Technology Thanyaburi Faculty of Engineering Department of Electronics and Telecommunication Engineering Pathum Thani Thailand Yothinburana School Regular Program in Mathematics-Science Bangkok Thailand King Mongkut's University of Technology North Bangkok Faculty of Engineering Department of Electrical and Computer Engineering Bangkok Thailand

ISBN: (纸本)9798331543952

关键词： Polynomial regression