检索结果-内蒙古大学图书馆

Outlier-Efficient Hopfield Layers for Large Transformer-Based Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hu, Jerry Yao-Chieh Chang, Pei-Hsuan Luo, Haozheng Chen, Hong-Yu Li, Weijian Wang, Wei-Po Liu, Han Department of Computer Science Northwestern University Evanston United States Department of Physics National Taiwan University Taipei Taiwan Department of Statistics and Data Science Northwestern University Evanston United States

We introduce an Outlier-Efficient Modern Hopfield Model (termed OutEffHop) and use it to address the outlier inefficiency problem of training gigantic transformer-based models. Our main contribution is a novel associative memory model facilitating outlier-efficient associative memory retrievals. Interestingly, this memory model manifests a model-based interpretation of an outlier-efficient attention mechanism (Softmax1): it is an approximation of the memory retrieval process of OutEffHop. Methodologically, this allows us to introduce novel outlier-efficient Hopfield layers as powerful alternatives to traditional attention mechanisms, with superior post-quantization performance. Theoretically, the Outlier-Efficient Modern Hopfield Model retains and improves the desirable properties of standard modern Hopfield models, including fixed point convergence and exponential storage capacity. Empirically, we demonstrate the efficacy of the proposed model across large-scale transformer-based and Hopfield-based models (including BERT, OPT, ViT, and STanHop-Net), benchmarking against state-of-the-art methods like Clipped Softmax and Gated Attention. Notably, OutEffHop achieves an average reduction of 22+% in average kurtosis and 26+% in the maximum infinity norm of model outputs across four models. Code is available at GitHub;future updates are on arXiv. © 2024, CC BY-NC-SA.

关键词： Associative processing

Exploring Automated Feature Engineering for Energy Consumption Forecasting with AutoML

学校读者我要写书评

暂无评论

Exploring Automated Feature Engineering for Energy Consumpti...

2024 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2024

作者： Alkhulaifi, Nasser Bowler, Alexander L. Pekaslan, Direnc Triguero, Isaac Watson, Nicholas J. School of Food Science and Nutrition University of Leeds Faculty of Environment LeedsLS2 9JT United Kingdom NottinghamNG8 1BB United Kingdom University of Granada Department of Computer Science and Artificial Intelligence Spain Lab NottinghamNG8 1BB United Kingdom DaSCI Andalusian Institute in Data Science and Computational Intelligence Granada Spain

ISBN: (纸本)9781665410205

Machine learning methods are widely used to predict energy consumption, aiming to enhance efficiency and support environmental goals. However, developing these models is traditionally time-consuming and expert-dependent. While Automated Machine Learning (AutoML) has emerged as a valuable approach to streamlining machine learning pipelines, including appropriate preprocessing and learning algorithms, it may still require human experts. These experts might be needed to generate new, interpretable features that could significantly improve model performance. This is particularly relevant in complex settings such as the energy domain, where deep learning's automatic feature extraction lacks interpretability. To address this challenge, this exploratory work introduces an automated feature engineering method tailored for energy forecasting problems. It involves generating a comprehensive set of features that can be fed into AutoML, thereby reducing the need for domain knowledge in feature engineering. The proposed method has been validated using eleven datasets from various energy domains, including residential buildings, renewable energy, and regional energy consumption, with state-of-the-art AutoML methods, namely H20, TPOT, AutoGluon, and FLAML. The results demonstrate a noticeable reduction in prediction errors across all the examined datasets. © 2024 IEEE.

关键词： Adversarial machine learning

Inferring Dynamic Networks from Marginals with Iterative Proportional Fitting

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chang, Serina Koehler, Frederic Qu, Zhaonan Leskovec, Jure Ugander, Johan Department of Computer Science Stanford University United States Department of Statistics Data Science Institute University of Chicago United States Department of Economics Stanford University United States Department of Management Science & Engineering Stanford University United States

A common network inference problem, arising from real-world data constraints, is how to infer a dynamic network from its time-aggregated adjacency matrix and time-varying marginals (i.e., row and column sums). Prior approaches to this problem have repurposed the classic iterative proportional fitting (IPF) procedure, also known as Sinkhorn’s algorithm, with promising empirical results. However, the statistical foundation for using IPF has not been well understood: under what settings does IPF provide principled estimation of a dynamic network from its marginals, and how well does it estimate the network? In this work, we establish such a setting, by identifying a generative network model whose maximum likelihood estimates are recovered by IPF. Our model both reveals implicit assumptions on the use of IPF in such settings and enables new analyses, such as structure-dependent error bounds on IPF’s parameter estimates. When IPF fails to converge on sparse network data, we introduce a principled algorithm that guarantees IPF converges under minimal changes to the network structure. Finally, we conduct experiments with synthetic and real-world data, which demonstrate the practical value of our theoretical and algorithmic contributions. Copyright © 2024, The Authors. All rights reserved.

关键词：

A GNN-based Spectral Filtering Mechanism for Imbalance Classification in Network Digital Twin

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Isah, Abubakar Aliyu, Ibrahim Rashid, Sulaiman Muhammad Park, Jaehyung Hahn, Minsoo Kim, Jinsul Department of Intelligent Electronics and Computer Engineering Chonnam National University Buk-gu Gwangju61186 Korea Republic of Department of Computer Science Ahmadu Bello University Zaria810241 Nigeria Department of Computational and Data Science Astana IT University Astana Kazakhstan

Graph Neural Networks are gaining attention in Fifth-Generation (5G) core network digital twins, which are data-driven complex systems with numerous components. Analyzing these data can be challenging due to rare failure types, leading to imbalanced classification in multiclass settings. Digital twins of 5G networks increasingly employ graph classification as the main method for identifying failure types. However, the skewed distribution of failure occurrences is a major class imbalance issue that prevents effective graph data mining. Previous studies have not sufficiently tackled this complex problem. In this paper, we propose Class-Fourier Graph Neural Network (CF-GNN) introduces a class-oriented spectral filtering mechanism that ensures precise classification by estimating a unique spectral filter for each class. We employ eigenvalue and eigenvector spectral filtering to capture and adapt to variations in the minority classes, ensuring accurate class-specific feature discrimination, and adept at graph representation learning for complex local structures among neighbors in an end-to-end setting. Extensive experiments have demonstrated that the proposed CF-GNN could help with both the creation of new techniques for enhancing classifiers and the investigation of the characteristics of the multi-class imbalanced data in network digital twin Copyright © 2025, The Authors. All rights reserved.

关键词： Wiener filtering

Comparison of Machine Learning Text Classification for Intent Sentiment Analysis 7

学校读者我要写书评

暂无评论

Comparison of Machine Learning Text Classification for Inten...

7th International Conference on Information Technology, Information Systems and Electrical Engineering, ICITISEE 2023

作者： Muhammad, Abulwafa Defit, Sarjon Cahyo, Gunadi Widi Nur Universitas Putra Indonesia Yptk Padang Faculty of Computer Science Department of Informatics Technology Padang Indonesia Universitas Putra Indonesia Yptk Padang Doctoral Program in Information Technology Padang Indonesia

ISBN: (纸本)9798350382266

Tourist destination reviews on Google Maps have become a valuable point of reference for visitors seeking enjoyable spots to visit. Additionally, users can gain insight into the reasons for writing reviews. Text classification is used to determine the motivation behind these reviews. The aim of this study is to compare machine learning (ML) text classification techniques for identifying the intent behind sentiments expressed as complaints (0), suggestions (1), opinions (2), statements(3), and awards (4). The ML techniques assessed are Multinomial Naive Bayes (MNB), Support Vector Machine (SVM), and K-Nearest Neighbor (KNN). To achieve this, tourist destination review data sets, comprising 738 reviews, were collected using web scraping from the Google Maps website. Preprocessing stages, including case-folding, tokenisation, filtering and stemming, are executed prior to the data being prepped for feature extraction with TF-IDF. The employed classification models perform well with KNN achieving 1.00 accuracy on a 90:10 data split, and SVM obtaining 0.90 on the same data split. Although, the Multinomial Naive Bayes algorithm with 0.70 accuracy on a 90:10 data split is classified as fair. Comparison of methods was conducted by means of a Receiver Operating Characteristic (ROC) diagram. The testing accuracy for the SVM method was found to be 95%, MNB at 91%, and KNN at 84%. © 2023 IEEE.

关键词： Sentiment analysis

ADEdgeDrop: Adversarial Edge Dropping for Robust Graph Neural Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Zhaoliang Wu, Zhihao Sadikaj, Ylli Plant, Claudia Dai, Hong-Ning Wang, Shiping Cheung, Yiu-Ming Guo, Wenzhong College of Computer and Data Science Fuzhou University Fuzhou350116 China Department of Computer Science Hong Kong Baptist University Hong Kong Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing Fuzhou University Fuzhou350116 China Faculty of Computer Science and with the research network Data Science @ Uni Vienna University of Vienna Vienna1090 Austria

Although Graph Neural Networks (GNNs) have exhibited the powerful ability to gather graph-structured information from neighborhood nodes via various message-passing mechanisms, the performance of GNNs is limited by poor generalization and fragile robustness caused by noisy and redundant graph data. As a prominent solution, Graph Augmentation Learning (GAL) has recently received increasing attention in the literature. Among the existing GAL approaches, edge-dropping methods that randomly remove edges from a graph during training are effective techniques to improve the robustness of GNNs. However, randomly dropping edges often results in bypassing critical edges. Consequently, the effectiveness of message passing is weakened. In this paper, we propose a novel adversarial edgedropping method (ADEdgeDrop) that leverages an adversarial edge predictor guiding the removal of edges, which can be flexibly incorporated into diverse GNN backbones. Employing an adversarial training framework, the edge predictor utilizes the line graph transformed from the original graph to estimate the edges to be dropped, which improves the interpretability of the edge-dropping method. The proposed ADEdgeDrop is optimized alternately by stochastic gradient descent and projected gradient descent. Comprehensive experiments on eight graph benchmark datasets demonstrate that the proposed ADEdgeDrop outperforms state-of-the-art baselines across various GNN backbones, demonstrating improved generalization and robustness. Copyright © 2024, The Authors. All rights reserved.

关键词： Adversarial machine learning

QoE Comparison between Multi-User OFDMA and Reliable Groupcast for Wireless LAN Audiovisual Transmission

学校读者我要写书评

暂无评论

QoE Comparison between Multi-User OFDMA and Reliable Groupca...

IEEE Region 10 Symposium

作者： Daiki Deguchi Toshiro Nunome Department of Engineering Computer Science Program Graduate School of Engineering Nagoya Institute of Technology Nagoya Japan

ISBN: (数字)9798350364866

ISBN: (纸本)9798350364873

This paper compares video and audio QoE of OFDMA multi-user transmission and reliable groupcast over wireless LAN. We assume video and audio transmission to several terminals simultaneously. As a reliable groupcast method, we employ Unsolicited Retry, a technique of IEEE 802.11aa GroupCast with Retries. We utilize IEEE 802.11ax for OFDMA transmission. We evaluate application-level QoS by computer simulation. We then assess QoE through a subjective experiment with video and audio streams generated by the output timing obtained from the simulation. We notice that each method has a situation to be applied appropriately.

关键词： Degradation Wireless LAN Receivers Quality of service Streaming media Media Timing Reliability Quality of experience Streams

Climate Impact Prediction: Whale-Optimized Conv-XGBoost with Remote Sensing and Sociological data

学校读者我要写书评

暂无评论

Remote Sensing in Earth Systems sciences 2024年第4期7卷 443-456页

作者： Jayakarthik, R. Chandrashekhara K.T Sampath, O. Kumar, Deepak Biban, Laxmi Maroor, Jnaneshwar Pai Malluvalasa, S. N. Lakshmi Department of Computer Science Saveetha College of Liberal Arts and Sciences SIMATS Chennai India Department of Information Science and Engineering BMS Institute of Technology and Management Bangalore India Department of CSE RGM College of Engineering and Technology Nandyal India Department of Applied Sciences Manav Rachna International Institute of Research and Studies Faridabad India Department of Botany Pt. C.L.S. Govt. College Haryana Karnal India Department of Business Administration Nitte Institute of Professional Education Nitte (Deemed to be University) Mangaluru Karnataka India Department of Artificial Intelligence & amp Data Science Koneru Lakshmaiah Education Foundation Vaddeswaram Guntur District Andhra Pradesh India

Climate change poses significant challenges globally, impacting ecosystems, economies, and societies. Accurate prediction of climate impacts is crucial for developing effective mitigation strategies and informed policy decisions. Traditional predictive models often struggle to capture the complex interactions between environmental and socio-economic factors, necessitating advanced approaches that leverage diverse data sources and sophisticated algorithms. This study presents a robust predictive framework integrating convolutional neural networks (CNN) and XGBoost, optimized by the whale optimization algorithm (WOA). The proposed model leverages high-resolution satellite imagery and socio-economic data, combining spatial feature extraction with advanced machine learning techniques to enhance prediction accuracy and efficiency. We evaluate the model’s performance against baseline models and various optimization techniques, including particle swarm optimization (PSO) and genetic algorithm (GA). The integrated CNN and XGBoost model optimized by WOA significantly outperforms these benchmarks, achieving an accuracy of 98%, with substantial improvements in precision, recall, F1-score, and redundant phrases, and focuses directly on the core points to enhance clarity such as mean squared error (MSE), area under the curve (AUC), specificity, sensitivity, mean absolute error (MAE), and R². The results highlight the efficacy of the proposed approach in capturing complex environmental and socio-economic interactions, providing a powerful tool for climate impact assessment and decision-making. © The Author(s), under exclusive licence to Springer Nature Switzerland AG 2024.

关键词： Climate Impact Prediction CNN Convolutional Neural Networks GA Genetic Algorithm Machine Learning Model Optimization Particle Swarm Optimization PSO Spatial Feature Extraction Whale Optimization Algorithm WOA XGBoost

Recommendation fairness and where to find it: An empirical study on fairness of user recommender systems

学校读者我要写书评

暂无评论

Recommendation fairness and where to find it: An empirical s...

IEEE International Conference on Big data

作者： Antonela Tommasel Ira Assent Department of Computer Science DIGIT Aarhus University Centre for Digitalisation Big Data and Data Analytics Denmark ISISTAN CONICET-UNCPBA Aarhus University Argentina Department of Computer Science DIGIT Aarhus University Centre for Digitalisation Big Data and Data Analytics Aarhus University Denmark

Recommender systems play a crucial role in how users consume information and establish new social relations. However, different factors (such as the data collection process, the designed recommendation model, or even the interpretation of findings) could make recommenders (unintendedly) prone to biases, favouring certain user groups or items, thus resulting in unfair outcomes. Recommenders also face fairness criticism for inducing filter bubbles, echo chambers, and, more generally, facilitating opinion manipulation. In this work, we study the impact of user recommender systems on fairness. To this end, we carry out a user recommendation task on a politically polarized Twitter data collection. Then, we evaluate how the different politically aligned user groups experience recommendation quality. Finally, we explore causal models to identify data and model-related features that could affect the fairness of recommender outcomes. Our study shows that political alignment is associated with the unfairness of recommenders affecting not only the relevance of recommendations, but also their diversity and the resulting interaction patterns.

关键词：