检索结果-内蒙古大学图书馆

Assessing the impact of differential privacy in transfer learning with deep neural networks and transformer language models

引用

Neural Computing and Applications 2025年第6期37卷 5097-5119页

作者： Sousa, Samuel Trügler, Andreas Kern, Roman Know Center Research GmbH Graz Austria Graz University of Technology Graz Austria Graz Center for Machine Learning Graz Austria University of Graz Graz Austria

The realization of trustworthy artificial intelligence strongly relies on privacy, fairness, and accountability requirements. Although model trustworthiness results from the synergy between these requirements, some effects often clash with one another and may propagate issues into downstream tasks. This paper empirically indicates that optimization algorithms with differential privacy (DP), such as differentially private stochastic gradient descent (DP-SGD) and differentially private Adam (DP-Adam), disproportionately increase differences in accuracy between groups and domains in transfer learning based on convolutional neural network (CNN) and long-short-term memory (LSTM) models. We investigate the impact of DP-SGD and DP-Adam in transfer learning models, which receive DP noise to hinder privacy risks and perform predictions on target domain data. In addition, after extensive experimental evaluation, we provide evidence that fine-tuning may reduce the differences in accuracy for most groups compared to gradient boosting and ensemble methods. However, differentially private optimizers may reverse gains in accuracy from transfer learning for CNNs and LSTMs. Unevenly reduced model accuracy may jeopardize the predictions’ fairness, causing numerous real-world implications, such as denying credit to people in need or undermining complex language processing. Identifying scenarios where privacy protection conflicts with fairness for transfer learning is a problem often overlooked yet directly aligned with the accountability requirement in trustworthy artificial intelligence. Therefore, assessing the impact of differentially private optimizers in transfer learning is the first step to developing mitigation strategies. © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024.

关键词： Differential privacy

来源：评论

学校读者我要写书评

暂无评论

Benchmarking One Class Classification in Banking, Insurance, and Cyber Security 12th

Benchmarking One Class Classification in Banking, Insurance,...

引用

12th International Conference on Frontiers of Intelligent Computing: Theory and Applications, FICTA 2024

作者： Priyanka, Chaduvula Vivek, Yelleti Ravi, Vadlamani Center for Artificial Intelligence and Machine Learning Institute for Development and Research in Banking Technology Hyderabad500076 India

ISBN: (纸本)9789819601387

Given the inherent rarity observed in imbalanced datasets, adopting one-class classification (OCC) stands out as a pragmatic approach to counteract bias toward the predominant class. This research endeavors to thoroughly assess ten state-of-the-art OCC methodologies across a spectrum of five diverse challenges within the Banking, Insurance, and Cybersecurity sectors. Moreover, we introduce an innovative unsupervised learning technique wherein (i) K-Means clustering is utilized during the training phase on negative sample data. To determine the optimal number of clusters, we utilized the Silhouette index and employed the maximum intra-class centroid distance as a cluster-specific threshold. These thresholds play a pivotal role in distinguishing between positive and negative samples. (ii) In the testing phase, a majority voting mechanism is employed to evaluate the discriminative capability of these thresholds, facilitating precise classification of test data. Empirical findings unequivocally demonstrate that our proposed approach outperforms several state-of-the-art techniques, achieving superior classification accuracy across four out of the five datasets. This underscores the effectiveness and potential applicability of our novel methodology in tackling the intricate challenges prevalent in the Banking, Insurance, and Cybersecurity sectors, particularly in the domains of fraud detection and related areas. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Banking

来源：评论

学校读者我要写书评

暂无评论

GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answering 31

GraphOTTER: Evolving LLM-based Graph Reasoning for Complex T...

引用

31st International Conference on Computational Linguistics, COLING 2025

作者： Li, Qianlong Huang, Chen Li, Shuai Xiang, Yuanxin Xiong, Deng Lei, Wenqiang Sichuan University China Stevens Institute of Technology United States Engineering Research Center of Machine Learning and Industry Intelligence Ministry of Education China

ISBN: (纸本)9798891761964

Complex Table Question Answering involves providing accurate answers to specific questions based on intricate tables that exhibit complex layouts and flexible header locations. Despite considerable progress having been made in the LLM era, the reasoning processes of existing methods are often implicit, feeding the entire table into prompts, making it difficult to effectively filter out irrelevant information in the table. To this end, we propose GraphOTTER that explicitly establishes the reasoning process to pinpoint the correct answers. In particular, GraphOTTER leverages a graph-based representation, transforming the complex table into an undirected graph. It then conducts step-by-step reasoning on the graph, with each step guided by a set of pre-defined intermediate reasoning actions. As such, it constructs a clear reasoning path and effectively identifies the answer to a given question. Comprehensive experiments on two benchmark datasets and two LLM backbones demonstrate the effectiveness of GraphOTTER. Further analysis indicates that its success may be attributed to the ability to efficiently filter out irrelevant information, thereby focusing the reasoning process on the most pertinent data. Our code and experimental datasets are available at https://***/JDing0521/GraphOTTER. © 2025 Association for Computational Linguistics.

关键词： Question answering

来源：评论

学校读者我要写书评

暂无评论

On the Role of Priors in Bayesian Causal learning

IEEE Transactions on Artificial Intelligence

引用

IEEE Transactions on Artificial Intelligence 2025年第5期6卷 1439-1445页

作者： Geiger, Bernhard C. Kern, Roman Graz University of Technology Signal Processing and Speech Communication Laboratory Graz8010 Austria Know Center Research GmbH Graz8010 Austria Graz University of Technology Institute of Machine Learning and Neural Computation Graz8010 Austria

In this work, we investigate causal learning of independent causal mechanisms (ICMs) from a Bayesian perspective. Confirming previous claims from the literature, we show in a didactically accessible manner that unlabeled data (i.e., cause realizations) do not improve the estimation of the parameters defining the mechanism. Furthermore, we observe the importance of choosing an appropriate prior for the cause and mechanism parameters, respectively. Specifically, we show that a factorized prior results in a factorized posterior, which resonates with Janzing and Schölkopf's definition of ICMs via the Kolmogorov complexity of the involved distributions and with the concept of parameter independence of Heckerman et al. © 2020 IEEE.

关键词： Contrastive learning

来源：评论

学校读者我要写书评

暂无评论

INTEGRATING DYNAMICAL SYSTEMS MODELING WITH SPATIOTEMPORAL SCRNA-SEQ DATA ANALYSIS

arXiv

引用

arXiv 2025年

作者： Zhang, Zhenyi Sun, Yuhao Peng, Qiangwei Li, Tiejun Zhou, Peijie LMAM and School of Mathematical Sciences Peking University China Center for Machine Learning Research Peking University China LMAM and School of Mathematical Sciences Center for Machine Learning Research Peking University China

Understanding the dynamic nature of biological systems is fundamental to deciphering cellular behavior, developmental processes, and disease progression. Single-cell RNA sequencing (scRNA-seq) has provided static snapshots of gene expression, offering valuable insights into cellular states at a single time point. Recent advancements in temporally resolved scRNA-seq, spatial transcriptomics (ST), and time-series spatial transcriptomics (temporal-ST) have further revolutionized our ability to study the spatiotemporal dynamics of individual cells. These technologies, when combined with computational frameworks such as Markov chains, stochastic differential equations (SDEs), and generative models like optimal transport and Schrödinger bridges, enable the reconstruction of dynamic cellular trajectories and cell fate decisions. This review discusses how these dynamical system approaches offer new opportunities to model and infer cellular dynamics from a systematic perspective. © 2025, CC BY.

关键词： Markov chains

来源：评论

学校读者我要写书评

暂无评论

Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts

arXiv

引用

arXiv 2025年

作者： Huang, Youcheng Huang, Chen Feng, Duanyu Lei, Wenqiang Lv, Jiancheng Sichuan University China Engineering Research Center of Machine Learning and Industry Intelligence Ministry of Education China

Understanding the inner workings of Large Language Models (LLMs) is a critical research frontier. Prior work has shown that a single LLM's concept representations can be captured as steering vectors (SVs), enabling the control of LLM behavior (e.g., towards generating harmful content). This paper takes a novel approach by exploring the intricate relationships between representations of concepts across different LLMs, drawing an intriguing parallel to the Plato's Allegory of the Cave. In particular, we introduce a linear transformation method to bridge these representations and present three key findings: 1) The representations of a same concept in different LLMs can be effectively aligned using simple linear transformations, enabling efficient cross-model transfer and behavioral control via SVs. 2) This linear transformation generalizes across multiple concepts, facilitating alignment and control of SVs representing different concepts across LLMs. 3) A weak-to-strong transferability exists between LLMs, whereby SVs extracted from smaller LLMs can effectively control behaviors of larger LLMs. Copyright © 2025, The Authors. All rights reserved.

关键词： Linear transformations

来源：评论

学校读者我要写书评

暂无评论

MoColl: Agent-Based Specific and General Model Collaboration for Image Captioning

arXiv

引用

arXiv 2025年

作者： Yang, Pu Dong, Bin The School of Mathematical Sciences Peking University China Beijing International Center for Mathematical Research Center for Machine Learning Research Peking University China

Image captioning is a critical task at the intersection of computer vision and natural language processing, with wide-ranging applications across various domains. For complex tasks such as diagnostic report generation, deep learning models require not only domain-specific image-caption datasets but also the incorporation of relevant general knowledge to provide contextual accuracy. Existing approaches exhibit inherent limitations: specialized models excel in capturing domain-specific details but lack generalization, while vision-language models (VLMs) built on large language models (LLMs) leverage general knowledge but struggle with domain-specific adaptation. To address these limitations, this paper proposes a novel agent-enhanced model collaboration framework, which we called MoColl, designed to effectively integrate domain-specific and general knowledge. Specifically, our approach is to decompose complex image captioning tasks into a series of interconnected question-answer subtasks. A trainable visual question answering (VQA) model is employed as a specialized tool to focus on domain-specific visual analysis, answering task-specific questions based on image content. Concurrently, an LLM-based agent with general knowledge formulates these questions and synthesizes the resulting question-answer pairs into coherent captions. Beyond its role in leveraging the VQA model, the agent further guides its training to enhance its domain-specific capabilities. Experimental results on radiology report generation validate the effectiveness of the proposed framework, demonstrating significant improvements in the quality of generated reports. Copyright © 2025, The Authors. All rights reserved.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection

arXiv

引用

arXiv 2025年

作者： Bi, Jinhe Wang, Yifan Yan, Danqi Xiao, Xun Hecker, Artur Tresp, Volker Ma, Yunpu Ludwig Maximilian University of Munich Germany Munich Research Center Huawei Technologies Germany Munich Center for Machine Learning Germany

Visual instruction tuning refines pre-trained Multimodal Large Language Models (MLLMs) to enhance their real-world task performance. However, the rapid expansion of visual instruction datasets introduces significant data redundancy, leading to excessive computational costs. Existing data selection methods predominantly rely on proxy models or loss-based metrics, both of which impose substantial computational overheads due to the necessity of model inference and backpropagation. To address this challenge, we propose PRISM, a novel training-free approach for efficient multimodal data selection. Unlike existing methods, PRISM eliminates the reliance on proxy models, warm-up pretraining, and gradient-based optimization. Instead, it leverages Pearson correlation analysis to quantify the intrinsic visual encoding properties of MLLMs, computing a task-specific correlation score to identify high-value instances. This not only enables data-efficient selection, but maintains the model’s original performance. Empirical evaluations across multiple MLLMs demonstrate that PRISM reduces the overall time required for visual instruction tuning and data selection to just 30% of conventional methods, while surpassing fully fine-tuned models across eight multimodal and three language understanding benchmarks, achieving a 101.7% relative improvement in final performance. Copyright © 2025, The Authors. All rights reserved.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

Channel Dependence, Limited Lookback Windows, and the Simplicity of Datasets: How Biased is Time Series Forecasting?

arXiv

引用

arXiv 2025年

作者： Abdelmalak, Ibram Madhusudhanan, Kiran Choi, Jungmin Stubbemann, Maximilian Schmidt-Thieme, Lars Information Science and Machine Learning Lab VWFS Data Analytics Research Center University of Hildesheim Niedersachsen Hildesheim Germany

Time-series forecasting research has converged to a small set of datasets and a standardized collection of evaluation scenarios. Such a standardization is to a specific extent needed for comparable research. However, the underlying assumption is, that the considered setting is a representative for the problem as a whole. In this paper, we challenge this assumption and show that the current scenario gives a strongly biased perspective on the state of time-series forecasting research. To be more detailed, we show that the current evaluation scenario is heavily biased by the simplicity of the current datasets. We furthermore emphasize, that when the lookback-window is properly tuned, current models usually do not need any information flow across channels. However, when using more complex benchmark data, the situation changes: Here, modeling channel-interactions in a sophisticated manner indeed enhances performances. Furthermore, in this complex evaluation scenario, Crossformer, a method regularly neglected as an important baseline, is the SOTA method for time series forecasting. Based on this, we present the Fast Channel-dependent Transformer (FaCT), a simplified version of Crossformer which closes the runtime gap between Crossformer and TimeMixer, leading to an efficient model for complex forecasting datasets. © 2025, CC BY.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Acceleration Algorithms in GNNs: A Survey

引用

IEEE Transactions on Knowledge and Data Engineering 2025年第6期37卷 3173-3192页

作者： Ma, Lu Sheng, Zeang Li, Xunkai Gao, Xinyi Hao, Zhezheng Yang, Ling Nie, Xiaonan Jiang, Jiawei Zhang, Wentao Cui, Bin Peking University School of Cs Beijing100871 China Beijing Institute of Technology School of Cs Beijing100811 China University of Queensland BrisbaneQLD4072 Australia Northwestern Polytechnical University School of Ai Xi'an710060 China Wuhan University School of Computer Science Wuhan430072 China Peking University Center for Machine Learning Research Beijing100871 China

Graph Neural Networks have demonstrated remarkable effectiveness in various graph-based tasks, but their inefficiency in training and inference poses significant challenges for scaling to real-world, large-scale applications. To address these challenges, a plethora of algorithms have been developed to accelerate GNN training and inference, garnering substantial interest from the research community. This paper presents a systematic review of these acceleration algorithms, categorizing them into three main topics: training acceleration, inference acceleration, and execution acceleration. For training acceleration, we discuss techniques like graph sampling and GNN simplification. In inference acceleration, we focus on knowledge distillation, GNN quantization, and GNN pruning. For execution acceleration, we explore GNN binarization and graph condensation. Additionally, we review several libraries related to GNN acceleration, including our Scalable Graph learning library, and propose future research directions. © 1989-2012 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：