检索结果-内蒙古大学图书馆

IEEE Transactions on Artificial Intelligence 2024年第10期5卷 5267-5276页

作者： Yang, Shuai Jiang, Tingting Dang, Qianlong Gu, Lichuan Wu, Xindong Anhui Agricultural University School of Information and Artificial Intelligence Hefei230036 China Anhui Provincial Engineering Research Center for Agricultural Information Perception and Intelligent Computing Hefei230036 China Northwest A & F University College of Science Yangling712100 China Hefei University of Technology Key Laboratory of Knowledge Engineering with Big Data The Ministry of Education of China Hefei230601 China Hefei University of Technology School of Computer Science and Information Engineering Hefei230601 China

Stable learning aims to learn a model that generalizes well to arbitrary unseen target domain by leveraging a single source domain. Recent advances in stable learning have focused on balancing the distribution of confounders for each feature to eliminate spurious correlations. However, previous studies treat all features equally without considering the difficulties of confounder balancing associated with different features, and regard irrelevant features as confounders, deteriorating generalization performance. To tackle these issues, this article proposes a novel triplex learning (TriL) based stable learning algorithm, which performs sample reweighting, causal feature selection, and representation learning to remove spurious correlations. Specifically, first, TriL adaptively assigns weights to the confounder balancing term of each feature in accordance with the difficulties of confounder balancing, and aligns the confounder distribution of each feature by learning a group of sample weights. Second, TriL integrates the sample weights into a weighted cross-entropy model to compute causal effects of features for excluding irrelevant features from the confounder set. Finally, TriL relearns a set of sample weights and uses them to guide a new supervised dual-autoencoder containing two classifiers to learn feature representations. TriL forces the results of two classifiers to remain consistent for removing spurious correlations by using a cross-classifier consistency regularization. Extensive experiments on synthetic and two real-world datasets show the superiority of TriL compared with seven methods. © 2024 IEEE.

关键词： Feature Selection

来源：评论

学校读者我要写书评

暂无评论

Medical Scene Graphs and Reasoning

Medical Scene Graphs and Reasoning

引用

2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021

作者： Cao, Chuxue He, Yiming Chen, Yuzhen Song, Chunli Ling, Hao Guan, Renchu Feng, Xiaoyue Jilin University The Key Laboratory for Symbolic Computation and Knowledge Engineering of the Ministry of Education College of Software Changchun China The Second Hospital of Jilin University Changchun China Jilin University The Key Laboratory for Symbolic Computation and Knowledge Engineering of the Ministry of Education College of Computer Science and Technology Changchun China

ISBN: (纸本)9781665401265

Medical scene graph contributes to cognitive tasks such as question answering. An automatic medical scene graph generator can annotate medical images with scene graphs conveniently. An end-to-end model is proposed, which contains an automatic medical scene graph generator based on medical images and surgery records. The novel model can extract symptom information of the images from surgery records via name entity recognition then convert it into scene graphs. The content and format of the scene graph are pre-defined according to the features of the images. A medical scene graph (MSG) dataset is also produced. The dataset contains 116 scene graphs where each graph includes 6 to 30 entities. Experiments show that our model performs well on making scene graphs of CT coronary angiography. Significantly, the scene graphs have the type, state, location, and index of arteries. To verify the practicability of the MSG dataset, We construct an intelligent model that can generate key diagnostic results. In addition, our medical scene graph generator can be applied to other medical areas to annotate different kinds of medical images intelligently. And the MSG dataset can directly be used in other cognitive visual tasks. © 2021 IEEE.

关键词： Graphic methods

来源：评论

学校读者我要写书评

暂无评论

Improving Local Search for Pseudo Boolean Optimization by Fragile Scoring Function and Deep Optimization 29

Improving Local Search for Pseudo Boolean Optimization by Fr...

引用

29th International Conference on Principles and Practice of Constraint Programming, CP 2023

作者： Zhou, Wenbo Zhao, Yujiao Wang, Yiyuan Cai, Shaowei Wang, Shimao Wang, Xinyu Yin, Minghao School of Information Science and Technology Northeast Normal University Changchun China Key Laboratory of Applied Statistics of MOE Northeast Normal University Changchun China Key Laboratory of Symbolic Computation and Knowledge Engineering of MOE Jilin University Changchun China State Key Laboratory of Computer Science Institute of Software Chinese Academy of Sciences Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China

ISBN: (纸本)9783959773003

Pseudo-Boolean optimization (PBO) is usually used to model combinatorial optimization problems, especially for some real-world applications. Despite its significant importance in both theory and applications, there are few works on using local search to solve PBO. This paper develops a novel local search framework for PBO, which has three main ideas. First, we design a two-level selection strategy to evaluate all candidate variables. Second, we propose a novel deep optimization strategy to disturb some search spaces. Third, a sampling flipping method is applied to help the algorithm jump out of local optimum. Experimental results show that the proposed algorithms outperform three state-of-the-art PBO algorithms on most instances. © Wenbo Zhou, Yujiao Zhao, Yiyuan Wang, Shaowei Cai, Shimao Wang, Xinyu Wang, and Minghao Yin.

关键词： Combinatorial optimization

来源：评论

学校读者我要写书评

暂无评论

Learning generalizable agents via saliency-guided features decorrelation 23

Learning generalizable agents via saliency-guided features d...

引用

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Sili Huang Yanchao Sun Jifeng Hu Siyuan Guo Hechang Chen Yi Chang Lichao Sun Bo Yang Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education and School of Artificial Intelligence Jilin University China Department of Computer Science University of Maryland College Park School of Artificial Intelligence Jilin University China Lehigh University Bethlehem Pennsylvania Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education

In visual-based Reinforcement Learning (RL), agents often struggle to generalize well to environmental variations in the state space that were not observed during training. The variations can arise in both task-irrelevant features, such as background noise, and task-relevant features, such as robot configurations, that are related to the optimal decisions. To achieve generalization in both situations, agents are required to accurately understand the impact of changed features on the decisions, i.e., establishing the true associations between changed features and decisions in the policy model. However, due to the inherent correlations among features in the state space, the associations between features and decisions become entangled, making it difficult for the policy to distinguish them. To this end, we propose Saliency-Guided Features Decorrelation (SGFD) to eliminate these correlations through sample reweighting. Concretely, SGFD consists of two core techniques: Random Fourier Functions (RFF) and the saliency map. RFF is utilized to estimate the complex non-linear correlations in high-dimensional images, while the saliency map is designed to identify the changed features. Under the guidance of the saliency map, SGFD employs sample reweighting to minimize the estimated correlations related to changed features, thereby achieving decorrelation in visual RL tasks. Our experimental results demonstrate that SGFD can generalize well on a wide range of test environments and significantly outperforms state-of-the-art methods in handling both task-irrelevant variations and task-relevant variations.

关键词：

来源：评论

学校读者我要写书评

暂无评论

BA-LORA: BIAS-ALLEVIATING LOW-RANK ADAPTATION TO MITIGATE CATASTROPHIC INHERITANCE IN LARGE LANGUAGE MODELS

arXiv

引用

arXiv 2024年

作者： Chang, Yupeng Chang, Yi Wu, Yuan School of Artificial Intelligence Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering Jilin University China International Center of Future Science Jilin University China

Large language models (LLMs) have demonstrated remarkable proficiency across various natural language processing (NLP) tasks. However, adapting LLMs to downstream applications requires computationally intensive and memory-demanding fine-tuning procedures. To alleviate these burdens, parameter-efficient fine-tuning (PEFT) techniques have emerged as a promising approach to tailor LLMs with minimal computational overhead. While PEFT methods offer substantial advantages, they do not fully address the pervasive issue of bias propagation from pre-training data. This work introduces Bias-Alleviating Low-Rank Adaptation (BA-LoRA), a novel PEFT method designed to counteract bias inheritance. BA-LoRA incorporates three distinct regularization terms: (1) a consistency regularizer, (2) a diversity regularizer, and (3) a singular value decomposition regularizer. These regularizers aim to enhance the models’ consistency, diversity, and generalization capabilities during fine-tuning. We conduct extensive experiments on natural language understanding (NLU) and natural language generation (NLG) tasks using prominent LLMs such as LLaMA, Mistral, and Gemma. The results demonstrate that BA-LoRA outperforms LoRA and its state-of-the-art variants. Moreover, the extended experiments demonstrate that our method effectively mitigates the adverse effects of pre-training bias, leading to more reliable and robust model outputs. The code is available at https://***/cyp-jlu-ai/BA-LoRA. Copyright © 2024, The Authors. All rights reserved.

关键词： Natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

ExplSched: Maximizing Deep Learning Cluster Efficiency for Exploratory Jobs

ExplSched: Maximizing Deep Learning Cluster Efficiency for E...

引用

IEEE International Conference on Cluster computing

作者： Hongliang Li Hairui Zhao Zhewen Xu Xiang Li Haixiao Xu College of Computer Science and Technology Jilin University Changchun China Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education Changchun China High Performance Computing Center Jilin University China

Resource management for Deep Learning (DL) clusters is essential for system efficiency and model training quality. Existing schedulers provided by DL frameworks are mostly adaptations from traditional HPC clusters and usually work on jobs’ makespan, assuming that DL training jobs finish completely. Unfortunately, it is reported that a fair amount of training jobs are exploratory jobs and often finish unsuccessfully (over 30%) in production clusters. This is due to the distinct characteristic of Deep Neural Network (DNN) training that it is an exploratory process of frequent user interventions, such as adjusting model structures, tuning hyperparameters, and exploring feature validity. Existing DL cluster schedulers using offline algorithms are not suitable for exploratory jobs when unexpected early terminations can cause noticeable resource waste. Moreover, DL training jobs are iterative and usually yield diminishing returns as they progress. Equally allocating resource among training iterations is not efficient, especially when dealing with exploratory jobs where it can worsen the degradation of system efficiency. The fundamental goal of a DL training job is to gain model quality improvement, usually indicated by the loss reduction (job profit) of a DNN model. This paper introduces a novel scheduling problem for exploratory jobs that seeks to maximize the overall training profit of a DL cluster. We propose ExplSched, an online scheduling solution based on the primal-dual framework, resulting in a competitive ratio of 2α that belongs to O(ln n). It uses a resource price function that emphasizes the importance of job profit to resource consumption ratio to make quick resource allocation decisions. Experimental results show that ExplSched achieved an average system utility improvement of 87.28% compared with other related work.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Enhancing Unsupervised Graph Few-shot Learning via Set Functions and Optimal Transport

arXiv

引用

arXiv 2025年

作者： Liu, Yonghao Giunchiglia, Fausto Li, Ximing Huang, Lan Feng, Xiaoyue Guan, Renchu College of Computer Science and Technology Jilin University Changchun China Department of Information Engineering and Computer Science University of Trento Trento Italy Key Laboratory of Symbolic Computation and Knowledge Engineering The Ministry of Education China

Graph few-shot learning has garnered significant attention for its ability to rapidly adapt to downstream tasks with limited labeled data, sparking considerable interest among researchers. Recent advancements in graph few-shot learning models have exhibited superior performance across diverse applications. Despite their successes, several limitations still exist. First, existing models in the meta-training phase predominantly focus on instance-level features within tasks, neglecting crucial set-level features essential for distinguishing between different categories. Second, these models often utilize query sets directly on classifiers trained with support sets containing only a few labeled examples, overlooking potential distribution shifts between these sets and leading to suboptimal performance. Finally, previous models typically require necessitate abundant labeled data from base classes to extract transferable knowledge, which is typically infeasible in real-world scenarios. To address these issues, we propose a novel model named STAR, which leverages Set funcTions and optimAl tRansport for enhancing unsupervised graph few-shot learning. Specifically, STAR utilizes expressive set functions to obtain set-level features in an unsupervised manner and employs optimal transport principles to align the distributions of support and query sets, thereby mitigating distribution shift effects. Theoretical analysis demonstrates that STAR can capture more task-relevant information and enhance generalization capabilities. Empirically, extensive experiments across multiple datasets validate the effectiveness of STAR. Our code can be found here. Copyright © 2025, The Authors. All rights reserved.

关键词： Zero-shot learning

来源：评论

学校读者我要写书评

暂无评论

From Incomplete Coarse-Grained to Complete Fine-Grained: A Two-Stage Framework for Spatiotemporal Data Reconstruction

arXiv

引用

arXiv 2024年

作者： Sun, Ziyu Su, Haoyang Wang, En Yang, Funing Yang, Yongjian Liu, Wenbin The College of Computer Science and Technology Jilin University Changchun130012 China The Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China

With the rapid development of various sensing devices, spatiotemporal data is becoming increasingly important nowadays. However, due to sensing costs and privacy concerns, the collected data is often incomplete and coarse-grained, limiting its application to specific tasks. To address this, we propose a new task called spatiotemporal data reconstruction, which aims to infer complete and fine-grained data from sparse and coarse-grained observations. To achieve this, we introduce a two-stage data inference framework, DiffRecon, grounded in the Denoising Diffusion Probabilistic Model (DDPM). In the first stage, we present Diffusion-C, a diffusion model augmented by ST-PointFormer, a powerful encoder designed to leverage the spatial correlations between sparse data points. Following this, the second stage introduces Diffusion-F, which incorporates the proposed T-PatternNet to capture the temporal pattern within sequential data. Together, these two stages form an end-to-end framework capable of inferring complete, fine-grained data from incomplete and coarse-grained observations. We conducted experiments on multiple real-world datasets to demonstrate the superiority of our method. © 2024, CC BY.

关键词： Spatio-temporal data

来源：评论

学校读者我要写书评

暂无评论

Reusable Generator Data-Free knowledge Distillation with Hard Loss Simulation for Image Classification

SSRN

引用

SSRN 2024年

作者： Sun, Yafeng Wang, Xingwang Huang, Junhong Chen, Shilin Hou, Minghui College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China

In many image classification scenarios where knowledge distillation (KD) is applied, multiple users need to train various student models that conform to the device's computational limitations at different times. However, due to security concerns, existing data-free KD (DFKD) methods discard the trained generator after completing a task, preventing its reuse in training subsequent students and resulting in wasted resources. Furthermore, the unlabeled nature of fake images eliminates the hard loss that plays a crucial role in data-driven KD. This paper introduces the reusable generator DFKD with hard loss simulation (RG-HLS) to address these challenges. RG-HLS employs a generator consisting of two generative networks and a decoder. The decoder and one of the generative networks can be reused in subsequent tasks. The adversarial example attack is also introduced to mimic the behavior of the teacher model in predicting real images, ensuring that DFKD exhibits a hard loss similar to that of data-driven KD. Extensive experiments across CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets validate the proposed method's superior performance. The reusable generator accelerates convergence without compromising security, and integrating the hard loss further boosts accuracy. Code is available at https://***/sunyafeng-jlu/RG-HLS. © 2024, The Authors. All rights reserved.

关键词： Image classification

来源：评论

学校读者我要写书评

暂无评论

AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models

arXiv

引用

arXiv 2023年

作者： Li, Baisong Wang, Xingwang Xu, Haixiao School of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University China

Large language models(LLMs) excellent performance across a variety of tasks, but they come with significant computational and storage costs. Quantizing these models is an effective way to alleviate this issue. However, existing methods struggle to strike a balance between model accuracy and hardware efficiency. This is where we introduce AWEQ, a post-training method that requires no additional training overhead. AWEQ excels in both ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization. There is an observation that weight quantization is less challenging than activation quantization. AWEQ transfers the difficulty of activation quantization to weights using channel equalization, achieving a balance between the quantization difficulties of both, and thereby maximizing performance. We have further refined the equalization method to mitigate quantization bias error, ensuring the robustness of the model. Extensive experiments on popular models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing post-training quantization methods for large models. Copyright © 2023, The Authors. All rights reserved.

关键词： Chemical activation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：