检索结果-内蒙古大学图书馆

DEEP GENERATIVE MODELING ON LIMITED data WITH REGULARIZATION BY NONTRANSFERABLE PRE-TRAINED MODELS

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhong, Yong Liu, Hongtao Liu, Xiaodong Bao, Fan Shen, Weiran Li, Chongxuan Gaoling School of AI Renmin University of China Beijing China Beijing Key Lab of Big Data Management and Analysis Methods Beijing China Department of Computer Science Technology Tsinghua University Beijing China

Deep generative models (DGMs) are data-eager because learning a complex model on limited data suffers from a large variance and easily overfits. Inspired by the classical perspective of the bias-variance tradeoff, we propose regularized deep generative model (Reg-DGM), which leverages a nontransferable pre-trained model to reduce the variance of generative modeling with limited data. Formally, Reg-DGM optimizes a weighted sum of a certain divergence and the expectation of an energy function, where the divergence is between the data and the model distributions, and the energy function is defined by the pre-trained model w.r.t. the model distribution. We analyze a simple yet representative Gaussian-fitting case to demonstrate how the weighting hyperparameter trades off the bias and the variance. Theoretically, we characterize the existence and the uniqueness of the global minimum of Reg-DGM in a non-parametric setting and prove its convergence with neural networks trained by gradient-based methods. Empirically, with various pre-trained feature extractors and a data-dependent energy function, Reg-DGM consistently improves the generation performance of strong DGMs with limited data and achieves competitive results to the state-of-the-art methods. Our implementation is available at https://***/ML-GSAI/Reg-ADA-APA. Copyright © 2022, The Authors. All rights reserved.

关键词： Machine learning

Has “Intelligent Manufacturing” Promoted the Productivity of Manufacturing Sector?--Evidence from China’s Listed Firms

学校读者我要写书评

暂无评论

Procedia computer science 2018年 139卷 299-305页

作者： Yi Qu Yong Shi Kun Guo Yuanchun Zheng School of Economics and Management University of Chinese Academy of Sciences Beijing 100190 China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing 100190 China Research Center on Fictitious Economy & Data Science Chinese Academy of Sciences Beijing 100190 China College of Information Science and Technology University of Nebraska at Omaha NE 68182 USA School of Computing and Control Engineering University of Chinese Academy of Sciences Beijing 100190 China

Intelligent Manufacturing has attracted global and continuous attention recent years, with more and more intelligent devices and systems applied in production. In this paper, we take China’s manufacturing listed firms to investigate the productivity difference between intelligent and general manufacturing firms. By the Cobb-Douglas production function, we built a Coefficient-varying Model and used Seemingly Unrelated Regression (SUR) to estimate the time-varying trend of productivity from 2011 to 2017. The empirical results show that “Intelligent Manufacturing” has obviously promoted the labor factor utilization efficiency and the Total Factor Productivity (TFP) through the advances in technology. But it doesn’t have universally enhancing effect of all industries. The impact of “Intelligent Manufacturing” still remains to be observed overtime.

关键词： Intelligent Manufacturing Cobb-Douglas Production Function Total Factor Productivity (TFP) Coefficient-varying Model Seemingly Unrelated Regression (SUR)

Dist-PU: Positive-Unlabeled Learning from a label Distribution Perspective

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Yunrui Xu, Qianqian Jiang, Yangbangyan Wen, Peisong Huang, Qingming School of Computer Science and Technology University of Chinese Academy of Sciences China Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS China State Key Laboratory of Information Security Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management University of Chinese Academy of Sciences China

Positive-Unlabeled (PU) learning tries to learn binary classifiers from a few labeled positive examples with many unlabeled ones. Compared with ordinary semi-supervised learning, this task is much more challenging due to the absence of any known negative labels. While existing cost-sensitive-based methods have achieved state-of-the-art performances, they explicitly minimize the risk of classifying unlabeled data as negative samples, which might result in a negative-prediction preference of the classifier. To alleviate this issue, we resort to a label distribution perspective for PU learning in this paper. Noticing that the label distribution of unlabeled data is fixed when the class prior is known, it can be naturally used as learning supervision for the model. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions, which is formulated by aligning their expectations. Moreover, we further adopt the entropy minimization and Mixup regularization to avoid the trivial solution of the label distribution consistency on unlabeled data and mitigate the consequent confirmation bias. Experiments on three benchmark datasets validate the effectiveness of the proposed method. Code available at: https://***/Ray-rui/Dist-PU-Positive-UnlabeledLearning-from-a-label-Distribution-Perspective. Copyright © 2022, The Authors. All rights reserved.

关键词： Entropy

Exact single-source SimRank computation on large graphs

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Hanzhi Wei, Zhewei Yuan, Ye Du, Xiaoyong Wen, Ji-Rong School of Information Renmin University of China Gaoling School of Artificial Intelligence Renmin University of China School of Computer Science and technology Beijing Institute of Technology MOE Key Lab DEKE Renmin University of China Beijing Key Lab of Big Data Management and Analysis Method Renmin University of China

SimRank is a popular measurement for evaluating the node-to-node similarities based on the graph topology. In recent years, single-source and top-k SimRank queries have received increasing attention due to their applications in web mining, social network analysis, and spam detection. However, a fundamental obstacle in studying SimRank has been the lack of ground truths. The only exact algorithm, Power Method, is computationally infeasible on graphs with more than 106 nodes. Consequently, no existing work has evaluated the actual trade-offs between query time and accuracy on large real-world graphs. In this paper, we present ExactSim, the first algorithm that computes the exact single-source and top-k SimRank results on large graphs. With high probability, this algorithm produces ground truths with a rigorous theoretical guarantee. We conduct extensive experiments on real-world datasets to demonstrate the efficiency of ExactSim. The results show that ExactSim provides the ground truth for any single-source SimRank query with a precision up to 7 decimal places within a reasonable query time. Copyright © 2020, The Authors. All rights reserved.

关键词： Graphic methods

Estimating Noisy Class Posterior with Part-level labels for Noisy label Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhao, Rui Shi, Bin Ruan, Jianfei Pan, Tianze Dong, Bo School of Computer Science and Technology Xi’an Jiaotong University China Shaanxi Province Key Lab of Big Data Knowledge Engineering Xi’an Jiaotong University China School of Physics Xi’an Jiaotong University China School of Distance Education Xi’an Jiaotong University China

In noisy label learning, estimating noisy class posteriors plays a fundamental role for developing consistent classifiers, as it forms the basis for estimating clean class posteriors and the transition matrix. Existing methods typically learn noisy class posteriors by training a classification model with noisy labels. However, when labels are incorrect, these models may be misled to overemphasize the feature parts that do not reflect the instance characteristics, resulting in significant errors in estimating noisy class posteriors. To address this issue, this paper proposes to augment the supervised information with part-level labels, encouraging the model to focus on and integrate richer information from various parts. Specifically, our method first partitions features into distinct parts by cropping instances, yielding part-level labels associated with these various parts. Subsequently, we introduce a novel single-to-multiple transition matrix to model the relationship between the noisy and part-level labels, which incorporates part-level labels into a classifier-consistent framework. Utilizing this framework with part-level labels, we can learn the noisy class posteriors more precisely by guiding the model to integrate information from various parts, ultimately improving the classification performance. Our method is theoretically sound, while experiments show that it is empirically effective in synthetic and real-world noisy benchmarks. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Finding Patterns of Stock Returns Based on Sequence Alignment

学校读者我要写书评

暂无评论

Procedia computer science 2018年 139卷 256-262页

作者： Yong Shi Ye-ran Tang Wen Long School of Economics & Management University of Chinese Academy of Sciences Beijing 100190 P.R. China Research Center on Fictitious Economy & Data Science Chinese Academy of Sciences Beijing 100190 P.R. China Key Laboratory of Big Data Mining & Knowledge Management Chinese Academy of Sciences Beijing 100190 P.R. China College of Information Science and Technology University of Nebraska at Omaha Omaha NE 68182 USA

In this paper, we propose the method based on sequence alignment to find patterns of stock returns. We use 5 minutes high frequency data of CSI 300 index to test this method, and find we can predict the sharply rise or drop for the stock returns according to patterns of the sample symbol sequence. The empirical analysis suggests it is possible to find and predict patterns in stock returns based on the method of sequence alignment.

关键词： finding patterns sequence alignment stock returns

The neural network methods for solving Traveling Salesman Problem

学校读者我要写书评

暂无评论

Procedia computer science 2022年 199卷 681-686页

作者： Yong Shi Yuanying Zhang School of Economics and Management University of Chinese Academy of Sciences Beijing 100190 China Research Center on Fictitious Economy and Data Science Chinese Academy of Sciences Beijing 100190 China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing 100190 China College of Information Sciences and Technology University of Nebraska at Omaha NE 68182 USA School of Mathematical Sciences University of Chinese Academy of Sciences Beijing 100190 China

Traveling Salesman Problem(TSP) is a main attention issue at present. Neural network can be used to solve combinatorial optimization problems. In recent years, there have existed many neural network methods for solving TSP, which has made a big step forward for solving combinatorial optimization problems. This paper reviews the neural network methods for solving TSP in recent years, including Hopfield neural network, graph neural network and neural network with reinforcement learning. Using neural network to solve TSP can effectively improve the accuracy of the approximate solution. Finally, we put forward the prospect of solving TSP in the future.

关键词： Traveling Salesman Problem Neural network Hopfield neural network Graph neural network Reinforcement learning

MULTIPLE HEADS ARE BETTER THAN ONE: MIXTURE OF MODALITY knowledge EXPERTS FOR ENTITY REPRESENTATION LEARNING

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yichi Chen, Zhuo Guo, Lingbing Xu, Yajing Hu, Binbin Liu, Ziqi Zhang, Wen Chen, Huajun College of Computer Science and Technology Zhejiang University China ZJU-Ant Group Joint Lab of Knowledge Graph China Ant Group China School of Software Technology Zhejiang University China Zhejiang Key Laboratory of Big Data Intelligent Computing China

Learning high-quality multi-modal entity representations is an important goal of multi-modal knowledge graph (MMKG) representation learning, which can enhance reasoning tasks within the MMKGs, such as MMKG completion (MMKGC). The main challenge is to collaboratively model the structural information concealed in massive triples and the multi-modal features of the entities. Existing methods focus on crafting elegant entity-wise multi-modal fusion strategies, yet they overlook the utilization of multi-perspective features concealed within the modalities under diverse relational contexts. To address this issue, we introduce a novel framework with Mixture of Modality knowledge experts (MOMOK for short) to learn adaptive multi-modal entity representations for better MMKGC. We design relation-guided modality knowledge experts to acquire relation-aware modality embeddings and integrate the predictions from multi-modalities to achieve joint decisions. Additionally, we disentangle the experts by minimizing their mutual information. Experiments on four public MMKG benchmarks demonstrate the outstanding performance of MOMOK under complex scenarios. Our code and data are available at https://***/zjukg/MoMoK. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation

学校读者我要写书评

暂无评论

arXiv 2024年

Multi-modal knowledge graph completion (MMKGC) aims to discover unobserved knowledge from given knowledge graphs, collaboratively leveraging structural information from the triples and multi-modal information of the entities to overcome the inherent incompleteness. Existing MMKGC methods usually extract multi-modal features with pre-trained models, resulting in coarse handling of multi-modal entity information, overlooking the nuanced, fine-grained semantic details and their complex interactions. To tackle this shortfall, we introduce a novel framework MYGO to tokenize, fuse, and augment the fine-grained multi-modal representations of entities and enhance the MMKGC performance. Motivated by the tokenization technology, MYGO tokenizes multi-modal entity information as fine-grained discrete tokens and learns entity representations with a cross-modal entity encoder. To further augment the multi-modal representations, MYGO incorporates fine-grained contrastive learning to highlight the specificity of the entity representations. Experiments on standard MMKGC benchmarks reveal that our method surpasses 19 of the latest models, underlining its superior performance. Code and data can be found in https://***/zjukg/MyGO. Copyright © 2024, The Authors. All rights reserved.

关键词： knowledge graph